The One Show legend revealed that she previously viewed the former Great British Bake Off judge's home, but one specific aspect meant that she and her husband didn't end up buying it