The gardening guru said he was stunned when he read the estate agent particulars for a house he bought in 1981