The gardening expert has been married to wife Sarah for over 40 years after they met at university