Adam Frost, who presents Gardeners' World, has opened up about his personal life, including a surprising revelation about his childhood