The veteran gardener, 75, was slammed by viewers last month for his 'Secret Garden' transformation but he insists he is 'very proud' of the work he had done