The 78-year-old celebrity chef said it would "affect me terribly" if he found himself in a similar situation to the MasterChef presenter