The comedian and BBC quiz show host has spoken honestly about his experience