ITV co-host explained reason for his frustration in aftershow admission