Holly Willoughby 's relationship with Phillip Schofield broke down irretrievably after she discovered what his brother had told him about engaging in a sex act with a boy.