Cat Deeley and Ben Shephard were brought in after Phillip Schofield and Holly Willoughby left