The British star let his frustration get to him as the match slipped out of his grasp