Sharon and Jeremy failed to be won over on the BBC show