The Sinnerman was angered by the comment from one contestant on The Chase