Fans of the ITV show all pointed out the same issue