Viewers "broke a record" with how they recognised the eliminated character was a famous face