Fans were devastated as Tom finally left the show in episode two