A Conservative peer has been slammed after he made an antisemitic jibe when debating a new Holocaust memorial in London.