Jacob Rees-Mogg admitted the Conservatives failed on immigration in a public apology - but refused to blame Brexit.