While most in Trudeau camps have brushed aside Trump’s comments, Mark Carney recently said the 'joke' has gone 'too far'