The numbers have been on the decline since December, around the same time that President Donald Trump's rhetoric about Canada ramped up