Pat McAfee said that the reaction from the Canadian fans ‘kinda sucks’