Numbers show a decline in wake of tariff threats and 51st state rhetoric, but weather may have role