The prime minister said that sitting in the stands would cost the taxpayer a ‘fortune’ in policing