Councillors claimed people leave the city when the event is on