Conservative politicians criticised money being spent on "vanity projects" in the city at a meeting