The 35-unit residential project was first green-lit by Victoria council in 2011, and some councillors say it’s too small to be approved today