Worried councillors are urging Labour to reconsider the proposal