Ward councillors have lashed out at the plans