There were dozens of objections to the original application, but one person said "families have nowhere to go"