A neglected former children’s home will be turned into flats despite dozens of objections to a planning application.