Concerns have been raised that a proposed care home for children on Sheringham Road will be too small.