A proposal to turn a large family house in Blackburn to a residential home for five adults with complex learning disabilities has been turned down.