The government said the site has potential to deliver up to 4,000 homes