Residents have argued a 'key asset could be erased'