A city centre factory left empty for half a century could finally be demolished to make way for new flats.