Two prominent town centre buildings will be left empty