The three brutalist buildings on the High Street have been empty for years