A councillor said that it has been an eyesore for two years since people were moved out and they thought something would happen sooner