If we leave the demolition to the administration, we might not be left with anything, residents near the newly found temple say.