Darlington has been branded a "ghost town" as residents react to plans to increase parking charges in the town.