Residents have branded Swindon’s Christmas market “depressing” amid fears that traders have struggled.