The judge described how the town used to have the nickname 'suffer in boredom' but is now a 'fun' spot