The remaining celebrities and their professional dance partners have been spending time in the seaside town