The site has been described by many as the heart of the popular town