The former factory near the town centre dates back to the Victorian era