Bricks tumbled from the derelict old Hat Factory during the storm