The mill-keeper's house was also once set alight