From an 1840s former hunting lodge to a red-brick almshouse