Where do Hollywood stars go to let their guard down? Novelist AM Homes celebrates the castle on the hill