Pennyghael House was on the market for £5m just four years ago