Hay-on-Wye is known throughout the world as the 'Town of Books' and stages an annual festival, but I discovered another side to this charming place