It's feared the 'exceptional' property could collapse into the River Ceiriog