The remains of the castle are now only visible from the air