The castle is found in the middle of the city