The Queen left a lasting impression after well over a dozen visits to the area