The future king first debuted his facial hair last summer