Prince Harry is halfway through his beloved Invictus Games, which are being hosted in Canada, and now he is flying solo after Meghan Markle left the event to fly back home to California