Meghan Markle, the Duchess of Sussex, had been with Prince Harry, the Duke of Sussex, for the beginning of the Invictus Games in and near Vancouver, Canada, before flying home