Prince Harry will attend the remainder of the Invictus Games alone as his wife heads back to Montecito, California.