Prince Harry has been told to "pull himself together" amid reports that he will spend Christmas in California instead of with the royal family.