A former royal butler has spoken out about Prince Harry's ongoing fears for his wife and their two children.