The royal residence has been the London home to many members of the Royal Family - including Queen Victoria and Diana, Princess of Wales - and has an unenviable reputation