The Duke of Sussex’s US visa application should remain private despite him admitting taking drugs in his memoir, a judge has ruled.