Novak Djokovic said the pursuit of Olympic gold left him without much in the tank for his US Open title defence.