He said there could be reasons that are separate from your diet