A man has been charged with displaying a Nazi symbol as alleged neo-Nazis gathered just a few blocks from an anti-Australia Day rally in Adelaide.