He also vowed to "keep men out of women's sports"