‘I never liked the way women were,’ actor said