The actor and singer addressed the controversy at the Santa Barbara International Film Festival on Sunday