Top Liberal has faced scrutiny for his French skills, particularly during party's leadership race