The Congress leader said Speaker Om Birla made remarks about him but did not give him an opportunity to speak