A handful of people rejected by public trying to control Parliament by creating disorder: PM Modi