PM Modi said since 2014 attempts were made by outside players to create a controversy or unrest just ahead of the session