The former security minister appeared to swipe at his opponents who have been more outwardly hawkish on migration.