The Albanese government is being accused of turning its back on migrants as it seeks to push through parliament controversial legislation.