Peter Dutton says federal public servants will need to return to the office full-time if the Coalition wins. Experts warn it will lead to an exodus of skilled women.