The federal opposition says it plans to force public servants back to the office full time, should it win the next election.