Unions pledge to fight on as public servants return to office for at least three days a week.