But rather than return to the office, some have quit