Many Indigenous public servants work in just three departments—a trend that is 'deeply concerning'