Ian Diamond defends agency’s handling of deep-seated problems with key labour force survey