Country star Wallen also stirred controversy with his ‘abrupt’ exit from show