The judge spoke out after a suggestion was made about him being 'replaced' on the hit dance show