Walliams and Cowell are allegedly ‘no longer on speaking terms’