The singer has been replaced by a Love Island star