Controversial rapper said he felt like he was experiencing a ‘setback’