The former Premier League football withdrew after derogatory comments