The former cabinet member has been critical of Labour in recent months, sparking her resignation from the party and the council