Sir Keir Starmer has been told by Commonwealth leaders he must come to the table to discuss reparations for the “ill effects” of slavery.