The former health secretary opened up about the row for the first time