The highly-contested plea deal has taken its toll on the families of victims