‘I could not wrap my head around saying something so horrific,’ actor said during reunion