The reality TV star opened up about his grandad's sad death at the age of 92