The TV star described himself as a "broken man"