The star appeared on Loose Men as he spoke candidly about being a single dad to two young men