Connor O'Brien said the Irish actor undoubtedly would have had to been strict on himself