Loose Men star Jeff Brazier has admitted that he's found it "really difficult" to come to terms with the fact that he can't shield his two sons from everything