The actor admitted that he was ‘in bits’ and ‘nearly in tears’ over it, feeling like he had let everyone else down having ‘dropped the ball’