The world champion has openly admitted he struggles to get up for events without fans