Author Tony Schwartz and former House Speaker Nancy Pelosi seem to agree that the former president has ‘lost a step or two’