Viewing figures plummeted after Joe Scarborough and Mika Brzezinski revealed they had met the president-elect