Comic Relief’s charity fundraiser Red Nose Day lost a million TV viewers compared with last year.