Matt Baker has decided to put his Channel 4 show Travels With Mum and Dad on hold for an emotional reason