He sent a message to fans as he was seen returning to the UK after leaving for his latest trip with Rob Rinder