Rob said he was determined to get his revenge after being annoyed at the travel restrictions