Sean 'Diddy' Combs repeatedly invited Prince William and Prince Harry to attend his famous Hollywood parties, but according to author Christopher Andersen the duo were wise to do so