Spencer Matthews ‘upset’ that Vogue Williams was forced to address their marriage