A minister has ruled out the UK joining a pan-European agreement to bolster post-Brexit trade.