Sir Keir Starmer must not accept further EU influence as he seeks closer ties with Britain’s nearest neighbours, the Conservatives have said.