‘You have no control over what you’re filming,’ the actor said