add explicit trust_remote_code controls to resolve the security issue

[lvhan028]

No description provided.

[Copilot]

Pull request overview

This PR introduces an explicit trust_remote_code flag (default False) and threads it through LMDeploy’s pipeline/serving/engine and VL model-loading paths to address a security concern around implicitly trusting Hugging Face remote code.

Changes:

• Add trust_remote_code parameters across the public entrypoints (API, pipeline, OpenAI server, engines) and propagate to HF loaders (AutoConfig, AutoTokenizer, AutoProcessor, AutoModel*, GenerationConfig).
• Update VL model implementations to use self.trust_remote_code instead of hardcoding trust_remote_code=True.
• Expose a CLI switch --trust-remote-code for lmdeploy serve api_server.

Reviewed changes

Copilot reviewed 29 out of 29 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
lmdeploy/vl/model/xcomposer2.py	Add trust_remote_code plumbing; stop hardcoding remote-code trust in HF model construction.
lmdeploy/vl/model/qwen3.py	Use self.trust_remote_code for AutoProcessor loading.
lmdeploy/vl/model/qwen.py	Use self.trust_remote_code for AutoModelForCausalLM.from_config.
lmdeploy/vl/model/phi3_vision.py	Use self.trust_remote_code for processor/model loading.
lmdeploy/vl/model/molmo.py	Use self.trust_remote_code for processor/model loading.
lmdeploy/vl/model/minicpmv.py	Add trust_remote_code to ctor and propagate to HF loaders.
lmdeploy/vl/model/llava.py	Use self.trust_remote_code for HF model construction.
lmdeploy/vl/model/llava_hf.py	Use self.trust_remote_code for AutoProcessor loading.
lmdeploy/vl/model/internvl3_hf.py	Add trust_remote_code to ctor; propagate into HF processors/models.
lmdeploy/vl/model/internvl.py	Add trust_remote_code to ctor; propagate into tokenizer/model loading.
lmdeploy/vl/model/internvl_llava.py	Use self.trust_remote_code for HF model construction.
lmdeploy/vl/model/glm4_v.py	Use self.trust_remote_code for HF model loading.
lmdeploy/vl/model/cogvlm.py	Use self.trust_remote_code for HF model loading.
lmdeploy/vl/model/builder.py	Add trust_remote_code to load_vl_model and pass through to arch detection and model ctor.
lmdeploy/vl/model/base.py	Store trust_remote_code on the base vision model; use it for config/tokenizer loading.
lmdeploy/vl/engine.py	Thread trust_remote_code into ImageEncoder and VL model loading.
lmdeploy/utils.py	Add trust_remote_code to HF GenerationConfig loading.
lmdeploy/tokenizer.py	Add trust_remote_code to tokenizer construction and config checks.
lmdeploy/serve/openai/api_server.py	Add trust_remote_code option and pass it into task selection/engine init.
lmdeploy/serve/core/vl_async_engine.py	Thread trust_remote_code into ImageEncoder and base async engine.
lmdeploy/serve/core/async_engine.py	Thread trust_remote_code through tokenizer/config loading and engine builders.
lmdeploy/pytorch/engine/executor/init.py	Pass trust_remote_code into ModelConfig.from_pretrained.
lmdeploy/pytorch/engine/engine.py	Default trust_remote_code to False and propagate into executor/specdecode config.
lmdeploy/pytorch/engine/config_builder.py	Thread trust_remote_code into speculative decode config creation.
lmdeploy/pytorch/config.py	Default trust_remote_code to False for model config loading; propagate for draft/spec configs.
lmdeploy/pipeline.py	Add user-facing trust_remote_code parameter and propagate into engine selection/construction.
lmdeploy/cli/serve.py	Add --trust-remote-code CLI flag and pass through to server startup.
lmdeploy/archs.py	Add trust_remote_code option to get_task/get_model_arch and propagate into AutoConfig.
lmdeploy/api.py	Expose trust_remote_code in the public pipeline() API wrapper.

Comments suppressed due to low confidence (1)

lmdeploy/vl/model/builder.py:76

• load_vl_model now passes trust_remote_code into every vision model constructor (model = module(**kwargs)), but at least one registered model (e.g. Gemma3VisionModel in lmdeploy/vl/model/gemma3_vl.py) still defines __init__(..., backend: str='') and will raise TypeError: __init__() got an unexpected keyword argument 'trust_remote_code'.

To avoid runtime failures, either update all VisionModel subclasses with custom __init__ to accept/forward trust_remote_code (or **kwargs), or make load_vl_model only pass trust_remote_code when the target class accepts it (similar to _accepts_arg used elsewhere).

    _, hf_config = get_model_arch(model_path, trust_remote_code=trust_remote_code)
    kwargs = dict(model_path=model_path,
                  with_llm=with_llm,
                  max_memory=max_memory,
                  hf_config=hf_config,
                  backend=backend,
                  trust_remote_code=trust_remote_code)

    for name, module in VISION_MODELS.module_dict.items():
        try:
            if module.match(hf_config):
                logger.info(f'matching vision model: {name}')
                model = module(**kwargs)
                model.build_preprocessor()

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

trust_remote_code is now a user-facing switch that changes how configs/tokenizers/models are loaded (defaulting to False). There are existing integration tests for pipeline() behavior (e.g. tests/test_lmdeploy/test_pipeline.py), but the new parameter and its default are not exercised here.

Please add/adjust tests to cover both: (1) default trust_remote_code=False behavior, and (2) successful loading when trust_remote_code=True is explicitly set for models that require remote code. This helps prevent regressions and ensures the security control remains effective.