Skip to content

Commit 4e2616b

Browse files
Add OrcaRouter as an LLM gateway option for PP-ChatOCRv4 and PP-DocTranslation
Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: jinhao.song <jinhao.song@myflashcloud.com>
1 parent 2661c7c commit 4e2616b

6 files changed

Lines changed: 77 additions & 2 deletions

File tree

‎docs/version3.x/pipeline_usage/PP-ChatOCRv4.en.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -974,6 +974,8 @@ Please note: If you encounter issues such as the program becoming unresponsive,
974974

975975
Before performing model inference, you first need to prepare the API key for the large language model. PP-ChatOCRv4 supports large model services on the [Baidu Cloud Qianfan Platform](https://console.bce.baidu.com/qianfan/ais/console/onlineService) or the locally deployed standard OpenAI interface. If using the Baidu Cloud Qianfan Platform, refer to [Authentication and Authorization](https://cloud.baidu.com/doc/qianfan-api/s/ym9chdsy5) to obtain the API key. If using a locally deployed large model service, refer to the [PaddleNLP Large Model Deployment Documentation](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/llm) for deployment of the dialogue interface and vectorization interface for large models, and fill in the corresponding `base_url` and `api_key`. If you need to use a multimodal large model for data fusion, refer to the OpenAI service deployment in the [PaddleMIX Model Documentation](https://github.com/PaddlePaddle/PaddleMIX/tree/develop/paddlemix/examples/ppdocbee2) for multimodal large model deployment, and fill in the corresponding `base_url` and `api_key`.
976976

977+
Alternatively, you can call the LLM through [OrcaRouter](https://www.orcarouter.ai), an OpenAI-compatible gateway. After obtaining an API key from [OrcaRouter](https://www.orcarouter.ai), pass it via the `--orcarouter_api_key` argument and the pipeline routes both the chat and vectorization requests to OrcaRouter.
978+
977979
**Note**: If local deployment of a multimodal large model is restricted due to the local environment, you can comment out the lines containing the `mllm` variable in the code and only use the large language model for information extraction.
978980

979981
### 2.1 Command Line Experience
@@ -986,6 +988,9 @@ paddleocr pp_chatocrv4_doc -i vehicle_certificate-1.png -k 驾驶室准乘人数
986988

987989
# 通过 --invoke_mllm 和 --pp_docbee_base_url 使用多模态大模型
988990
paddleocr pp_chatocrv4_doc -i vehicle_certificate-1.png -k 驾驶室准乘人数 --qianfan_api_key your_api_key --invoke_mllm True --pp_docbee_base_url http://127.0.0.1:8080/
991+
992+
# Use the OrcaRouter OpenAI-compatible gateway via --orcarouter_api_key
993+
paddleocr pp_chatocrv4_doc -i vehicle_certificate-1.png -k 驾驶室准乘人数 --orcarouter_api_key your_api_key
989994
```
990995

991996
The examples above use local Paddle inference engines by default. By default, each module selects the appropriate local Paddle inference engine according to the default model name: models that support only dynamic graph use `paddle_dynamic`, while models that support both static and dynamic graph prefer `paddle_static`. To run them, first install PaddlePaddle by following [PaddlePaddle Framework Installation](../paddlepaddle_installation.en.md).
@@ -1386,6 +1391,12 @@ Any float > <code>0</code></li>
13861391
<td><code>str</code></td>
13871392
<td></td>
13881393
</tr>
1394+
<tr>
1395+
<td><code>orcarouter_api_key</code></td>
1396+
<td><b>Meaning:</b>API key for the [OrcaRouter](https://www.orcarouter.ai) gateway.</td>
1397+
<td><code>str</code></td>
1398+
<td></td>
1399+
</tr>
13891400
<td><code>pp_docbee_base_url</code></td>
13901401
<td><b>Meaning:</b>URL for the multimodal large language model service.</td>
13911402
<td><code>str</code></td>

‎docs/version3.x/pipeline_usage/PP-ChatOCRv4.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -864,6 +864,8 @@ devanagari_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="https://padd
864864

865865
在进行模型推理之前,首先需要准备大语言模型的 api_key,PP-ChatOCRv4 支持在[百度云千帆平台](https://console.bce.baidu.com/qianfan/ais/console/onlineService)或者本地部署的标准 OpenAI 接口大模型服务。如果使用百度云千帆平台,可以参考[认证鉴权](https://cloud.baidu.com/doc/qianfan-api/s/ym9chdsy5) 获取 api_key。如果使用本地部署的大模型服务,可以参考[PaddleNLP大模型部署文档](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/llm)进行大模型部署对话接口部署和向量化接口部署,并填写对应的 base_url 和 api_key 即可。如果需要使用多模态大模型进行数据融合,可以参考[PaddleMIX模型文档](https://github.com/PaddlePaddle/PaddleMIX/tree/develop/paddlemix/examples/ppdocbee2)中的OpenAI服务部署进行多模态大模型部署,并填写对应的 base_url 和 api_key 即可。
866866

867+
此外,也可以使用 [OrcaRouter](https://www.orcarouter.ai) 这一 OpenAI 兼容网关来调用大模型:在[OrcaRouter](https://www.orcarouter.ai) 获取 API key 后,使用 `--orcarouter_api_key` 参数即可,产线会将对话与向量化请求统一路由到 OrcaRouter。
868+
867869
**注:** 如果因本地环境限制无法在本地部署多模态大模型,可以将代码中的含有“mllm”变量的行注释掉,仅使用大语言模型完成信息抽取。
868870

869871
### 2.1 命令行方式体验
@@ -875,6 +877,9 @@ paddleocr pp_chatocrv4_doc -i vehicle_certificate-1.png -k 驾驶室准乘人数
875877

876878
# 通过 --invoke_mllm 和 --pp_docbee_base_url 使用多模态大模型
877879
paddleocr pp_chatocrv4_doc -i vehicle_certificate-1.png -k 驾驶室准乘人数 --qianfan_api_key your_api_key --invoke_mllm True --pp_docbee_base_url http://127.0.0.1:8080/
880+
881+
# 通过 --orcarouter_api_key 使用 OrcaRouter OpenAI 兼容网关
882+
paddleocr pp_chatocrv4_doc -i vehicle_certificate-1.png -k 驾驶室准乘人数 --orcarouter_api_key your_api_key
878883
```
879884

880885
上述命令默认使用本地飞桨推理引擎。默认情况下,各模块会根据默认模型名称选择合适的本地飞桨推理引擎:仅支持动态图的模型使用 `paddle_dynamic`;同时支持静态图和动态图的模型优先使用 `paddle_static`。如需运行,请先参考[飞桨框架安装说明](../paddlepaddle_installation.md)安装 PaddlePaddle。
@@ -1245,6 +1250,12 @@ paddleocr pp_chatocrv4_doc -i vehicle_certificate-1.png -k 驾驶室准乘人数
12451250
<td></td>
12461251
</tr>
12471252
<tr>
1253+
<td><code>orcarouter_api_key</code></td>
1254+
<td><b>含义:</b>[OrcaRouter](https://www.orcarouter.ai) 网关的API key。</td>
1255+
<td><code>str</code></td>
1256+
<td></td>
1257+
</tr>
1258+
<tr>
12481259
<td><code>pp_docbee_base_url</code></td>
12491260
<td><b>含义:</b>多模态大模型服务的URL。</td>
12501261
<td><code>str</code></td>

‎docs/version3.x/pipeline_usage/PP-DocTranslation.en.md‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -686,14 +686,17 @@ Before using the PP-DocTranslation pipeline locally, please ensure that you have
686686

687687
Please note: If you encounter issues such as the program becoming unresponsive, unexpected program termination, running out of memory resources, or extremely slow inference during execution, please try adjusting the configuration according to the documentation, such as disabling unnecessary features or using lighter-weight models.
688688

689-
Before use, you need to prepare the API key for a large language model, which supports the [Baidu Cloud Qianfan Platform](https://console.bce.baidu.com/qianfan/ais/console/onlineService) or local large model services that comply with the OpenAI interface standards.
689+
Before use, you need to prepare the API key for a large language model, which supports the [Baidu Cloud Qianfan Platform](https://console.bce.baidu.com/qianfan/ais/console/onlineService) or local large model services that comply with the OpenAI interface standards. Alternatively, you can use [OrcaRouter](https://www.orcarouter.ai), an OpenAI-compatible gateway: after obtaining an API key from [OrcaRouter](https://www.orcarouter.ai), pass it via the `--orcarouter_api_key` argument.
690690

691691
### 2.1 Experience via Command Line
692692

693693
You can download the [test file](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/vehicle_certificate-1.png) and quickly experience the pipeline effect with a single command:
694694

695695
```bash
696696
paddleocr pp_doctranslation -i vehicle_certificate-1.png --target_language en --qianfan_api_key your_api_key
697+
698+
# Use the OrcaRouter OpenAI-compatible gateway via --orcarouter_api_key
699+
paddleocr pp_doctranslation -i vehicle_certificate-1.png --target_language en --orcarouter_api_key your_api_key
697700
```
698701

699702
The examples above use local Paddle inference engines by default. By default, each module selects the appropriate local Paddle inference engine according to the default model name: models that support only dynamic graph use `paddle_dynamic`, while models that support both static and dynamic graph prefer `paddle_static`. To run them, first install PaddlePaddle by following [PaddlePaddle Framework Installation](../paddlepaddle_installation.en.md).
@@ -1255,6 +1258,12 @@ If not set, the pipeline initialized value will be used, default is <code>True</
12551258
<td></td>
12561259
</tr>
12571260
<tr>
1261+
<td><code>orcarouter_api_key</code></td>
1262+
<td><b>Meaning:</b>API key for the [OrcaRouter](https://www.orcarouter.ai) gateway.</td>
1263+
<td><code>str</code></td>
1264+
<td></td>
1265+
</tr>
1266+
<tr>
12581267
<td><code>device</code></td>
12591268
<td><b>Meaning:</b>Device used for inference.<br/>
12601269
<b>Description:</b>

‎docs/version3.x/pipeline_usage/PP-DocTranslation.md‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -683,14 +683,17 @@ devanagari_PP-OCRv3_mobile_rec_infer.tar">推理模型</a>/<a href="https://padd
683683

684684
**请注意,如果在执行过程中遇到程序失去响应、程序异常退出、内存资源耗尽、推理速度极慢等问题,请尝试参考文档调整配置,例如关闭不需要使用的功能或使用更轻量的模型。**
685685

686-
使用前需准备大语言模型的API密钥,支持[百度云千帆平台](https://console.bce.baidu.com/qianfan/ais/console/onlineService)或符合OpenAI接口标准的本地大模型服务。
686+
使用前需准备大语言模型的API密钥,支持[百度云千帆平台](https://console.bce.baidu.com/qianfan/ais/console/onlineService)或符合OpenAI接口标准的本地大模型服务。此外,也可以使用 [OrcaRouter](https://www.orcarouter.ai) 这一 OpenAI 兼容网关:在[OrcaRouter](https://www.orcarouter.ai) 获取 API key 后,使用 `--orcarouter_api_key` 参数即可。
687687

688688
### 2.1 命令行方式体验
689689

690690
可以下载 [测试文件](https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/vehicle_certificate-1.png),使用一行命令即可快速体验产线效果:
691691

692692
```bash
693693
paddleocr pp_doctranslation -i vehicle_certificate-1.png --target_language en --qianfan_api_key your_api_key
694+
695+
# 通过 --orcarouter_api_key 使用 OrcaRouter OpenAI 兼容网关
696+
paddleocr pp_doctranslation -i vehicle_certificate-1.png --target_language en --orcarouter_api_key your_api_key
694697
```
695698

696699
上述命令默认使用本地飞桨推理引擎。默认情况下,各模块会根据默认模型名称选择合适的本地飞桨推理引擎:仅支持动态图的模型使用 `paddle_dynamic`;同时支持静态图和动态图的模型优先使用 `paddle_static`。如需运行,请先参考[飞桨框架安装说明](../paddlepaddle_installation.md)安装 PaddlePaddle。
@@ -1276,6 +1279,12 @@ paddleocr pp_doctranslation -i vehicle_certificate-1.png --target_language en --
12761279
<td></td>
12771280
</tr>
12781281
<tr>
1282+
<td><code>orcarouter_api_key</code></td>
1283+
<td><b>含义:</b>[OrcaRouter](https://www.orcarouter.ai) 网关的API key。</td>
1284+
<td><code>str</code></td>
1285+
<td></td>
1286+
</tr>
1287+
<tr>
12791288
<td><code>device</code></td>
12801289
<td><b>含义:</b>用于推理的设备。<br/>
12811290
<b>含义:</b>

‎paddleocr/_pipelines/pp_chatocrv4_doc.py‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -675,6 +675,11 @@ def _update_subparser(self, subparser):
675675
type=str,
676676
help="Configuration for the embedding model.",
677677
)
678+
subparser.add_argument(
679+
"--orcarouter_api_key",
680+
type=str,
681+
help="API key for the OrcaRouter OpenAI-compatible gateway.",
682+
)
678683
subparser.add_argument(
679684
"--pp_docbee_base_url",
680685
type=str,
@@ -703,6 +708,22 @@ def execute_with_args(self, args):
703708
"api_type": "openai",
704709
"api_key": qianfan_api_key,
705710
}
711+
orcarouter_api_key = params.pop("orcarouter_api_key")
712+
if orcarouter_api_key is not None:
713+
params["retriever_config"] = {
714+
"module_name": "retriever",
715+
"model_name": "openai/text-embedding-3-small",
716+
"base_url": "https://api.orcarouter.ai/v1",
717+
"api_type": "openai",
718+
"api_key": orcarouter_api_key,
719+
}
720+
params["chat_bot_config"] = {
721+
"module_name": "chat_bot",
722+
"model_name": "deepseek/deepseek-v4-flash-0731",
723+
"base_url": "https://api.orcarouter.ai/v1",
724+
"api_type": "openai",
725+
"api_key": orcarouter_api_key,
726+
}
706727
pp_docbee_base_url = params.pop("pp_docbee_base_url")
707728
if pp_docbee_base_url is not None:
708729
params["mllm_chat_bot_config"] = {

‎paddleocr/_pipelines/pp_doctranslation.py‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -906,6 +906,11 @@ def _update_subparser(self, subparser):
906906
type=str,
907907
help="Configuration for the embedding model.",
908908
)
909+
subparser.add_argument(
910+
"--orcarouter_api_key",
911+
type=str,
912+
help="API key for the OrcaRouter OpenAI-compatible gateway.",
913+
)
909914

910915
def execute_with_args(self, args):
911916
params = get_subcommand_args(args)
@@ -921,6 +926,15 @@ def execute_with_args(self, args):
921926
"api_type": "openai",
922927
"api_key": qianfan_api_key,
923928
}
929+
orcarouter_api_key = params.pop("orcarouter_api_key")
930+
if orcarouter_api_key is not None:
931+
params["chat_bot_config"] = {
932+
"module_name": "chat_bot",
933+
"model_name": "deepseek/deepseek-v4-flash-0731",
934+
"base_url": "https://api.orcarouter.ai/v1",
935+
"api_type": "openai",
936+
"api_key": orcarouter_api_key,
937+
}
924938

925939
chatocr = PPDocTranslation(**params)
926940

0 commit comments

Comments
 (0)