Skip to content
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,12 @@
"---\n",
"\n",
"**[Deploy a hybrid reasoning LLM](https://docs.ray.io/en/latest/serve/tutorials/deployment-serve-llm/hybrid-reasoning-llm/README.html)** \n",
"Deploy models that can switch between reasoning and non-reasoning modes for flexible usage, such as Qwen-3."
"Deploy models that can switch between reasoning and non-reasoning modes for flexible usage, such as Qwen-3.\n",
"\n",
"---\n",
"\n",
"**[Deploy gpt-oss](https://docs.ray.io/en/latest/ray-overview/examples/deployment-serve-llm/gpt-oss/README.html)** \n",
"Deploy gpt-oss reasoning models, including `gpt-oss-20b` for lower latency use cases and `gpt-oss-120b` for high-reasoning, production-scale workloads."
Comment thread
Aydin-ab marked this conversation as resolved.
Outdated
]
}
],
Expand Down
5 changes: 5 additions & 0 deletions doc/source/serve/tutorials/deployment-serve-llm/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,3 +39,8 @@ Deploy models with reasoning capabilities designed for long-context tasks, codin

**[Deploy a hybrid reasoning LLM](https://docs.ray.io/en/latest/serve/tutorials/deployment-serve-llm/hybrid-reasoning-llm/README.html)**
Deploy models that can switch between reasoning and non-reasoning modes for flexible usage, such as Qwen-3.

---

**[Deploy gpt-oss](https://docs.ray.io/en/latest/ray-overview/examples/deployment-serve-llm/gpt-oss/README.html)**
Deploy gpt-oss reasoning models, including `gpt-oss-20b` for lower latency use cases and `gpt-oss-120b` for high-reasoning, production-scale workloads.
Comment thread
Aydin-ab marked this conversation as resolved.
Outdated
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@ for nb in \
"large-size-llm/notebook" \
"vision-llm/notebook" \
"reasoning-llm/notebook" \
"hybrid-reasoning-llm/notebook"
"hybrid-reasoning-llm/notebook" \
"gpt-oss/notebook"
do
python ci/nb2py.py "${nb}.ipynb" "${nb}.py" --ignore-cmds
python "${nb}.py"
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
FROM anyscale/ray:2.49.0-slim-py312-cu128

# C compiler for Triton’s runtime build step (vLLM V1 engine)
# https://github.com/vllm-project/vllm/issues/2997
RUN sudo apt-get update && \
sudo apt-get install -y --no-install-recommends build-essential
Comment thread
Aydin-ab marked this conversation as resolved.

RUN pip install vllm==0.10.1
Comment thread
Aydin-ab marked this conversation as resolved.
Loading