Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ The following sections will guide you through the process of deploying and query
- Create a Python virtual environment and install the `vllm` package (version `>=0.6.1.post1` to ensure maximum compatibility with all Mistral models).
- Authenticate on the HuggingFace Hub using your access token `$HF_TOKEN`:
```bash
huggingface-cli login --token $HF_TOKEN
hf auth login --token $HF_TOKEN
```

<SectionTab as="h2" variant="secondary" sectionId="offline-mode-inference">Offline Mode Inference</SectionTab>
Expand Down Expand Up @@ -120,4 +120,4 @@ If you are looking to deploy vLLM as a containerized inference server, you can l
```
</TabItem>
</Tabs>
Once the container is up and running, you will be able to run inference on your model using the same code as in a standalone deployment.
Once the container is up and running, you will be able to run inference on your model using the same code as in a standalone deployment.