Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions assets/agw-docs/standalone/quickstart/llm.md
Original file line number Diff line number Diff line change
Expand Up @@ -172,11 +172,10 @@ export OPENAI_API_KEY='<your-api-key>'

### Step 2: Start agentgateway

You add the model from the UI in the next steps, so you can start agentgateway with an empty config file.
You add the model from the UI in the next steps, so you can start agentgateway without a config file. When you run `agentgateway` without specifying a config, it bootstraps a basic config at `~/.config/agentgateway/config.yaml` and uses it automatically.

```sh
echo '{}' > config.yaml
agentgateway -f config.yaml
agentgateway
```

Example output:
Expand Down
5 changes: 2 additions & 3 deletions assets/agw-docs/standalone/quickstart/mcp.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,11 +134,10 @@ PORT=3005 npx -y @modelcontextprotocol/server-everything streamableHttp

### Step 2: Start agentgateway

You add the MCP server from the UI in the next steps, so you can start agentgateway with an empty config file.
You add the MCP server from the UI in the next steps, so you can start agentgateway without a config file. When you run `agentgateway` without specifying a config, it bootstraps a basic config at `~/.config/agentgateway/config.yaml` and uses it automatically.

```sh
echo '{}' > config.yaml
agentgateway -f config.yaml
agentgateway
```

Example output:
Expand Down
5 changes: 2 additions & 3 deletions assets/agw-docs/standalone/quickstart/non-agentic-http.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,13 +103,12 @@ sleep 3
{{< /doc-test >}}
{{< /version >}}
{{< version exclude-if="1.2.x,1.1.x,1.0,x" >}}
You add the listener and route from the UI, so you can start agentgateway with an empty config file.
You add the listener and route from the UI, so you can start agentgateway without a config file. When you run `agentgateway` without specifying a config, it bootstraps a basic config at `~/.config/agentgateway/config.yaml` and uses it automatically.

1. In a separate terminal, start agentgateway.

```sh
echo '{}' > config.yaml
agentgateway -f config.yaml
agentgateway
```

2. Open the [agentgateway UI](http://localhost:15000/ui/). On the **Gateway Overview**, find the **Traffic** row and click **Enable Traffic**.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,6 @@ Additionally, authentication can run in three different modes:
* **Permissive**: Requests are never rejected. This setting is useful for usage of claims in later steps such as authorization or logging.
*Warning*: This allows requests without an API key!

For a standalone LLM listener example using `llm.policies.apiKey.mode: permissive`, see [Manage API keys]({{< link-hextra path="/llm/api-keys/" >}}).

```yaml
apiKey:
mode: strict
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -108,8 +108,6 @@ You can control how agentgateway handles requests that lack valid credentials by
| `optional` | If a token is present, it is validated. Requests without a token are permitted. |
| `permissive` | Requests are never rejected based on authentication. |

For the same permissive-mode concept on the standalone LLM listener, see [Manage API keys]({{< link-hextra path="/llm/api-keys/" >}}) (`llm.policies.apiKey.mode: permissive`).

The following example sets the mode to `permissive`:

```yaml
Expand Down
2 changes: 1 addition & 1 deletion content/docs/standalone/latest/llm/about.md
Original file line number Diff line number Diff line change
Expand Up @@ -209,5 +209,5 @@ llm:
In this example, both routes have one header matcher, so they have equal specificity. Because the Fireworks route is listed first, it takes priority when both routes match.

{{< callout type="info" >}}
For advanced routing based on request body fields like the `model` name, see [Content-based routing]({{< link-hextra path="/llm/content-routing/" >}}).
For advanced routing based on request body fields like the `model` name, see [Virtual models]({{< link-hextra path="/llm/virtual-models/" >}}).
{{< /callout >}}
125 changes: 0 additions & 125 deletions content/docs/standalone/latest/llm/api-keys.md

This file was deleted.

78 changes: 78 additions & 0 deletions content/docs/standalone/latest/llm/api-types/embeddings.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
---
title: Embeddings
weight: 35
description: Send embedding requests through agentgateway using the OpenAI-compatible Embeddings API.
test: skip
---

The Embeddings API (`/v1/embeddings`) creates vector representations of text that you can use for search, retrieval, clustering, and other semantic workflows.

## About

Agentgateway supports the OpenAI-compatible Embeddings API. Requests to `/v1/embeddings` are routed to your configured provider while agentgateway applies the same routing, authentication, observability, and policy framework that you use for other LLM traffic.

## Route type configuration

In the simplified `llm` configuration, agentgateway automatically maps `/v1/embeddings` requests to the `embeddings` route type, so no explicit route configuration is required.

```yaml
# yaml-language-server: $schema=https://agentgateway.dev/schema/config
llm:
models:
- name: "*"
provider: openAI
params:
apiKey: "$OPENAI_API_KEY"
```

To configure the route type explicitly, use the `binds/listeners/routes` format and set the `embeddings` route type in the `policies.ai.routes` map.

```yaml
# yaml-language-server: $schema=https://agentgateway.dev/schema/config
binds:
- port: 4000
listeners:
- routes:
- backends:
- ai:
name: openai
provider:
openAI: {}
policies:
ai:
routes:
"/v1/embeddings": "embeddings"
backendAuth:
key: "$OPENAI_API_KEY"
```

{{< callout type="info" >}}
For detailed information about model routing and configuration modes, see [Model routing and aliases]({{< link-hextra path="/llm/about/" >}}).
{{< /callout >}}

## Using the API

Send a request to the `/v1/embeddings` endpoint. The response includes an embedding vector for each input item.

{{< tabs items="Curl,Other" >}}
{{% tab %}}

```shell
curl 'http://localhost:4000/v1/embeddings' \
--header 'Content-Type: application/json' \
--data '{
"model": "text-embedding-3-small",
"input": [
"agentgateway routes LLM traffic",
"embeddings turn text into vectors"
]
}'
```

{{% /tab %}}
{{% tab %}}

[View other LLM client integrations](/docs/standalone/latest/integrations/llm-clients/).

{{% /tab %}}
{{< /tabs >}}
49 changes: 49 additions & 0 deletions content/docs/standalone/latest/llm/api-types/models.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
---
title: Models
weight: 55
description: List available models through agentgateway using the OpenAI-compatible Models API.
test: skip
---

The Models API (`/v1/models`) lists the models that are available through the configured LLM provider.

## About

Agentgateway supports the OpenAI-compatible Models API. Use this endpoint when clients need to discover available model IDs, such as web UIs, SDKs, or developer tools that populate model selectors from `/v1/models`.

## Route type configuration

In the simplified `llm` configuration, agentgateway automatically maps `/v1/models` requests to the `models` route type, so no explicit route configuration is required.

```yaml
# yaml-language-server: $schema=https://agentgateway.dev/schema/config
llm:
models:
- name: "*"
provider: openAI
params:
apiKey: "$OPENAI_API_KEY"
```

{{< callout type="info" >}}
For detailed information about model routing and configuration modes, see [Model routing and aliases]({{< link-hextra path="/llm/about/" >}}).
{{< /callout >}}

## Using the API

Send a request to the `/v1/models` endpoint to list models from the upstream provider.

{{< tabs items="Curl,Other" >}}
{{% tab %}}

```shell
curl 'http://localhost:4000/v1/models'
```

{{% /tab %}}
{{% tab %}}

[View other LLM client integrations](/docs/standalone/latest/integrations/llm-clients/).

{{% /tab %}}
{{< /tabs >}}
Loading
Loading