Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
# A2UI Client-to-Server Actions
# A2UI Renderer-to-Agent Actions

Interactivity in A2UI relies on a bidirectional communication loop. While the Agent drives the UI by streaming component and data updates, the Client communicates user intent back to the Agent through **Actions** and **Data Model Synchronization**.
Interactivity in A2UI relies on a bidirectional communication loop. While the Agent drives the UI by streaming component and data updates, the Renderer communicates user intent back to the Agent through **Actions** and **Data Model Synchronization**.

## Action Architecture

Actions allow UI components to trigger behavior. They are defined in the `Action` schema in [`common_types.json`](../specification/v0_9/json/common_types.json) and come in two flavors:

1. **Server Events**: Dispatched to the Agent for processing (e.g., clicking "Submit").
2. **Local Function Calls**: Executed entirely on the client (e.g., opening a URL).
1. **Agent Events**: Dispatched to the Agent for processing (e.g., clicking "Submit").
2. **Local Function Calls**: Executed entirely on the renderer (e.g., opening a URL).

### Action Wiring in Schema

Components like `Button` expose an `action` property. Here is how a server event is wired:
Components like `Button` expose an `action` property. Here is how an agent event is wired:

```json
{
Expand All @@ -36,9 +36,9 @@ Components like `Button` expose an `action` property. Here is how a server event
> [!NOTE]
> **Context vs. Data Model**: While the Data Model represents the entire state tree of a surface, the `context` in an action is effectively a hand-picked **"view"** or subset of that state. This simplifies the Agent's job by providing exactly the values needed for a specific event, without requiring the Agent to navigate a potentially large and complex data model.

### Local Actions vs. Server Events
### Local Actions vs. Agent Events
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Local Actions => Local Function Calls

here and below

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is covered by #1088

Thank you!


While Server Events are the primary way to interact with an agent, **Local Actions** allow for immediate client-side behavior without a network round-trip. This is essential for responsive UI patterns.
While Agent Events are the primary way to interact with an agent, **Local Actions** allow for immediate renderer-side behavior without a network round-trip. This is essential for responsive UI patterns.

```json
{
Expand All @@ -56,14 +56,14 @@ While Server Events are the primary way to interact with an agent, **Local Actio

Common uses for Local Actions include:

- **Validation**: Validating inputs for a form before submitting it to the server.
- **Validation**: Validating inputs for a form before submitting it to the agent.
- **Formatting**: Using `formatString` to format a local display value.

### Basic Catalog Action Validation (Checks)

The basic catalog defines a limited set of checks that can be performed on the client. Interactive components can define a list of `checks` (using the `Checkable` schema in [`common_types.json`](../specification/v0_9/json/common_types.json)). For a `Button`, if any check fails, the button is **automatically disabled** on the client.
The basic catalog defines a limited set of checks that can be performed on the renderer. Interactive components can define a list of `checks` (using the `Checkable` schema in [`common_types.json`](../specification/v0_9/json/common_types.json)). For a `Button`, if any check fails, the button is **automatically disabled** on the renderer.

- **UX Focus**: Action checks are designed to manage **UI State (User Experience)** by preventing invalid interactions before they happen. They are not a replacement for **Data Integrity** checks, which must still be performed on the server.
- **UX Focus**: Action checks are designed to manage **UI State (User Experience)** by preventing invalid interactions before they happen. They are not a replacement for **Data Integrity** checks, which must still be performed on the agent.

This allows the UI to enforce requirements (like a non-empty field) before the user even tries to submit.

Expand All @@ -87,12 +87,12 @@ This allows the UI to enforce requirements (like a non-empty field) before the u

## Local State Updates & The "Write" Contract

Before an action is even dispatched, the client is already managing the state of the UI locally. A2UI defines a **Read/Write Contract** for all input components (like `TextField`, `CheckBox`, or `Slider`).
Before an action is even dispatched, the renderer is already managing the state of the UI locally. A2UI defines a **Read/Write Contract** for all input components (like `TextField`, `CheckBox`, or `Slider`).

1. **Read (Model → View)**: When a component renders, it pulls its value from the bound `path` in the Data Model.
2. **Write (View → Model)**: As soon as a user interacts (e.g., typing a character or clicking a checkbox), the client **immediately** writes the new value into the local Data Model.
2. **Write (View → Model)**: As soon as a user interacts (e.g., typing a character or clicking a checkbox), the renderer **immediately** writes the new value into the local Data Model.

This means the local model is **always** the source of truth for the UI's current state. This "View-to-Model" synchronization happens purely on the client. Only when a User Action (like a Button click) is triggered is this state synchronized back to the server.
This means the local model is **always** the source of truth for the UI's current state. This "View-to-Model" synchronization happens purely on the renderer. Only when a User Action (like a Button click) is triggered is this state synchronized back to the agent.

> [!IMPORTANT]
> **Synchronous Updates**: Local model updates are **synchronous**. This guarantees that the Data Model is fully updated before any Action resolves its `context` paths or a `DataModelSync` payload is packaged. There are no race conditions between typing and clicking; the "Write" is always committed first.
Expand All @@ -105,19 +105,19 @@ This separation allows for a robust form submission pattern:

- **Binding**: A `TextField` is bound to `/reservationTime`.
- **Interaction**: The user types "7:00 PM". The local model at `/reservationTime` is updated instantly.
- **Submission**: The user clicks a "Book" button. The button's action resolves the `path: "/reservationTime"` from the local model and sends the current value to the server.
- **Submission**: The user clicks a "Book" button. The button's action resolves the `path: "/reservationTime"` from the local model and sends the current value to the agent.

## User Interaction Flow

When a user interacts with a component (e.g., clicks a button):

1. **Resolve**: The client resolves all `path` references in the `context` against the local **Data Model**.
2. **Construct**: The client builds an `action` payload conforming to [`client_to_server.json`](../specification/v0_9/json/client_to_server.json).
1. **Resolve**: The renderer resolves all `path` references in the `context` against the local **Data Model**.
2. **Construct**: The renderer builds an `action` payload conforming to [`client_to_server.json`](../specification/v0_9/json/client_to_server.json).
3. **Dispatch**: The payload is sent via the chosen transport (e.g., A2A, WebSockets).

### Example: The Action Payload (v0.9)

If a user clicks the button above with a data model containing `{"reservationTime": "7:00 PM", "partySize": 4}`, the client sends a message using the `action` key:
If a user clicks the button above with a data model containing `{"reservationTime": "7:00 PM", "partySize": 4}`, the renderer sends a message using the `action` key:

```json
{
Expand Down Expand Up @@ -153,13 +153,13 @@ if action_name == "submit_reservation":
response = await llm.generate(query)
```

## Client-to-Server Error Reporting
## Renderer-to-Agent Error Reporting

In addition to user actions, the client can report system-level errors back to the server using the `error` payload defined in [`client_to_server.json`](../specification/v0_9/json/client_to_server.json).
In addition to user actions, the renderer can report system-level errors back to the agent using the `error` payload defined in [`client_to_server.json`](../specification/v0_9/json/client_to_server.json).

### Validation Failures

If the agent sends A2UI JSON that violates the catalog schema or protocol rules, the client sends a `VALIDATION_FAILED` error. This is a critical feedback loop for agentic systems:
If the agent sends A2UI JSON that violates the catalog schema or protocol rules, the renderer sends a `VALIDATION_FAILED` error. This is a critical feedback loop for agentic systems:

```json
{
Expand All @@ -177,11 +177,11 @@ The agent can catch this error, apologize (or self-correct internally), and re-s

## Data Model Sync (v0.9)

In A2UI v0.9, we introduced a powerful "stateless" synchronization feature. This allows the client to automatically include the **entire data model** of a surface in the metadata of every message it sends to the server.
In A2UI v0.9, we introduced a powerful "stateless" synchronization feature. This allows the renderer to automatically include the **entire data model** of a surface in the metadata of every message it sends to the agent.

### Enabling Sync

Synchronization is requested by the agent during surface initialization. By setting `sendDataModel: true` in the `createSurface` message, the agent instructs the client to start the sync loop.
Synchronization is requested by the agent during surface initialization. By setting `sendDataModel: true` in the `createSurface` message, the agent instructs the renderer to start the sync loop.

```json
{
Expand All @@ -196,7 +196,7 @@ Synchronization is requested by the agent during surface initialization. By sett

### Sync on the Wire

When sync is enabled, the client does not send the data model as a separate message. Instead, it attaches it as **metadata** to the outgoing transport envelope (e.g., an A2A message).
When sync is enabled, the renderer does not send the data model as a separate message. Instead, it attaches it as **metadata** to the outgoing transport envelope (e.g., an A2A message).

In an A2A (Agent-to-Agent) binding, the data model is placed in an `a2uiClientDataModel` object within the envelope's `metadata` field.

Expand Down Expand Up @@ -226,13 +226,13 @@ In an A2A (Agent-to-Agent) binding, the data model is placed in an `a2uiClientDa
- **Stateless Agents**: The agent doesn't need to maintain local state for every user session; it receives the full current context with every single interaction.
- **Verbal Shortcuts**: Allows the user to trigger actions via voice or text (e.g., "okay submit") even without clicking a specific button. Since the agent receives the updated data model with the text message, it can process the request immediately.

## Client Metadata & Capabilities
## Renderer Metadata & Capabilities

Before an agent can safely send a UI, the client must advertise which component catalogs it supports. This is handled via the `a2uiClientCapabilities` object.
Before an agent can safely send a UI, the renderer must advertise which component catalogs it supports. This is handled via the `a2uiClientCapabilities` object.

### Advertising Capabilities

Clients include an `a2uiClientCapabilities` object in the **metadata** of their messages to the server (e.g., in the `metadata` field of an A2A envelope).
Renderers include an `a2uiClientCapabilities` object in the **metadata** of their messages to the agent (e.g., in the `metadata` field of an A2A envelope).

```json
{
Expand All @@ -246,7 +246,7 @@ Clients include an `a2uiClientCapabilities` object in the **metadata** of their
}
```

- **`supportedCatalogIds`**: An array of catalog URIs the client can render.
- **`supportedCatalogIds`**: An array of catalog URIs the renderer can render.
- **`inlineCatalogs`**: (Optional) For development or specialized environments, allows sending the full catalog schema inline.

Without this handshake, an agent cannot be certain that the renderer can handle the specific components being sent.
Expand Down Expand Up @@ -287,18 +287,18 @@ A2UI is designed with secure, sandboxed communication as a core principle. Becau

### Sandboxed Execution

A core selling point of A2UI is security through restriction. By prohibiting arbitrary code execution (like injecting raw JavaScript) from the agent, A2UI ensures that agents can only trigger pre-registered, client-side behaviors. The `functionCall` mechanism acts as a safe, sandboxed way for the agent to interact with the client environment without exposing the user to malicious scripts.
A core selling point of A2UI is security through restriction. By prohibiting arbitrary code execution (like injecting raw JavaScript) from the agent, A2UI ensures that agents can only trigger pre-registered behaviors. The `functionCall` mechanism acts as a safe, sandboxed way for the agent to interact with the renderer's environment without exposing the user to malicious scripts.

### Data Model Isolation and Orchestrator Routing

When `sendDataModel: true` is enabled, the client includes the surface's entire data model in outgoing messages. Developers must understand the visibility of this data:
When `sendDataModel: true` is enabled, the renderer includes the surface's entire data model in outgoing messages. Developers must understand the visibility of this data:

- **Point-to-Point Visibility**: Only the backend receiving the transport envelope (the Agent that created the surface, or an intermediate Orchestrator) can read this payload.
- **The Orchestrator's Responsibility**: In a multi-agent architecture, a central Orchestrator often routes user intents to specialized sub-agents. The Orchestrator must enforce **data isolation**. It is responsible for parsing the `a2uiClientDataModel`, identifying the `surfaceId`, and ensuring that the data model is only passed to the specific sub-agent that owns that surface. Data from one agent's surface must never leak to another agent.

## Orchestration & Routing

In multi-agent systems, a central **Orchestrator** often manages interactions between a user and several specialized sub-agents. A key challenge is ensuring that `action` messages from the client are routed back to the specific sub-agent that generated the UI surface.
In multi-agent systems, a central **Orchestrator** often manages interactions between a user and several specialized sub-agents. A key challenge is ensuring that `action` messages from the renderer are routed back to the specific sub-agent that generated the UI surface.

### The Surface Ownership Pattern

Expand All @@ -317,7 +317,7 @@ def on_surface_created(surface_id, agent_name, session):

#### 2. Routing User Actions

When the client sends an `action` back to the orchestrator, the orchestrator looks up the `surfaceId` and transfers the request to the correct sub-agent.
When the renderer sends an `action` back to the orchestrator, the orchestrator looks up the `surfaceId` and transfers the request to the correct sub-agent.

```python
# Simplified Orchestrator Logic: Route Action
Expand Down Expand Up @@ -412,8 +412,8 @@ The agent created the surface with `sendDataModel: true`:
}
```

**Client Transmission:**
The client sends an A2A message containing the user's text and the data model in the metadata:
**Renderer Transmission:**
The renderer sends an A2A message containing the user's text and the data model in the metadata:

```json
{
Expand Down
18 changes: 10 additions & 8 deletions docs/glossary.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,7 @@ And, sometimes, an agent is using a predefined catalog, thus forcing the rendere

### GenUI Component

UI component, allowed for use by AI. Examples: date picker, carousel, button, hotel selector.

UI component, allowed for use by agent. Examples: date picker, carousel, button, hotel selector.

### Catalog

Expand All @@ -44,7 +43,6 @@ It is observed that depending on use case, catalog components may be more or les

Components like HotelCheckout or FlightSelector.


### Basic Catalog

A catalog maintained by the A2UI team to get up and running quickly with A2UI.
Expand Down Expand Up @@ -106,7 +104,7 @@ Functionality of A2UI renderer consists of layers that can be developed separate

Code that implements the execution of the agent’s instructions in a concrete framework. For example:

- JavaScript core and catalogs may be adapted to Angular, Electron and Lit frameworks.
- JavaScript core and catalogs may be adapted to Angular, Electron, React and Lit frameworks.
- Dart core and catalogs may be adapted to Flutter and Jaspr frameworks.

See [Angular adapter](https://github.com/google/A2UI/tree/main/renderers/angular/README.md).
Expand Down Expand Up @@ -136,6 +134,10 @@ As the protocol allows streaming, any message can be finished (completely delive

See [data flow](https://github.com/google/A2UI/blob/main/docs/concepts/data-flow.md).

### Agent turn

Set of messages sent by agent, before it starts waiting for user input.

### Data model

Observable, hierarchical, JSON-like object, shared between renderer and agent and updatable by both. Each Surface has a separate Data Model.
Expand All @@ -154,7 +156,7 @@ See [example in basic catalog](https://github.com/google/A2UI/blob/db1fbe726b8d4

### Client function

A function provided for AI to invoke when needed.
A function provided for agent to invoke when needed.

Do not confuse with LLM tool:

Expand All @@ -170,11 +172,11 @@ See [example](https://github.com/google/A2UI/blob/main/specification/v0_9/json/c

### Action

A string that explains to the AI what should be done.
A string that explains to the agent what should be done.

It may be an alias (like “option1”) or detailed explanation (like “order three pounds of ice cream of different flavors for a kids party”).

See [detailed guide on actions](https://github.com/google/A2UI/blob/main/docs/concepts/client_to_server_actions.md).
See [detailed guide on actions](https://github.com/google/A2UI/blob/main/docs/concepts/actions.md).

## Generative UI terms

Expand Down Expand Up @@ -204,4 +206,4 @@ Information, categorized as **not accessible by AI** (for example, credit card i

Which information should not be accessible by AI is defined by owners of the application and it is **different in different contexts**. For example, in some contexts medical history should never go to AI, while in others AI is heavily used to help with medical diagnostics and thus needs medical history.

This term is important in GenUI context, because end users want to **clearly see** what their input is allowed to go to AI and which is not allowed.
This term is important in the GenUI context, because end users want to **clearly see** what their input is allowed to go to the AI and what is not allowed.
6 changes: 3 additions & 3 deletions docs/guides/theming.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@ Customize the look and feel of A2UI components to match your brand.

## The A2UI Styling Philosophy

A2UI follows a **client-controlled styling** approach:
A2UI follows a **renderer-controlled styling** approach:

- **Agents describe *what* to show** (components and structure)
- **Clients decide *how* it looks** (colors, fonts, spacing)
- **Renderers decide *how* it looks** (colors, fonts, spacing)

This ensures:

Expand All @@ -32,7 +32,7 @@ flowchart TD

## Layer 1: Semantic Hints

Agents provide semantic hints (not visual styles) to guide client rendering:
Agents provide semantic hints (not visual styles) to guide rendering:

```json
{
Expand Down
4 changes: 2 additions & 2 deletions mkdocs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ nav:
- Data Binding: concepts/data-binding.md
- Catalogs: concepts/catalogs.md
- Transports: concepts/transports.md
- Client-to-Server Actions: concepts/client_to_server_actions.md
- Actions: concepts/actions.md
- Guides:
- Client Setup: guides/client-setup.md
- Agent Development: guides/agent-development.md
Expand Down Expand Up @@ -191,7 +191,7 @@ plugins:
# Files moved in docs-file-org reorganization
'catalogs.md': 'concepts/catalogs.md'
'transports.md': 'concepts/transports.md'
'client_to_server_actions.md': 'concepts/client_to_server_actions.md'
'actions.md': 'concepts/actions.md'
'renderers.md': 'reference/renderers.md'
'agents.md': 'reference/agents.md'
'community.md': 'ecosystem/community.md'
Expand Down
Loading
Loading