Skip to content
Open
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions specs/activity/protocol-activity.md
Original file line number Diff line number Diff line change
Expand Up @@ -1952,6 +1952,7 @@ Activity [entities](#entity) communicate extra metadata about the activity, such
| number | N/A | Number |
| clientInfo | N/A | Skype client info |
| streamInfo | N/A | Streaming text metadata |
| context | N/A | Query context details |
Comment thread
sveluvil marked this conversation as resolved.
Outdated

### string and number

Expand Down Expand Up @@ -2200,6 +2201,32 @@ Voice streaming uses `event` activities with [Media.* events](#reserved-events-f
}
```

### context
Comment thread
sveluvil marked this conversation as resolved.
The context entity contains contextual information between users and agents to improve understanding of ongoing conversations. This enables more relevant, personalized, and accurate responses, especially in multimodal interactions. By embedding context directly in the activity message, agents can leverage contextual information such as images, custom objects, and audio/video-related context.

The context entity in an activity message can contain a `screenContext` object. This can be a list of screenshots of the page the user is currently on. The images are shared either in base64-encoded format or as a URL.
Comment thread
sveluvil marked this conversation as resolved.
Comment thread
sveluvil marked this conversation as resolved.

`A9263`: If the context contains no elements, it should be ignored.
Comment thread
sveluvil marked this conversation as resolved.
Outdated
Comment thread
sveluvil marked this conversation as resolved.
Outdated

`A9264`: If the context entity contains any syntactic errors, it should be ignored by the recipient to prevent processing invalid data.

```
Comment thread
sveluvil marked this conversation as resolved.
// Sending context information as part of the activity message
Comment thread
sveluvil marked this conversation as resolved.
{
"type": "message",
"text": "What pizzas are available for $150?",
"entities": [
{
"type": "context",
"screenContext": {
"images": ["iVBORw0KGgoAAAANSUhEUgAAAAUA...", "https://example.com/product-page"] // a list of base64-encoded image or URL that should be used as context.
Comment thread
sveluvil marked this conversation as resolved.
Outdated
}
}
]
}
```


# Appendix III - Protocols using the Invoke activity

The [invoke activity](#invoke-activity) is designed for use only within protocols supported by Activity Protocol channels (i.e., it is not a generic extensibility mechanism). This appendix contains a list of all protocols using this activity.
Expand Down
Loading