[Components] speak_ai - add instant-hook sources and AI Chat / Find Media actions#21311
[Components] speak_ai - add instant-hook sources and AI Chat / Find Media actions#21311speakai wants to merge 12 commits into
Conversation
Additive upgrade to the existing speak_ai app. Adds run-magic-prompt and find-media actions and new-transcription/new-recording/new-captions/ new-magic-prompt/new-sentiment sources, and points read helpers at the neutral /v1/apps/* endpoints. Existing components and methods unchanged.
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
dannyroosevelt
moved this to Ready for PR Review
in Component (Source and Action) Backlog
|
Thank you so much for submitting this! We've added it to our backlog to review, and our team has been notified. |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThis PR adds Speak AI app helpers, two actions, a shared webhook source base, five instant sources, supporting test fixtures, and a package version bump. ChangesSpeak AI Components Addition
Estimated code review effort: 3 (Moderate) | ~30 minutes Sequence Diagram(s)sequenceDiagram
participant Caller as Action caller
participant App as speak_ai app
participant API as Speak AI API
Caller->>App: runPrompt or getInsights
App->>API: POST /prompt or apps request
API-->>App: response data
App->>App: firstResult(results, fallback)
App-->>Caller: response / resolved item
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@components/speak_ai/actions/run-magic-prompt/run-magic-prompt.mjs`:
- Around line 47-70: The summary in run-magic-prompt.mjs can render empty
backticks when response.data.promptId is missing because run() falls back to an
empty string. Update the $.export("$summary", ...) in run() to use a safer
fallback or conditional message so it never emits `` when promptId is absent,
and keep the logic tied to response.data.promptId.
In `@components/speak_ai/sources/common/base.mjs`:
- Around line 58-60: The `run()` method in `Base` currently forwards every
`body` directly into `processResource()`, which can break on empty or
ping/verification payloads. Add an early guard in `run()` (or before
`processResource()` is called) to detect missing or minimal webhook bodies and
safely return without calling `hydrate()` or `generateMeta()`. Keep the fix
localized to `run()` and `processResource()` so `resource.eventType` and
`resource.deliveryId` are only read for real Speak AI event payloads.
- Around line 46-52: The shared meta builder in generateMeta currently stamps
every event with Date.now(), which ignores the payload’s own timing. Update
generateMeta in base.mjs to prefer the event/resource timestamp field from the
incoming data (using the payload value when present) and only fall back to
Date.now() when no timestamp exists, keeping the id and summary behavior
unchanged.
In `@components/speak_ai/sources/new-magic-prompt/new-magic-prompt.mjs`:
- Around line 24-31: The hydrate flow in new-magic-prompt.mjs is only fetching
the first 25 history records, so a matching prompt can be missed and the code
falls back to the raw webhook payload. Update the history lookup in hydrate() to
either request more entries via getPromptsHistory or paginate until the target
promptId/messageId is found or the history is exhausted, so the matched record
can include the full stored fields like answer and assistantType.
In `@components/speak_ai/sources/new-recording/new-recording.mjs`:
- Around line 23-36: The result normalization logic in hydrate and the related
sources/actions is duplicated, so extract the repeated Array.isArray(results) ?
results[0] : results plus fallback behavior into a shared helper such as
_firstInsight(results, fallback). Add it in a common location like
common/base.mjs or the app file, then update new-recording.mjs,
new-sentiment.mjs, new-transcription.mjs, and find-media.mjs to call the helper
instead of inlining the pattern.
In `@components/speak_ai/speak_ai.app.mjs`:
- Around line 66-71: The mediaIds prop description in speak_ai.app.mjs is too
vague and should explain both the expected ID format and where to obtain those
values. Update the mediaIds field description to mention that it accepts one or
more media record identifiers, and point agents to the source of those IDs such
as getInsights or Find Media results, with a concrete inline example of the kind
of ID value expected.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: ff8313ab-48ab-44c4-8f3c-ff32c83ee7aa
📒 Files selected for processing (15)
components/speak_ai/actions/find-media/find-media.mjscomponents/speak_ai/actions/run-magic-prompt/run-magic-prompt.mjscomponents/speak_ai/package.jsoncomponents/speak_ai/sources/common/base.mjscomponents/speak_ai/sources/new-captions/new-captions.mjscomponents/speak_ai/sources/new-captions/test-event.mjscomponents/speak_ai/sources/new-magic-prompt/new-magic-prompt.mjscomponents/speak_ai/sources/new-magic-prompt/test-event.mjscomponents/speak_ai/sources/new-recording/new-recording.mjscomponents/speak_ai/sources/new-recording/test-event.mjscomponents/speak_ai/sources/new-sentiment/new-sentiment.mjscomponents/speak_ai/sources/new-sentiment/test-event.mjscomponents/speak_ai/sources/new-transcription/new-transcription.mjscomponents/speak_ai/sources/new-transcription/test-event.mjscomponents/speak_ai/speak_ai.app.mjs
| generateMeta(resource, data) { | ||
| return { | ||
| id: resource.deliveryId || resource.mediaId || resource.messageId, | ||
| summary: this.getSummary(resource, data), | ||
| ts: Date.now(), | ||
| }; | ||
| }, |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🌐 Web query:
Speak AI webhook payload documentation event fields timestamp deliveryId
💡 Result:
Speak AI provides webhook functionality to support event-driven architectures, specifically for notifying your application when tasks such as transcription and analysis are complete [1][2]. To integrate with Speak AI webhooks, you must register your endpoint using the Speak AI REST API (specifically POST /v1/webhook) [3]. While Speak AI's developer portal at docs.speakai.co serves as the primary authoritative source for specific payload schemas [4][2], developers typically implement these webhooks to handle events like media.analyzed [3]. General industry practice for webhook payloads—and features often found in such integrations—include the following fields: 1. Event Type: A string identifying the specific event that triggered the webhook (e.g., media.analyzed) [3]. 2. Timestamp: A field indicating when the event occurred, often formatted as an ISO 8601 string [5][6]. 3. DeliveryId: A unique identifier for the specific webhook delivery attempt, which is commonly used to ensure idempotency and handle retries [5][6]. For the most accurate and up-to-date schema definitions, including exact field names and nested data structures for specific events, refer directly to the Speak AI documentation at https://docs.speakai.co [4][2].
Citations:
- 1: https://help.speakai.co/en/articles/6150175-how-to-set-up-and-use-webhooks-in-speak-ai
- 2: https://speakai.co/ca/developers/
- 3: https://speakai.co/integrations/slack/
- 4: https://docs.speakai.co/
- 5: https://hyperleap.ai/help/bot-studio/webhooks
- 6: https://www.syllabi-ai.com/docs/api-reference/webhooks
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Locate the shared base and the Speak AI sources that extend it
git ls-files 'components/speak_ai/**' | sed -n '1,200p'
echo '--- base outline ---'
ast-grep outline components/speak_ai/sources/common/base.mjs --view expanded
echo '--- source outlines ---'
for f in $(git ls-files 'components/speak_ai/sources/**/*.mjs'); do
echo "### $f"
ast-grep outline "$f" --view expanded
doneRepository: PipedreamHQ/pipedream
Length of output: 4199
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect the shared base and all Speak AI source files for generateMeta usage and timestamp/id fields
for f in components/speak_ai/sources/common/base.mjs $(git ls-files 'components/speak_ai/sources/**/*.mjs' | grep -v '^components/speak_ai/sources/common/base.mjs$'); do
echo "===== $f ====="
wc -l "$f"
rg -n "generateMeta|getSummary|getEvents|dedupe|emit|timestamp|createdAt|deliveryId|mediaId|messageId|promptId|eventTime|Date\.now" "$f" -n -C 3 || true
echo
doneRepository: PipedreamHQ/pipedream
Length of output: 13438
🌐 Web query:
Speak AI webhook payload messageId promptId deliveryId mediaId timestamp event schema
💡 Result:
Speak AI webhooks are designed to notify external systems of specific events occurring within the Speak AI platform, most notably the media.analyzed event [1][2]. When a configured event is triggered, Speak AI sends an HTTP POST request containing a JSON payload to your specified callback URL [1]. While the exact structure of the payload can depend on the event type, standard integrations typically utilize the mediaId to retrieve comprehensive details about the analysis through the Speak AI REST API [2]. Core components frequently involved in Speak AI webhook and data integration workflows include: mediaId: A unique identifier for the processed media file, used to fetch full insight data, transcriptions, and sentiment analysis results via the API [2][3][4]. event: Specifies the type of occurrence, such as media.analyzed, which indicates that transcription and AI processing are complete [1][2]. timestamp: Included in the event data to provide the exact time the event occurred or was processed [1]. To implement a robust webhook handler, you should verify the requests using your Speak AI webhook secret to ensure they originate from the platform [5]. Because webhooks often serve as notifications, the recommended pattern for accessing full data is to use the mediaId provided in the webhook payload to make a secondary authenticated request to the Speak AI API [2]. This ensures you receive the complete, structured analysis rather than relying solely on the data contained in the initial webhook event [2][3]. For detailed API documentation regarding endpoints and specific response schemas, developers should refer to the official Speak AI documentation [6][7]. Webhooks can be registered via the API by providing your callback URL and the list of events you wish to subscribe to [2].
Citations:
- 1: https://help.speakai.co/en/articles/6150175-how-to-set-up-and-use-webhooks-in-speak-ai
- 2: https://speakai.co/integrations/slack/
- 3: https://help.speakai.co/en/articles/14294698-how-to-connect-speak-ai-with-zapier
- 4: https://speakai.co/integrations/notion/
- 5: https://eliteai.tools/agent-skills/speak-deploy-integration-3
- 6: https://docs.speakai.co/
- 7: https://help.speakai.co/en/articles/6150141-how-to-get-your-api-key
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Search the Speak AI app for event-field assumptions and any mention of timestamp/id shapes
rg -n "promptId|messageId|mediaId|deliveryId|eventTime|createdAt|timestamp|generateMeta|getSummary|dedupe" components/speak_ai .github/pipedream-*.md -C 2Repository: PipedreamHQ/pipedream
Length of output: 36344
🌐 Web query:
site:docs.speakai.co Speak AI webhook timestamp promptId messageId deliveryId
💡 Result:
The available documentation for Speak AI does not explicitly define the specific payload fields promptId, messageId, or deliveryId in relation to their webhook system [1]. While Speak AI supports webhooks to send data to your platform, the official documentation does not provide a standard schema or example payload containing these specific identifiers [1]. If you are implementing webhooks with Speak AI, you should refer directly to the webhook configuration settings within your Speak AI dashboard or the internal API logs to inspect the actual JSON structure being delivered to your endpoint.
Citations:
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Read the shared base and source headers to see which sources inherit generateMeta from base vs webhook
sed -n '1,120p' components/speak_ai/sources/common/base.mjs
echo '---'
sed -n '1,120p' components/speak_ai/sources/common/webhook.mjs
echo '---'
for f in \
components/speak_ai/sources/new-captions/new-captions.mjs \
components/speak_ai/sources/new-magic-prompt/new-magic-prompt.mjs \
components/speak_ai/sources/new-recording/new-recording.mjs \
components/speak_ai/sources/new-sentiment/new-sentiment.mjs \
components/speak_ai/sources/new-transcription/new-transcription.mjs \
components/speak_ai/sources/new-media-created-instant/new-media-created-instant.mjs \
components/speak_ai/sources/new-text-analyzed-instant/new-text-analyzed-instant.mjs
do
echo "--- $f ---"
sed -n '1,80p' "$f"
doneRepository: PipedreamHQ/pipedream
Length of output: 12189
Use the event timestamp instead of Date.now(). The shared webhook meta builder should prefer the payload’s own timestamp when present and only fall back to Date.now() if none is available; otherwise delayed deliveries will all look like they happened at ingestion time.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@components/speak_ai/sources/common/base.mjs` around lines 46 - 52, The shared
meta builder in generateMeta currently stamps every event with Date.now(), which
ignores the payload’s own timing. Update generateMeta in base.mjs to prefer the
event/resource timestamp field from the incoming data (using the payload value
when present) and only fall back to Date.now() when no timestamp exists, keeping
the id and summary behavior unchanged.
Source: Path instructions
|
@speakai - Kindly address the coderabbit review comments. |
… history lookup, mediaIds docs, DRY normalization)
|
Hi @vetrivigneshwaran — thanks for the review! We've addressed all of CodeRabbit's comments:
All of the above are in the latest commit |
…Chat Response source)
speakai
changed the title
There was a problem hiding this comment.
♻️ Duplicate comments (1)
components/speak_ai/sources/common/base.mjs (1)
46-52: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
tsstill usesDate.now()instead of the event's own timestamp.This was flagged previously and not addressed in this pass —
generateMetastill stamps every event with ingestion time rather than preferring a timestamp field fromresource/datawhen present. Delayed webhook deliveries will all appear to have occurred at the same time they were received.Additionally, the
idfallback chain (resource.deliveryId || resource.mediaId || resource.messageId) doesn't account forpromptId, which the AI Chat source (new-ai-chat.mjs) uses as its own fallback identifier whenmessageIdis absent (see itsgetSummary/hydrate). If achat.statuswebhook payload carries onlypromptId, the emittedidhere will beundefined, breakingdedupe: "unique"for that source's events.🛠️ Proposed fix
generateMeta(resource, data) { return { - id: resource.deliveryId || resource.mediaId || resource.messageId, + id: resource.deliveryId || resource.mediaId || resource.messageId || resource.promptId, summary: this.getSummary(resource, data), - ts: Date.now(), + ts: resource.timestamp || resource.eventTime || Date.now(), }; },Based on learnings, "
tsshould use the event's own timestamp from the API when available, notDate.now()" — this rule from the source guidelines is still violated here. Since Speak AI's exact webhook payload schema (whetherpromptIdis present withoutmessageId) could not be confirmed from public documentation, please verify actual payloads before finalizing theidfallback fix.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@components/speak_ai/sources/common/base.mjs` around lines 46 - 52, Update generateMeta in base.mjs so it prefers the event’s own timestamp from resource/data instead of always using Date.now(), and extend the id fallback chain to include promptId so chat.status events from new-ai-chat.mjs can still dedupe correctly. Use the existing generateMeta/getSummary flow to locate the change, and keep the timestamp/id selection defensive by checking resource first, then data, then falling back only if no event-provided value exists.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Duplicate comments:
In `@components/speak_ai/sources/common/base.mjs`:
- Around line 46-52: Update generateMeta in base.mjs so it prefers the event’s
own timestamp from resource/data instead of always using Date.now(), and extend
the id fallback chain to include promptId so chat.status events from
new-ai-chat.mjs can still dedupe correctly. Use the existing
generateMeta/getSummary flow to locate the change, and keep the timestamp/id
selection defensive by checking resource first, then data, then falling back
only if no event-provided value exists.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 3fba5ab6-a5a5-41ea-820a-8e1b53f64301
📒 Files selected for processing (10)
components/speak_ai/actions/find-media/find-media.mjscomponents/speak_ai/actions/run-ai-chat/run-ai-chat.mjscomponents/speak_ai/sources/common/base.mjscomponents/speak_ai/sources/new-ai-chat/new-ai-chat.mjscomponents/speak_ai/sources/new-ai-chat/test-event.mjscomponents/speak_ai/sources/new-captions/new-captions.mjscomponents/speak_ai/sources/new-recording/new-recording.mjscomponents/speak_ai/sources/new-sentiment/new-sentiment.mjscomponents/speak_ai/sources/new-transcription/new-transcription.mjscomponents/speak_ai/speak_ai.app.mjs
💤 Files with no reviewable changes (1)
- components/speak_ai/sources/new-ai-chat/test-event.mjs
…fallback (CodeRabbit)
|
Closed out the last remaining CodeRabbit comment on
All CodeRabbit review comments on this PR are now addressed. @vetrivigneshwaran — ready for another look whenever you have a moment. Thanks! |
6 participants
Summary
This upgrades the existing
@pipedream/speak_aiapp with a set of richer instant-hook sources and two new actions. The change is fully additive: every existing component and app method is kept byte-for-byte, and the new components layer on top of the same app file and the same registry conventions (common/constants.mjs,sources/commonbase pattern,test-event.mjs+sampleEmit, actionannotations).New sources (instant, webhook-backed)
speak_ai-new-transcriptionspeak_ai-new-captionsspeak_ai-new-sentimentspeak_ai-new-ai-chatspeak_ai-new-recordingNew actions
speak_ai-run-ai-chatreadOnlyHint: false.speak_ai-find-mediareadOnlyHint: true.Each new source registers a webhook with Speak AI when enabled and removes it when disabled, hydrates each delivery with the full insight, stores the webhook id in
db, usesdedupe: "unique", and emits viagenerateMeta. The existing sources (new-media-created-instant,new-text-analyzed-instant), actions (upload-media,analyze-text,get-transcription,list-folder-id-options), and the sharedcommon/base files are byte-identical to upstream.speak_ai.app.mjsis a strict superset — only added prop definitions and methods, nothing existing changed.Checklist
Please check the following items before your PR can be reviewed:
Versioning
0.0.1for new ones)package.json's version updated (→0.3.0)New app
If this is a new app, please submit an app integration request - the PR will only be reviewed after the app is integrated.
speak_aiapp, not a new app)CodeRabbit review
After the PR is opened, and if new changes are pushed, CodeRabbit will automatically review it. Do not 'mark as resolved' CodeRabbit's comments, but reply to them instead, whether you agree (and update the PR accordingly) or disagree.