Skip to content

Commit eeb23aa

Browse files
stewartjarodclaude
stewartjarod
and
claude
committed
fix(inbound): keep email.received events under the EventBridge size limit
The inbound Lambda put the whole parsed email in the EventBridge detail. A large mail could exceed the PutEvents entry limit (1 MB; 256 KB in ap-southeast-5/6/7, ap-east-2, mx-central-1 and GovCloud), so the Lambda failed and retried while the mail never reached the consumer. Oversized details now drop html, text and headers (then attachments, then recipients) with truncated, truncatedFields and parsedS3Key, so consumers can fetch the full parsed JSON from S3. Small mail is sent unchanged. A FailedEntryCount > 0 response now throws so the Lambda retries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015wXo12iE7EnsbdjDvgqR1B
1 parent 2eb59f0 commit eeb23aa

6 files changed

Lines changed: 699 additions & 60 deletions

File tree

‎apps/website/src/app/docs/infrastructure/events/page-content.tsx‎

Lines changed: 101 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1562,7 +1562,7 @@ export default function InfrastructureEventsPageContent() {
15621562
Email subject line
15631563
</td>
15641564
</tr>
1565-
<tr>
1565+
<tr className="border-b">
15661566
<td className="p-3">
15671567
<code className="rounded bg-muted px-1 py-0.5">
15681568
detail.receivedAt
@@ -1572,6 +1572,41 @@ export default function InfrastructureEventsPageContent() {
15721572
When the inbound email was received
15731573
</td>
15741574
</tr>
1575+
<tr className="border-b">
1576+
<td className="p-3">
1577+
<code className="rounded bg-muted px-1 py-0.5">
1578+
detail.rawS3Key
1579+
</code>
1580+
</td>
1581+
<td className="p-3 text-muted-foreground">
1582+
S3 key of the raw MIME message in your inbound
1583+
bucket
1584+
</td>
1585+
</tr>
1586+
<tr className="border-b">
1587+
<td className="p-3">
1588+
<code className="rounded bg-muted px-1 py-0.5">
1589+
detail.truncated
1590+
</code>
1591+
</td>
1592+
<td className="p-3 text-muted-foreground">
1593+
Present and <code>true</code> only when large fields
1594+
were dropped to fit the EventBridge size limit;{" "}
1595+
<code>detail.truncatedFields</code> lists them
1596+
</td>
1597+
</tr>
1598+
<tr>
1599+
<td className="p-3">
1600+
<code className="rounded bg-muted px-1 py-0.5">
1601+
detail.parsedS3Key
1602+
</code>
1603+
</td>
1604+
<td className="p-3 text-muted-foreground">
1605+
Set on truncated events: S3 key of the full parsed
1606+
email JSON, in the same bucket as{" "}
1607+
<code>rawS3Key</code>
1608+
</td>
1609+
</tr>
15751610
</tbody>
15761611
</table>
15771612
</div>
@@ -1581,6 +1616,71 @@ export default function InfrastructureEventsPageContent() {
15811616
code={inboundPayload}
15821617
filename="inbound-event.json"
15831618
/>
1619+
<p className="text-muted-foreground text-sm">
1620+
The full parsed email is always stored in your bucket at{" "}
1621+
<code className="rounded bg-muted px-1.5 py-0.5">
1622+
parsed/{"{emailId}"}.json
1623+
</code>
1624+
. If it would exceed the EventBridge PutEvents size limit (1 MB,
1625+
or 256 KB in Regions without the 1 MB increase), the event is
1626+
trimmed: the largest of{" "}
1627+
<code className="rounded bg-muted px-1.5 py-0.5">html</code>,{" "}
1628+
<code className="rounded bg-muted px-1.5 py-0.5">text</code> and{" "}
1629+
<code className="rounded bg-muted px-1.5 py-0.5">headers</code>{" "}
1630+
is dropped first, then the others, then{" "}
1631+
<code className="rounded bg-muted px-1.5 py-0.5">
1632+
attachments
1633+
</code>
1634+
, until it fits, and the detail gains{" "}
1635+
<code className="rounded bg-muted px-1.5 py-0.5">
1636+
truncated: true
1637+
</code>
1638+
,{" "}
1639+
<code className="rounded bg-muted px-1.5 py-0.5">
1640+
truncatedFields
1641+
</code>{" "}
1642+
and{" "}
1643+
<code className="rounded bg-muted px-1.5 py-0.5">
1644+
parsedS3Key
1645+
</code>
1646+
. When{" "}
1647+
<code className="rounded bg-muted px-1.5 py-0.5">
1648+
truncated
1649+
</code>{" "}
1650+
is set, read the full email from{" "}
1651+
<code className="rounded bg-muted px-1.5 py-0.5">
1652+
parsedS3Key
1653+
</code>{" "}
1654+
(or the raw MIME from{" "}
1655+
<code className="rounded bg-muted px-1.5 py-0.5">rawS3Key</code>
1656+
). If that is still too large,{" "}
1657+
<code className="rounded bg-muted px-1.5 py-0.5">to</code>/
1658+
<code className="rounded bg-muted px-1.5 py-0.5">cc</code> are
1659+
replaced by{" "}
1660+
<code className="rounded bg-muted px-1.5 py-0.5">toCount</code>/
1661+
<code className="rounded bg-muted px-1.5 py-0.5">ccCount</code>{" "}
1662+
and the subject is cut to 1,000 characters; as a last resort the
1663+
detail holds only{" "}
1664+
<code className="rounded bg-muted px-1.5 py-0.5">emailId</code>,{" "}
1665+
<code className="rounded bg-muted px-1.5 py-0.5">rawS3Key</code>
1666+
,{" "}
1667+
<code className="rounded bg-muted px-1.5 py-0.5">
1668+
parsedS3Key
1669+
</code>
1670+
,{" "}
1671+
<code className="rounded bg-muted px-1.5 py-0.5">
1672+
receivedAt
1673+
</code>
1674+
,{" "}
1675+
<code className="rounded bg-muted px-1.5 py-0.5">
1676+
replyToken
1677+
</code>{" "}
1678+
and{" "}
1679+
<code className="rounded bg-muted px-1.5 py-0.5">
1680+
autoReply
1681+
</code>
1682+
.
1683+
</p>
15841684
<p className="text-muted-foreground text-sm">
15851685
When replying to a signed message,{" "}
15861686
<code className="rounded bg-muted px-1.5 py-0.5">

‎apps/website/src/app/docs/quickstart/email/inbound/page-content.tsx‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -687,6 +687,36 @@ TTL: 300`,
687687
.
688688
</p>
689689

690+
<p className="mb-4 text-muted-foreground text-sm">
691+
Very large emails are trimmed to fit the EventBridge size limit: the
692+
event then has{" "}
693+
<code className="rounded bg-muted px-1.5 py-0.5">
694+
truncated: true
695+
</code>
696+
, drops <code className="rounded bg-muted px-1.5 py-0.5">html</code>,{" "}
697+
<code className="rounded bg-muted px-1.5 py-0.5">text</code>,{" "}
698+
<code className="rounded bg-muted px-1.5 py-0.5">headers</code> or{" "}
699+
<code className="rounded bg-muted px-1.5 py-0.5">attachments</code>{" "}
700+
(listed in{" "}
701+
<code className="rounded bg-muted px-1.5 py-0.5">
702+
truncatedFields
703+
</code>
704+
), and gives the full parsed email's S3 key in{" "}
705+
<code className="rounded bg-muted px-1.5 py-0.5">parsedS3Key</code>.
706+
Sender, recipients, subject and verdicts are kept unless they alone
707+
are too large; then{" "}
708+
<code className="rounded bg-muted px-1.5 py-0.5">to</code>/
709+
<code className="rounded bg-muted px-1.5 py-0.5">cc</code> are
710+
replaced by counts and the subject is cut, and as a last resort only{" "}
711+
<code className="rounded bg-muted px-1.5 py-0.5">emailId</code>,{" "}
712+
<code className="rounded bg-muted px-1.5 py-0.5">rawS3Key</code>,{" "}
713+
<code className="rounded bg-muted px-1.5 py-0.5">parsedS3Key</code>,{" "}
714+
<code className="rounded bg-muted px-1.5 py-0.5">receivedAt</code>,{" "}
715+
<code className="rounded bg-muted px-1.5 py-0.5">replyToken</code> and{" "}
716+
<code className="rounded bg-muted px-1.5 py-0.5">autoReply</code> are
717+
sent.
718+
</p>
719+
690720
<p className="mb-2 font-medium text-sm">Example Lambda Handler</p>
691721
<CodeBlock
692722
className="h-auto"

‎packages/core/CLAUDE.md‎

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -113,11 +113,18 @@ These resolve to the pre-bundled Lambda code location for Pulumi/CDK to deploy.
113113
- Uses `mailparser` to parse raw MIME
114114
- Extracts headers, body (max 200KB HTML), attachments
115115
- Uploads attachments to S3 with sanitized filenames
116-
- Publishes EventBridge event `email.received`
116+
- Stores the full parsed email at `parsed/{emailId}.json`
117+
- Publishes EventBridge event `email.received` on the default bus. If the entry
118+
would exceed the PutEvents limit (1 MB; 256 KB in the Regions listed in
119+
`LEGACY_LIMIT_REGIONS` and GovCloud), `buildEventDetail` drops `html`/`text`/`headers`
120+
(largest first), then `attachments`, then `to`/`cc` (and cuts the subject),
121+
and as a last resort sends a bounded envelope; it adds `truncated: true`,
122+
`truncatedFields` and `parsedS3Key`. Small mail is sent unchanged.
123+
- Throws when PutEvents returns `FailedEntryCount > 0`
117124
- Extracts SES spam/virus verdicts from headers
118125
- **Re-throws errors** (unlike other processors) to trigger retry/DLQ
119126

120-
**Env vars**: `PARSED_BUCKET`, `EVENT_BUS_NAME`, `WRAPS_ACCOUNT_ID`, `AWS_REGION`
127+
**Env vars**: `BUCKET_NAME`, `INBOUND_EVENT_SOURCE` (default `wraps.inbound`), `REPLY_SECRET_PARAMETER_PREFIX` (optional), `AWS_REGION`
121128

122129
## Lambda Loading Priority (in CLI)
123130

0 commit comments

Comments
 (0)