[polylane]

The hourly account-health sweep has timed out on every run since a deploy on 28 October, so customers stopped receiving SES health checks. Its memory peaked at the function's 256 MB limit in that deploy, starving it of CPU and pushing a three-minute sweep past the ten-minute timeout. After this change it loads notification code only when it notifies and has room to finish.

flowchart TD
  A["byo-notifications deploy 2026-10-28 05:31Z"] --> B["notifications-config imports @wraps/email"]
  B --> C["account-health statically imports @wraps/auth/notify"]
  C --> D["cold start evaluates notify + auth + email graph"]
  D --> E["Max Memory Used 194 MB to 256 MB"]
  E --> F["256 MB function pinned at ceiling, CPU-starved"]
  F --> G["29-account sweep 173 s to over 600 s"]
  G --> H["Status: timeout, 2-3 errors per hour"]
  H --> I["account-wide Lambda Errors alarm fires"]
  J["Fix: lazy-import notify"] -.-> D
  K["Fix: 1024 MB headroom and 4x CPU"] -.-> F

Loading

Why this fix

CloudWatch shows the sweep's maximum duration stepping from 156-211 s to the 600 s timeout at the 28 October deploy, while peak memory moved from 194 MB to the 256 MB ceiling on an unchanged configuration. That deploy connected the notification code to the SES client and renderer, and the sweep loaded it on every cold start. Loading it lazily and raising memory to 1024 MB restores headroom and quadruples CPU, so the sweep finishes inside its timeout. The heavier notify path is covered by the extra memory, and the sweep's completion log will confirm recovery.

Out of scope

• Other cron workers still import the notify module statically; only the long-running sweep reached the ceiling.
• Why the alarm stayed OK until 16:07 despite earlier error datapoints.

Production impact: Only the account-health cron changes; other cron jobs keep their config

wraps-production-AccountHealthHandlerFunction-bcxerths is the only function this diff changes: its Maximum Duration was 156-211 s for a week and then 600000 ms on every run from 2026-10-28T05:00Z, with Max Memory Used at 255-256 MB against a 256 MB configuration. The edited worker file is bundled only into that function, and only that job's memory value changes in infra/cron.ts.

• The twelve cron and subscriber functions in the same file (AuditLogCleanup, BroadcastReaper, EventFeedStaleness, MessageSendCleanup, NotificationDeliveries, WorkflowReaper, MarketplaceEvents queue and DLQ, Workflow queue and DLQ, Batch queue and DLQ) keep their declared configuration; I read their Errors metric for the alarm window and all were zero.
• ApiHandlerFunction is invoked by API Gateway and reaches Neon, Sentry and app.wraps.dev. It does not import the account-health worker, and its single 30 s timeout at 16:05:53Z predates this diff.
• Nothing removes a binding, endpoint, column or serialization format, so a rolling deploy has no mixed-version window.

Affected resources: arn:aws:lambda:us-east-1:905130073023:function:wraps-production-AccountHealthHandlerFunction-bcxerths.

Checked and not affected: arn:aws:lambda:us-east-1:905130073023:function:wraps-production-ApiHandlerFunction-szkssafu, arn:aws:lambda:us-east-1:905130073023:function:MarketplaceEventsQueueSubscriberWdnrfwFunctionFunction-kbbwcxkn, arn:aws:lambda:us-east-1:905130073023:function:pr-MarketplaceEventsDlqSubscriberMdosonFunctionFunction-bbehwxns, arn:aws:lambda:us-east-1:905130073023:function:productio-WorkflowQueueSubscriberMnahuuFunctionFunction-sskcosms, arn:aws:lambda:us-east-1:905130073023:function:production-WorkflowDlqSubscriberBbrurmFunctionFunction-ndksofav, arn:aws:lambda:us-east-1:905130073023:function:w-production-BatchQueueSubscriberEwchhwFunctionFunction-caerssmd, arn:aws:lambda:us-east-1:905130073023:function:wra-production-BatchDlqSubscriberXmtfhdFunctionFunction-sbssczbc, arn:aws:lambda:us-east-1:905130073023:function:wraps-production-AuditLogCleanupHandlerFunction-xscaudmf, arn:aws:lambda:us-east-1:905130073023:function:wraps-production-BroadcastReaperHandlerFunction-sdfnrbux, arn:aws:lambda:us-east-1:905130073023:function:wraps-production-EventFeedStalenessHandlerFunction-kwxacmcz, arn:aws:lambda:us-east-1:905130073023:function:wraps-production-MessageSendCleanupHandlerFunction-vnszhmta, arn:aws:lambda:us-east-1:905130073023:function:wraps-production-NotificationDeliveriesHandlerFunction-bcffmcch, arn:aws:lambda:us-east-1:905130073023:function:wraps-production-WorkflowReaperHandlerFunction-knutfdmz.

Observability

• CloudWatch Logs /aws/lambda/wraps-production-AccountHealthHandlerFunction-bcbtnuto: the hourly [account-health] Sweep complete line with accountsTotal/checkedCount is the signal that the sweep finished.
• CloudWatch AWS/Lambda Duration (Maximum) and Errors for the same function: Duration should return from 600000 ms to a few minutes and the hourly error datapoints should stop.
• REPORT lines carry Status: timeout and Max Memory Used; both are existing signals, unchanged by this diff.

Polylane assessed this pull request at 4141c71.

Causal chain

• Signal (metric): AWS/Lambda Errors (no dimensions), Sum over 300 s, threshold 0
• Surfacing site: wraps-production-AccountHealthHandlerFunction-bcxerths at aws.lambda.function in 905130073023/us-east-1; completion REPORT logged as Status: timeout
• Mechanism: The hourly sweep's resident memory rose from 194 MB to 256 MB against an unchanged 256 MB configuration, leaving the runtime at its hard ceiling where garbage collection consumes most of its CPU; the 29-account serial sweep then exceeds the 10-minute Lambda timeout on every invocation, and each timed-out run is recorded as one invocation Error.
• Producer: AccountHealthHandlerFunction worker, at wraps-team/wraps:apps/api/src/workers/account-health.ts#handler
• Trigger: Deploy of 5cf47b6 at 2026-10-28T05:31Z (byo-notifications notification-email work)
• What happens to the failed unit today: The sweep catches per-account failures so one broken role cannot abort it, and closes the DB connection in a finally block; the invocation itself is not bounded, so elapsed time past the Lambda timeout is fatal rather than degraded.
• Cadence check: Hourly schedule cron(45 * * * ? *) plus Lambda retries produced 2-3 six-hundred-second invocations per hour from 05:45Z.
• Blast radius: 1 other resource(s), unknown other tenant(s); data at risk: No rows are lost, but accounts after the timeout point never receive health verdicts, so SES sending-paused, reputation, quota and production-access states are not evaluated or notified for them.
• Producer evidence:
  • Hourly max Duration stepped from 156-211 s to 600 s at 2026-10-28T05:00Z, the deploy of 5cf47b6.
  • REPORT Max Memory Used moved 194-195 MB to 255-256 MB on the same 256 MB configuration; Init Duration 1050 ms to 1710 ms.
  • 5cf47b6 added packages/auth/src/notification-email.ts and connected notifications-config.ts to @wraps/email, which the worker loads via a static import of @wraps/auth/notify.
  • The worker still completes 29 accounts via accountsTotal 29 on 2026-10-27, so the account count did not grow.

3 files changed (+14/-3)

• apps/api/src/workers/account-health.ts: modified, +6/-1
• infra/cron.ts: modified, +6/-1
• infra/selfhost.config.ts: modified, +2/-1

---

Generated by Polylane. You can ask follow-ups by mentioning @polylane in a comment.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
wraps-web	Ready	Preview	Oct 28, 2026 4:31pm UTC
wraps-website	Ready	Preview	Oct 28, 2026 4:31pm UTC