Skip to content

Plugin Jobs — bounded, resumable background work for sandboxed plugins - #3870

Closed
chrmoller wants to merge 1 commit into
emdash-cms:mainfrom
chrmoller:plugin-jobs
Closed

chrmoller wants to merge 1 commit into
emdash-cms:mainfrom
chrmoller:plugin-jobs

Conversation

@chrmoller

Copy link
Copy Markdown

What does this PR do?

Closes #

Type of change

  • Bug fix
  • Feature (requires maintainer-approved Discussion)
  • Refactor (no behavior change)
  • Translation
  • Documentation
  • Performance improvement
  • Tests
  • Chore (dependencies, CI, tooling)

Checklist

  • I have read CONTRIBUTING.md
  • pnpm typecheck passes
  • pnpm lint passes
  • pnpm test passes (or targeted tests for my change)
  • pnpm format has been run
  • I have added/updated tests for my changes (if applicable)
  • User-visible strings in the admin UI are wrapped for translation (if applicable). Do not include messages.po changes except in translation PRs — a workflow extracts catalogs on merge to main.
  • I have added and reviewed the user-facing changeset (if this PR changes a published package)
  • New features link to an approved Discussion: https://github.com/emdash-cms/emdash/discussions/...
  • I have included screenshots below if this PR changes the UI

AI-generated code disclosure

  • This PR includes AI-generated code — model/tool:

Screenshots / test output

@changeset-bot

changeset-bot Bot commented Oct 5, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: df49152

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@github-actions github-actions Bot added the review/needs-review No maintainer or bot review yet label Oct 5, 2026
@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

PR template validation failed

Please fix the following issues by editing your PR description:

  • Fill out the "What does this PR do?" section with a description of your change.
  • Check at least one "Type of change" checkbox.
  • Check the "I have read CONTRIBUTING.md" checkbox.

See CONTRIBUTING.md for the full contribution policy.

@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor


Thank you for your submission, we really appreciate it. Like many open-source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution. You can sign the CLA by just posting a Pull Request Comment same as the below format.


I have read the CLA Document and I hereby sign the CLA


You can retrigger this bot by commenting recheck in this Pull Request. Posted by the CLA Assistant Lite bot.

@github-actions

github-actions Bot commented Oct 5, 2026

Copy link
Copy Markdown
Contributor

Scope check

This PR changes 509 lines across 1 files. Large PRs are harder to review and more likely to be closed without review.

If this scope is intentional, no action needed. A maintainer will review it. If not, please consider splitting this into smaller PRs.

See CONTRIBUTING.md for contribution guidelines.

@emdashbot emdashbot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reviewed PR #3870 as an RFC-only change. I checked AGENTS.md, rfcs/README.md, and the relevant implementation files (packages/core/src/plugins/sandbox/types.ts, packages/core/src/plugins/sandbox/runner-options.ts, packages/cloudflare/src/sandbox/runner.ts, packages/core/src/astro/middleware.ts, packages/cloudflare/src/worker.ts, packages/core/src/plugins/scheduler/node.ts, packages/plugin-types/src/routes.ts, and packages/plugin-types/src/manifest-schema.ts).

Headline: the RFC is in the wrong place and contains one inaccurate technical reference. The resource-limit defaults it quotes (50 ms CPU, 10 subrequests, 30 s wall time) do match the current ResourceLimits defaults in packages/core/src/plugins/sandbox/types.ts. The route request: { body: "text", maxBytes: ... } example matches the plugin route request schema. The fallback scheduler references (NodeCronScheduler on Node, scheduled() on Cloudflare) exist. The proposed table name _emdash_plugin_jobs follows the plural system-table convention used elsewhere.

Two things need correction before this should land:

  1. The file must be moved from the repo root to rfcs/0003-plugin-jobs.md per rfcs/README.md.
  2. The helper named as the source of default limits is misidentified; it lives in a different file and does not itself apply defaults.

I’m leaving a comment rather than requesting changes only because the PR introduces no code/build regression, but both findings are real process/accuracy issues that should be resolved.

Comment thread 0003-plugin-jobs.md
@@ -0,0 +1,509 @@
---

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[needs fixing] The RFC is added at the repository root (0003-plugin-jobs.md), but rfcs/README.md requires every RFC to be a new file in rfcs/ named NNNN-short-slug.md. Merging it here would leave the RFC outside the tracked RFC directory and break the numbering/project conventions the repo enforces.

Suggested change
---
Move/add the file as `rfcs/0003-plugin-jobs.md` instead of `0003-plugin-jobs.md` at the repo root.

Comment thread 0003-plugin-jobs.md
| Subrequests | 10 | Worker Loader, documented in `WorkerLoaderLimits` as "fetch/service-binding calls" |
| Wall time | 30 s | runner `Promise.race` |

The host constructs the runner without passing limits (`createSandboxRunnerOptions` in `emdash-runtime.ts`), so every sandboxed plugin gets these defaults. Each `ctx.storage`, `ctx.kv` and settings call crosses the `PluginBridge` loopback service binding. Cloudflare's Service Bindings documentation states that "each request to a Worker via a Service binding counts toward your subrequest limit", and that "a single request has a maximum of 32 Worker invocations, and each call to a Service binding counts towards this limit."

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[needs fixing] This line says the host calls createSandboxRunnerOptions in emdash-runtime.ts to construct the runner and that this is where the default limits originate. In the current codebase, createSandboxRunnerOptions is defined in packages/core/src/plugins/sandbox/runner-options.ts (it is only imported and called from packages/core/src/emdash-runtime.ts). Moreover, that helper only normalizes siteInfo; it does not supply the default ResourceLimits. The defaults are applied by the platform-specific runner, e.g. DEFAULT_LIMITS in packages/cloudflare/src/sandbox/runner.ts.

Suggested change
The host constructs the runner without passing limits (`createSandboxRunnerOptions` in `emdash-runtime.ts`), so every sandboxed plugin gets these defaults. Each `ctx.storage`, `ctx.kv` and settings call crosses the `PluginBridge` loopback service binding. Cloudflare's Service Bindings documentation states that "each request to a Worker via a Service binding counts toward your subrequest limit", and that "a single request has a maximum of 32 Worker invocations, and each call to a Service binding counts towards this limit."
The host constructs the runner without passing limits (`createSandboxRunnerOptions` in `packages/core/src/plugins/sandbox/runner-options.ts`), so every sandboxed plugin gets the defaults supplied by the platform runner (e.g. `DEFAULT_LIMITS` in `packages/cloudflare/src/sandbox/runner.ts`).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla: needed review/needs-review No maintainer or bot review yet size/XL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant