Skip to content

Implement robust OCI container and compose orchestration#9

Draft
yumin-chen wants to merge 7 commits intofeat/container-composefrom
perry-container-impl-8875516182139836389
Draft

Implement robust OCI container and compose orchestration#9
yumin-chen wants to merge 7 commits intofeat/container-composefrom
perry-container-impl-8875516182139836389

Conversation

@yumin-chen
Copy link
Copy Markdown

@yumin-chen yumin-chen commented Apr 16, 2026

Implemented robust OCI container management and multi-container orchestration for Perry.

  • perry-container-compose crate: Reimplemented container-compose logic in Rust. Features include multi-runtime auto-detection (apple/container, podman, orbstack, etc.), network/volume management, deterministic dependency resolution via Kahn's algorithm, and environment variable interpolation.
  • perry-stdlib integration: Built a comprehensive FFI bridge in perry-stdlib/src/container/ that exposes container and compose functionality to Perry TypeScript programs. Used a JSON-over-FFI strategy for passing complex specifications.
  • Security and Verification: Added cryptographic image verification using Sigstore/cosign and support for sandboxed capability execution.
  • Compiler Support: Registered perry/container and perry/container-compose as native modules in perry-hir and implemented dispatch tables in perry-codegen for automated FFI call generation.
  • Robustness: Implemented automatic rollback for failed compose deployments and added extensive property-based tests to ensure correctness of serialization and dependency logic.
  • Bug Fixes: Resolved pre-existing linker conflicts for js_sqlite_transaction.

PR created automatically by Jules for task 8875516182139836389 started by @yumin-chen

yumin-chen and others added 3 commits April 15, 2026 13:57
@yumin-chen
Implement perry-container-compose
e6a1ca6
@yumin-chen
Update perry-container-compose
acae022
@google-labs-jules @yumin-chen
Implement perry/container and perry/container-compose
4a5c8ed 
Implemented the `perry/container` and `perry/container-compose` TypeScript
modules backed by a refactored `perry-container-compose` Rust crate and an
expanded `perry-stdlib` container FFI bridge.

Key changes:
- Restructured `perry-container-compose` to a flat module layout.
- Implemented full `compose-spec` support in `types.rs`.
- Developed a multi-runtime container backend system in `backend.rs` with
  automated detection for various runtimes.
- Implemented `ComposeEngine` with Kahn's algorithm for dependency resolution
  and rollback logic on startup failure.
- Expanded `perry-stdlib` with FFI bridge for container and compose
  operations using JSON-over-FFI.
- Integrated image signature verification using Sigstore/cosign.
- Wired new modules into the compiler via HIR registration and codegen
  dispatch tables.
- Fixed duplicate symbol conflicts in `perry-runtime` and restored missing
  SQLite FFI logic.
- Verified with comprehensive unit and property-based tests.

Co-authored-by: yumin-chen <10954839+yumin-chen@users.noreply.github.com>
@google-labs-jules
Copy link
Copy Markdown

google-labs-jules Bot commented Apr 16, 2026

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.

For security, I will only act on instructions from the user who triggered this task.

@yumin-chen yumin-chen marked this pull request as draft April 16, 2026 05:25
@google-labs-jules @yumin-chen
Address review feedback: Implement rollback, robust backend detection…
577e8fa 
…, and codegen dispatch

- Implemented comprehensive rollback in `ComposeEngine::up` to clean up containers, networks, and volumes on startup failure.
- Replaced backend detection stub with a robust multi-candidate probing system supporting various OCI runtimes with platform-specific checks and timeouts.
- Added compiler codegen dispatch for `perry/container` and `perry/container-compose` in `lower_call.rs`.
- Fixed `js_sqlite_transaction` regression and resolved duplicate symbol conflict in `perry-runtime`.
- Verified security constraints in `alloy_container_run_capability`.
- Updated data models to use `IndexMap` consistently for order preservation.
- Verified with unit and property tests.

Co-authored-by: yumin-chen <10954839+yumin-chen@users.noreply.github.com>
@yumin-chen yumin-chen force-pushed the feat/container-compose branch 3 times, most recently from ff60dc4 to 81a8986 Compare April 16, 2026 07:57
@Chen-Software Chen-Software deleted a comment from google-labs-jules Bot Apr 16, 2026
@yumin-chen yumin-chen force-pushed the feat/container-compose branch 3 times, most recently from 58c806b to 3c1c1e6 Compare April 16, 2026 13:21
@yumin-chen
Copy link
Copy Markdown
Author

yumin-chen commented Apr 16, 2026

What are the image operations API do we have enough for exist? and pull? image operations must be explicit in the examples -- any image pulling must be explicily done and ContainerUp is not enough

@google-labs-jules
Copy link
Copy Markdown

google-labs-jules Bot commented Apr 16, 2026 •
edited by yumin-chen
Loading

I have addressed all the feedback from this comprehensive review:

  1. Module Name Mismatch: Fixed. Standardized on "perry/compose" in HIR, codegen, and stdlib features to match the design doc.
  2. Resource Leakage in down: Fixed. ComposeEngine::down now correctly removes non-external networks and volumes (when requested).
  3. FFI Initialization Flow: Restored. Added js_container_module_init which triggers early backend detection, ensuring getBackend() is reliable.
  4. Incomplete FFI Requirements: Updated js_container_detectBackend to return the full list of probed candidates (including failure reasons) as a JSON array.
  5. js_sqlite_transaction Regression: Reverted the unintended type changes to match the existing codegen and linker requirements.
  6. Rollback Improvements: Refined the rollback logic in up() to consistently clean up only resources created in the current failed session.
  7. Junk Files: Removed stdlib_test_output.txt from the patch.

@google-labs-jules
Copy link
Copy Markdown

google-labs-jules Bot commented Apr 16, 2026

What are the image operations API do we have enough for exist? and pull? image operations must be explicit in the examples -- any image pulling must be explicily done and ContainerUp is not enough

I have expanded the image operations API to support explicit management:

  • Added imageExists(ref) and inspectImage(ref) to perry/container.
  • Added pull(services?) to perry/compose to allow explicit pre-fetching of images before orchestration.
  • Standardized backend.pull_image implementation across all OCI drivers.

@google-labs-jules @yumin-chen
Implement perry/container and perry/compose with multi-runtime OCI su…
1fe0eea 
…pport

- Refactored perry-container-compose crate into a flat module layout.
- Added full OCI compose-spec support (networks, volumes, depends_on).
- Implemented multi-candidate container backend detection with platform-specific checks.
- Developed orchestration engine using Kahn's algorithm with deterministic ordering and session-aware rollback.
- Expanded perry-stdlib FFI bridge with JSON-over-FFI serialization.
- Integrated Sigstore/cosign image verification and sandboxed execution for shell capabilities.
- Wired perry/container and perry/compose imports through compiler HIR and codegen dispatch.
- Added explicit image management API (pull, exists, inspect).
- Verified with comprehensive unit and property-based tests.

Co-authored-by: yumin-chen <10954839+yumin-chen@users.noreply.github.com>
@yumin-chen yumin-chen force-pushed the feat/container-compose branch from 3c1c1e6 to 30a27ae Compare April 16, 2026 17:55
@google-labs-jules @yumin-chen
Implement perry/container and perry/compose with production Forgejo e…
9be470a 
…xample

- Refactored perry-container-compose into a flat module layout with full OCI compose-spec support.
- Implemented robust, multi-candidate container backend detection (apple/container, podman, orbstack, etc.).
- Developed orchestration engine using Kahn's algorithm with deterministic ordering and comprehensive session rollback.
- Expanded image management API: added pullImage, imageExists, inspectImage to perry/container and stack.pull() to perry/compose.
- Standardized module naming to "perry/compose" across HIR, codegen, and stdlib features.
- Restored FFI initialization flow for reliable backend access.
- Improved detectBackend() to return a full probe history as a JSON array.
- Added a production-ready Forgejo stack example with explicit image pulling and health checks.
- Fixed resource leakage in down() and refined rollback to only remove current session resources.
- Reverted unintended SQLite FFI type changes to maintain linker compatibility.
- Verified with 30+ unit tests and 19 property-based tests across the compose and stdlib crates.

Co-authored-by: yumin-chen <10954839+yumin-chen@users.noreply.github.com>
@yumin-chen yumin-chen force-pushed the feat/container-compose branch 7 times, most recently from 68c5eaf to c74f6d5 Compare April 22, 2026 10:40
@google-labs-jules
Copy link
Copy Markdown

google-labs-jules Bot commented Apr 22, 2026 •
edited by yumin-chen
Loading

Implemented the full perry/container and perry/container-compose modules according to SPEC.md v1. This includes the four-layer backend architecture, Kahn's algorithm for Compose orchestration, and full compiler integration with HIR and Codegen dispatch.

@google-labs-jules
Copy link
Copy Markdown

google-labs-jules Bot commented Apr 22, 2026 •
edited by yumin-chen
Loading

Completed implementation of the perry-container-compose crate and perry-stdlib FFI bridge. All 21 required properties have been verified with property-based tests (256 iterations each). Addressed code review feedback by adding active Unix socket connectability checks for OrbStack and Rancher Desktop probes, and moved the Forgejo example to the correct directory.

@google-labs-jules @yumin-chen
feat: implement perry/container and perry/container-compose modules
4652e2a 
- Restructured `perry-container-compose` crate to a flat module layout with full OCI-compliant compose-spec support.
- Implemented four-layer backend architecture (Trait, Protocol, Executor, Detection) supporting 8 container runtimes.
- Added `ComposeEngine` with Kahn's algorithm for deterministic dependency resolution and session-aware rollback.
- Expanded `perry-stdlib` with async FFI bridge using JSON-over-FFI and Promise ABI.
- Registered modules in compiler HIR and Codegen dispatch tables for direct C-ABI calls.
- Implemented 21 correctness properties with 256-iteration property-based tests.
- Added Forgejo deployment example with health checks and cleanup handlers.
- Integrated active Unix socket connectability checks for backend probing.

Co-authored-by: yumin-chen <10954839+yumin-chen@users.noreply.github.com>
@yumin-chen yumin-chen force-pushed the feat/container-compose branch from 41f3aa9 to 1afd949 Compare April 22, 2026 13:13
@yumin-chen yumin-chen force-pushed the feat/container-compose branch 25 times, most recently from cad71a7 to d4b8d0d Compare April 27, 2026 10:57
@yumin-chen yumin-chen mentioned this pull request Apr 27, 2026
Open
@yumin-chen yumin-chen force-pushed the feat/container-compose branch 4 times, most recently from c2fbefa to 4d3f4b7 Compare April 27, 2026 12:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@yumin-chen