Skip to content

latest tag points to an arm64-only manifest since v0.0.200 (exec format error on amd64) #344

Closed
paketo-buildpacks/github-config
#1450

Description

@mikrethor

Summary

Since the v0.0.200 release (2026-10-23 ~14:50 UTC), the latest tag of paketobuildpacks/builder-noble-java-tiny points to a single-arch arm64 manifest instead of a multi-arch image index. The versioned tag 0.0.200, pushed at the same time, is correctly multi-arch (amd64 + arm64).

As a result, builds that use latest on amd64 hosts fail with exec format error.

Evidence

Tag Digest Media type Platforms
latest sha256:771c7c33710f656bd0326daface2968d73b44b1f7b3a21caeaf1cedcb322c512 application/vnd.docker.distribution.manifest.v2+json linux/arm64 only
0.0.200 sha256:7c62dd7e66b277ec9fa2b60d0ad74c4e9ef15db7910dcdf51126e62a02a31463 image index linux/amd64, linux/arm64
0.0.198 sha256:be9118fdc9fa4fbfee6e8f81195b0e30b26b5a1d2660601f31a3d4f31374d455 application/vnd.oci.image.index.v1+json linux/amd64, linux/arm64

Reproduce:

docker buildx imagetools inspect paketobuildpacks/builder-noble-java-tiny:latest
# MediaType: application/vnd.docker.distribution.manifest.v2+json
# Digest:    sha256:771c7c33710f656bd0326daface2968d73b44b1f7b3a21caeaf1cedcb322c512

curl -s "https://hub.docker.com/v2/repositories/paketobuildpacks/builder-noble-java-tiny/tags?page_size=3" \
  | jq '.results[] | {name, digest, archs: [.images[].architecture]}'
# latest  -> ["arm64"]
# 0.0.200 -> ["amd64", "arm64"]

Impact

Spring Boot spring-boot-maven-plugin:4.1.1:build-image with the default builder on a GitHub-hosted ubuntu-24.04 (amd64) runner picks up the arm64 builder, pulls the arm64 run image, and the lifecycle fails:

[INFO]  > Pulled builder image 'paketobuildpacks/builder-noble-java-tiny@sha256:771c7c33710f656bd0326daface2968d73b44b1f7b3a21caeaf1cedcb322c512'
[INFO]  > Pulling run image 'docker.io/paketobuildpacks/ubuntu-noble-run-tiny:0.0.150' for platform 'linux/arm64' 100%
[INFO]  > Executing lifecycle version v0.21.26
[INFO]  > Running creator
[INFO]     [creator]     exec /cnb/lifecycle/creator: exec format error
[ERROR] ... Builder lifecycle 'creator' failed with status code 255

The same pipeline succeeded the day before with latest = 0.0.198 (multi-arch index, run image pulled for linux/amd64).

Workaround

Pin the builder to a versioned tag, e.g. paketobuildpacks/builder-noble-java-tiny:0.0.200.

Expected

latest should point to the same multi-arch image index as the newest versioned tag.

Activity

  1. JohnNiang commented on Oct 23, 2026

    @JohnNiang

    I have the same problem.

  2. wsams commented on Oct 23, 2026

    @wsams

    We added this to our pom.xml to work around the issue. This pins to 0.0.200

    <plugin>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-maven-plugin</artifactId>
        <configuration>
            <image>
                <builder>paketobuildpacks/builder-noble-java-tiny@sha256:7c62dd7e66b277ec9fa2b60d0ad74c4e9ef15db7910dcdf51126e62a02a31463</builder>
            </image>
        </configuration>
    </plugin>
    
  3. fafeichter commented on Oct 23, 2026

    @fafeichter

    I can confirm too.

  4. landsman commented on Oct 23, 2026

    @landsman

    Yeah, x86 is missing.
    Fix for gradle:

    tasks.bootBuildImage {
        publish = false
        imageName = "your-image:latest"
        builder = "paketobuildpacks/builder-noble-java-tiny:0.0.200"
    }
  5. mschout commented on Oct 23, 2026

    @mschout

    Yeah, this is a significant problem.

  6. alekseinovikov commented on Oct 23, 2026

    @alekseinovikov

    Looks like a race in push-image.yml, not a build change. Run 39973018505 (0.0.200) and run 39892077505 (0.0.199) used the same runner image, crane v0.22.1, pack v0.40.9 and lifecycle v0.21.26. For 0.0.199, crane tag … latest wrote the index (sha256:07b5e2d0…, 513 bytes). For 0.0.200 it wrote sha256:771c7c33… (6636 bytes), which is the linux/arm64 child of the 0.0.200 index sha256:7c62dd7e…. So crane tag read :0.0.200 about 2 s after pack pushed the manifest list and got a single-platform manifest. Re-tagging latest from the index digest (crane tag paketobuildpacks/builder-noble-java-tiny@sha256:7c62dd7e66b277ec9fa2b60d0ad74c4e9ef15db7910dcdf51126e62a02a31463 latest) should fix it. Tagging by the digest pack reports, rather than by tag name, would avoid the race.

  7. AganzeFelicite commented on Oct 24, 2026

    @AganzeFelicite

    Same here: amd64 GitLab CI runners, Spring Boot Gradle plugin 4.1.0, bootBuildImage with the default builder.

    • Last good build, 2026-10-23 12:55 UTC: :latest resolved to sha256:07b5e2d0… (the 0.0.199 index) and the plugin pulled ubuntu-noble-run-tiny:0.0.149 for linux/amd64.
    • From 14:55 UTC: :latest resolved to sha256:771c7c33…, the plugin pulled ubuntu-noble-run-tiny:0.0.150 for linux/arm64, and the lifecycle failed: exec /cnb/lifecycle/creator: exec format error.
    • Still the case at 2026-10-24 11:28 UTC: docker buildx imagetools inspect …:latest shows a single manifest (architecture arm64), while 0.0.200 is an index with amd64 and arm64.

    Worked around by pinning builder = "paketobuildpacks/builder-noble-java-tiny:0.0.199". Could a maintainer re-tag latest to the 0.0.200 index (sha256:7c62dd7e66b277ec9fa2b60d0ad74c4e9ef15db7910dcdf51126e62a02a31463)?

  8. added a commit that references this issue on Oct 24, 2026
  9. chadselph commented on Oct 26, 2026

    @chadselph

    This was "fixed" by 0.0.201 and 0.0.202 being pushed, but it would be nice to address @alekseinovikov's findings so this won't keep happening

  10. pacostas commented on Oct 27, 2026

    @pacostas
    Member

    @alekseinovikov Would you like to provide a fix? Even checking if the latest tag, has the same amount of architectures as the tag with number and failing the workflow, would be useful too. Another solution could be tagging with the digest

  11. alekseinovikov commented on Oct 27, 2026

    @alekseinovikov

    @pacostas Sure, opened paketo-buildpacks/github-config#1450. It waits until the version tag lists every target of builder.toml, then tags latest by that manifest's digest, so it does both the platform check and the digest tagging.

  12. pacostas commented on Oct 27, 2026

    @pacostas
    Member

    Nice, thank you!! Reviewing it asap

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions