Cross-platform buildx and proper use of $BUILDPLATFORM

[caseybarker]

I'm trying to craft a Dockerfile to build a cross-platform (amd64 and arm64) application with buildx build. Some precursors are platform-independent and only need to be done once, e.g., compiling .proto files to .h files (or whatever). But for my output binary, it's easier/better to use qemu and just emulate a native compiler environment.

I reduced my problem to the simplest possible Dockerfile I could, and I'm having difficulty getting buildx build to honor the --platform=$BUILDPLATFORM for the proto-build target. I added a RUN line that fails if it's not using my host architecture:

# syntax = docker/dockerfile:1

FROM ubuntu:20.04 AS builder
RUN echo "install build toolchain"

FROM --platform=$BUILDPLATFORM builder AS proto-build
RUN echo "compile protobufs using host architecture" \
  && touch /protos.h
ARG BUILDPLATFORM
RUN if [ "$(uname -m)" != "x86_64" ]; then echo "Host: $(uname -m); BUILDPLATFORM: ${BUILDPLATFORM}" && exit 1; fi

FROM builder AS binary-build
COPY --link --from=proto-build /protos.h /
RUN echo "compile using target architecture" \
  && touch a.out

FROM --platform=$TARGETPLATFORM ubuntu:20.04 AS runtime
COPY --link --from=binary-build /a.out /
CMD exec /a.out

I'm then invoking:
docker buildx build --platform linux/arm64 --target=runtime .

As noted, this tries to build the protobufs under arm instead of using the $BUILDPLATFORM, so it outputs:
Host: aarch64; BUILDPLATFORM: linux/amd64

The real problems start when I try to use bake and multiple output platforms; bake ends up building the proto-build target twice, once for each platform.

As I understand it, the --platform argument only changes the $TARGETPLATFORM argument to the Dockerfile, not the $BUILDPLATFORM. So, questions:

1. What am I misunderstanding here?
2. Any suggestions on how to untangle this so I can build precursors one time only, always using my host's native architecture?

[caseybarker]

The effort of simplifying my Dockerfile for this post helped me explore the problem, and I discovered that if I break the common dependency on the builder image, so that proto-build doesn't share the same base image as binary-build, but instead just descends directly from ubuntu, then the proto-build works as I'd expect and honors $BUILDPLATFORM. Further, bake builds both platforms fine, and only invokes the proto-build stage one time.

So why is this? I'd like to have all the dev tools containerized on either host architecture, so I can build the whole stack from either architecture. (Further down in my real Dockerfile, there's also a vscode devcontainer built on top of this, and I can run it on my arm host.) But having a shared ancestor image in the same Dockerfile seems to cause it to ignore the $BUILDPLATFORM. Curiously, I can still have both binary-build and proto-build depending on ubuntu as a common ancestor, and that works fine.