Airgap example

[jaudiger]

I was looking for an example about how to use Porter images (https://porter.sh/author-bundles/#images) with the mixin docker-compose.

Once published (https://porter.sh/distribute-bundles/#image-references-after-publishing), the images referenced are built into the archive, but if I try to use this archive on a machine without internet access, docker is trying to pull the images.

This is what I did so far:

porter.yaml:

name: redis-application
version: 0.1.0
description: Redis as a CNAB application
registry: MY_ACCOUNT_ID.dkr.ecr.eu-west-3.amazonaws.com

required:
  - docker:
      privileged: false

mixins:
  - docker-compose

images:
  redis:
    imageType: docker
    repository: docker.io/redis
    tag: 6.2.4

install:
  - docker-compose:
      description: Start redis
      suppress-output: true
      arguments:
      - up
      - -d

upgrade:
  - docker-compose:
      description: Run redis
      suppress-output: true
      arguments:
      - up
      - -d

uninstall:
  - docker-compose:
      description: Stop redis
      suppress-output: true
      arguments:
      - down

docker-compose.yml

version: "3.7"

services:
  redis-master:
    container_name: redis-master
    image: redis:6.2.4
    ports:
      - 6379:6379

  redis-slave:
    container_name: redis-slave
    image: redis:6.2.4
    ports:
      - 6380:6380
    command: redis-server --slaveof localhost 6379 --port 6380
    depends_on:
      - redis-master

My commands used:

porter build
porter publish

porter archive --reference MY_ACCOUNT_ID.dkr.ecr.eu-west-3.amazonaws.com/redis-application:v0.1.0 redis-application-installer.tgz

tar -zxvf redis-application-installer.tgz
porter install --allow-docker-host-access --cnab-file redis-application-installer/bundle.json redis-application

I think I missed something but I don't know what... I also tried to look at helm airgap example: https://github.com/getporter/porter/tree/main/examples/airgap

[vdice]

@jaudiger this reminds me of the following Porter issue: getporter/porter#1609

When convenient, could you try the same steps using https://github.com/getporter/porter/releases/tag/v0.38.3 or higher (or latest v1 build?) Those versions should have a fix related to this issue...

[jaudiger]

Ok, so I think I'm definitively doing something wrong here (maybe I missed a command). I already have the latest version available of Porter:

jaudiger@fedora: …/porter-application [🌱 develop](9139c2f0)(🛤️)
➜ porter mixin list
Name             Version   Author
arm              v0.8.2    Porter Authors
aws              v0.3.2    Porter Authors
az               v0.5.2    Porter Authors
docker           v0.2.3    Porter Authors
docker-compose   v0.4.1    Porter Authors
exec             v0.38.4   Porter Authors
gcloud           v0.4.2    Porter Authors
helm             v0.13.4   Porter Authors
kubernetes       v0.28.4   Porter Authors
terraform        v0.6.2    Porter Authors
jaudiger@fedora: …/porter-application [🌱 develop](9139c2f0)(🛤️)
➜ porter -v
porter v0.38.4 (3d6d0f77)

[vdice]

Hi @jaudiger, thanks for checking the Porter version used.

I believe the issue lies in commands used (thanks for listing). Here's what you have:

porter build
porter publish

porter archive --reference MY_ACCOUNT_ID.dkr.ecr.eu-west-3.amazonaws.com/redis-application:v0.1.0 redis-application-installer.tgz

tar -zxvf redis-application-installer.tgz
porter install --allow-docker-host-access --cnab-file redis-application-installer/bundle.json redis-application

However, I believe the flow should look like this:

porter build
porter publish

porter archive --reference MY_ACCOUNT_ID.dkr.ecr.eu-west-3.amazonaws.com/redis-application:v0.1.0 redis-application-installer.tgz

# transport redis-application-installer.tgz over the airgap

# publish from archive to local registry (or whichever registry is accessible on this side of airgap)
porter publish --archive redis-application-installer.tgz --reference localhost:5000/redis-application:v0.1.0

# install bundle from published reference
porter install --allow-docker-host-access --reference localhost:5000/redis-application:v0.1.0 redis-application

Basically, the bundle needs to be published on the other side of the airgap, using the archive, and then installed from that new location.

In the original approach, installation of the bundle was attempted via the archived cnab-file/bundle.json directly... which, without all of the image artifacts that are part of the .tgz archive, will always try to look up the original images from the references in that bundle.json.

[jaudiger]

Ok, I see @vdice. I got another error while trying to publish the images to the local registry:

Beginning bundle publish to localhost:5000/redis-application:0.1.0. This may take some time.
Error: unable to push image localhost:5000/redis: failed to write image sha256:0145cc5df5e657154b83fa04b90e48f6f89b608850d0906addfc0d1681a66a1e to localhost:5000/redis: failed to write image index localhost:5000/redis: unexpected media type: application/vnd.docker.image.rootfs.foreign.diff.tar.gzip for layer: sha256:4612f6d0b889cad0ed0292fae3a0b0c8a9e49aff6dea8eb049b2386d9b07986f
Error: unable to pull bundle before uninstall: unable to pull bundle localhost:5000/redis-application:0.1.0: unable to pull remote bundle: localhost:5000/redis-application:0.1.0: not found

I used the following commands to create the local registry, and then to publish:

# Local registry
docker run -d -p 5000:5000 --restart=always --name registry registry:2.7

# Publish images
porter publish --archive redis-application-installer.tgz --reference localhost:5000/redis-application:v0.1.0

[vdice]

Interesting. Do any errors crop up in the docker registry container logs?

[jaudiger]

I got a lot of logs, find the two latest errors:

172.17.0.1 - - [06/Aug/2021:12:26:15 +0000] "PUT /v2/redis/manifests/sha256:c8fdbd07bf7ca5751cbbcab04c41debc293f4f9f0348b5a96c1b11a5a35857f8 HTTP/1.1" 201 0 "" "Go-http-client/1.1"
time="2021-08-06T12:26:15.226460952Z" level=error msg="response completed with error" err.code="manifest unknown" err.detail="unknown manifest name=redis revision=sha256:d0bfb8f110aaf97fdd185e13ecce83d51bf126af99a0da2c796fff4e9c140f83" err.message="manifest unknown" go.version=go1.11.2 http.request.host="localhost:5000" http.request.id=d8cc8993-a6a3-4e56-9024-07b75ed41d1c http.request.method=HEAD http.request.remoteaddr="172.17.0.1:55456" http.request.uri="/v2/redis/manifests/sha256:d0bfb8f110aaf97fdd185e13ecce83d51bf126af99a0da2c796fff4e9c140f83" http.request.useragent="Go-http-client/1.1" http.response.contenttype="application/json; charset=utf-8" http.response.duration=1.295004ms http.response.status=404 http.response.written=181 vars.name=redis vars.reference="sha256:d0bfb8f110aaf97fdd185e13ecce83d51bf126af99a0da2c796fff4e9c140f83" 

172.17.0.1 - - [06/Aug/2021:12:26:15 +0000] "HEAD /v2/redis/manifests/sha256:d0bfb8f110aaf97fdd185e13ecce83d51bf126af99a0da2c796fff4e9c140f83 HTTP/1.1" 404 181 "" "Go-http-client/1.1"
time="2021-08-06T12:26:15.778692097Z" level=error msg="response completed with error" err.code="manifest unknown" err.detail="unknown tag=0.1.0" err.message="manifest unknown" go.version=go1.11.2 http.request.host="localhost:5000" http.request.id=3b1f98dd-9fc7-4582-9990-4ce3f9d3a1f7 http.request.method=HEAD http.request.remoteaddr="172.17.0.1:55488" http.request.uri="/v2/redis-application/manifests/0.1.0" http.request.useragent="containerd/1.3.0+unknown" http.response.contenttype="application/json; charset=utf-8" http.response.duration="897.368µs" http.response.status=404 http.response.written=95 vars.name=redis-application vars.reference=0.1.0

@vdice I'll be (once again) on vacation for the next two weeks, I'll be back the last week of August!

[bherw]

I'm working on a similar airgap deployment right now.

I think the issue with docker-compose trying to contact the docker registry is that the images in the docker-compose.yaml file are still referring to "redis:6.2.4", but after publishing the redis image is relocated to localhost:5000/redis-application@sha256:0145cc5df5e657154b83fa04b90e48f6f89b608850d0906addfc0d1681a66a1e (or something like that).

The docs at https://porter.sh/distribute-bundles/#image-references-after-publishing say "Bundles should not hard-code image references and instead should use the images section and templating so that they are referencing the published location of the image.".

The approach in the docs works fine in the helm airgap example because there they pass the repository and digest to the helm command in the porter.yaml. Is there a similar way to use porter's templating in the docker-compose.yaml? Another solution might be to pass the relocated repository and perhaps image digests as environment variables to docker-compose. docker-compose will substitute environment variables into the compose.yaml.

[vdice]

Interesting @bherw , you may be right. Or, if not directly responsible for this issue, it would indeed make sense to figure out how to update the compose yaml with relocated images -- otherwise if the airgap prevents accessing the original app images, as would be expected, the republished bundle would fail to install (or at least, I'd expect the compose app not to come up.) I'd definitely like to help with ideas here, if anyone would like to create an issue for this...

Looking again at the original error, I see "unexpected media type: application/vnd.docker.image.rootfs.foreign.diff.tar.gzip". Perusing https://docs.docker.com/registry/spec/manifest-v2-2/ I see:

application/vnd.docker.image.rootfs.foreign.diff.tar.gzip: “Layer”, as a gzipped tar that should never be pushed

I wasn't able to reproduce the issue publishing/archiving from a GitHub Container Registry to a local docker registry. Worth looking to see if some other origin registries, perhaps like ECR used above, push layers of those types and then error out when publishing to a Docker registry container. Not sure... just theorizing here. Worth exploring more.

[bherw]

Added an issue at #26.