Skip to content

boot_standard.md

Latest commit

 

History

History
182 lines (153 loc) · 11.9 KB

boot_standard.md

File metadata and controls

182 lines (153 loc) · 11.9 KB
Raw

Goal

Boot a QOS node by submitting an approved manifest and pivot binary, then provision the node with quorum key shares. Share set members verify the node's attestation document and submit their shares encrypted to the node's Ephemeral Key. Once the configured threshold of shares has been posted, the node reconstructs the Quorum Key and starts the Pivot App.

When Is Boot Standard Required?

Boot Standard is the provisioning mode where Share Holders post Quorum Key shares to a freshly booted node after verifying its setup attestation. The alternative, Key Forward, lets an already provisioned node in the same namespace (the source node) export the Quorum Key to a fresh node with no human involvement.

Every change deployed to an enclave — new pivot app, new QOS image, new manifest configuration — is shipped by booting a fresh node with a new Manifest-Set-approved manifest. The only question is whether that fresh node can be provisioned by Key Forward or needs a Boot Standard share ceremony. The answer depends on the source node: export_key refuses to forward the Quorum Key unless the new manifest passes its checks. Boot Standard is required exactly when no node can (or may) act as the source.

The deployer must ensure that each enclave and all associated host services are bound to a single approved manifest. This binding must remain unchanged for the workload's lifetime. A different approved manifest requires a new enclave and new host services.

Changes that do not require Boot Standard — Key Forward works, as long as at least one provisioned node in the namespace is able to act as the source:

  • Pivot App updates (new pivot hash) and pivot configuration changes (args, env, restart policy, socket pool size), via a new approved manifest with an incremented nonce. This is the common update flow.
  • QOS image updates (new PCR0/PCR1/PCR2). The source node checks the new manifest's PCRs against the new node's attestation document, not against its own measurements; the Manifest Set's threshold approvals are what vouch for the new QOS version.
  • Adding capacity or replacing instances with an unchanged manifest (equal nonce is accepted when the manifest hash is identical).

Changes that do require Boot Standard — the source node refuses to export, or no source node exists:

  • First boot of a namespace after genesis: no provisioned node exists yet.
  • Loss of every provisioned node in the namespace (disaster recovery).
  • Manifest Set changes, members or threshold (DifferentManifestSet): a source node only trusts approvals from its own Manifest Set, so rotating the set requires re-provisioning by the Share Set.
  • PCR3 changes, i.e. a new host IAM role or AWS account (DifferentPcr3).
  • Namespace name changes (DifferentNamespaceName).
  • A manifest nonce lower than the source node's, or an equal nonce with a different manifest hash (LowNonce, DifferentManifest). A rollback can only be key-forwarded from a source node whose own nonce is low enough.
  • A different Quorum Key (DifferentQuorumKey) — in practice a new namespace, starting with its own genesis ceremony.
  • QOS releases that break the key-forward protocol itself: if a source node on the old release cannot decode the new node's request or verify its attestation document, the first node on the new release must be Boot Standard provisioned. Wire compatibility is normally maintained to avoid this, so treat any breaking change to ProtocolMsg, manifest encoding, or attestation verification as a potential Boot Standard event and verify Key Forward across the version boundary.

New terms

  • Node: The un-provisioned QOS node being booted.
  • Manifest Envelope: The manifest used to boot the Node, plus approvals from the Manifest Set.
  • Share Holder: A member of the Share Set who holds an encrypted quorum key share and can approve provisioning for a specific manifest.
  • Setup Ephemeral Key: A temporary P-256 key generated by the Node during boot. Share Holders encrypt their quorum key shares to this key before posting them to the Node. Setup attestations commit this key and the manifest hash in PCR16.
  • Live Ephemeral Key: A P-256 key generated during boot and installed after quorum key provisioning completes. Live/app attestations commit this key and the manifest hash in PCR17.

Other relevant terms

  • Nonce: A strictly monotonically increasing counter used to track the latest manifest of a Namespace. Concretely, anytime a new manifest is created for a Namespace, we expect it to have a Nonce at least 1 greater than the previous most recent manifest.
  • Pivot App: The application a Node is intended to run. The application is also commonly referred to as an Enclave Application.
  • Manifest Set: The set of members authorized to approve manifests for a Namespace.
  • Share Set: The set of members authorized to provision quorum key shares to a Node.
  • Quorum Key: The namespace key reconstructed from Share Set shares after the booted Node has been attested.

Routine

The details for how clients and QOS nodes communicate can vary, but for the sake of this example we assume that all requests are made from a client and the Node is reachable through qos_host.

  1. Before contacting the Node, the Client prepares a Manifest Envelope.

    struct ManifestEnvelope {
  manifest: Manifest,
  manifest_set_approvals: Vec<Approval>,
  share_set_approvals: Vec<Approval>,
}

    The manifest specifies all environment configuration including the namespace, nonce, expected Quorum Key public key, Pivot App hash, restart policy, optional pivot configuration, Manifest Set, Share Set, and Nitro PCR values. The Manifest Envelope must have at least manifest.manifest_set.threshold valid approvals from distinct members of the Manifest Set. The Share Set has its own threshold, manifest.share_set.threshold, which controls quorum key reconstruction during provisioning rather than manifest approval.

  2. The Node gets a BootStandardRequest from the Client.

    struct BootStandardRequest {
  manifest_envelope: ManifestEnvelope,
  pivot: Vec<u8>,
}

  3. The Node processes the request by performing the following steps. The implementation lives in put_manifest_and_pivot and boot_standard.

    1. Check signatures over the manifest envelope with ManifestEnvelope::check_approvals. This ensures that at least manifest.manifest_set.threshold distinct Manifest Set Members approved the manifest.

    2. Reject the request if the Manifest Envelope already contains Share Set approvals. Share Set approvals are recorded as shares are posted during provisioning, not during the initial boot instruction.

    3. Hash the submitted Pivot App binary and check that it matches manifest_envelope.manifest.pivot.hash.

    4. Generate the setup Ephemeral Key and live Ephemeral Key.

    5. Extend PCR16 with the setup manifest/key commitment and PCR17 with the live manifest/key commitment, then lock the full attestable PCR range before publishing pivot start files.

    6. Write the setup Ephemeral Key, Pivot App binary, and Manifest Envelope to the Node's filesystem.

    7. Make a setup attestation request, placing the manifest hash in the user_data field and the setup Ephemeral Key public key in the public_key field.

    8. Return the NSM Response containing the COSE Sign1 encoded attestation document.

      struct BootStandardResponse {
  nsm_response: NsmResponse,
}

  4. Each Share Holder verifies the attestation document before provisioning shares. An example of this flow is implemented by proxy_re_encrypt_share.

    1. Check the basic validity of the attestation doc (cert chain etc). This ensures that the attestation document is actually from an AWS controlled NSM module and that the document's timestamp was recent.
    2. Check that the manifest hash is in the user_data field of the attestation doc.
    3. Check that PCR0, PCR1, PCR2, and PCR3 in the manifest match the PCRs in the attestation document. This ensures the manifest was used against a Nitro enclave booted with the intended version of QOS and the intended AWS identity.
    4. Check that every PCR in the release-pinned attestable range is present in the attestation document.
    5. Recompute the setup manifest/key commitment from the manifest hash and the attested public_key, then check that it matches PCR16.
    6. Extract the setup Ephemeral Key public key from the public_key field of the attestation document.
    7. Check that the Manifest Envelope has at least manifest.manifest_set.threshold valid Manifest Set approvals.
    8. Check that the Share Holder belongs to the Share Set in the manifest.
    9. Perform any required human checks, such as confirming the namespace name, nonce, IAM role, and Manifest Set approvers.

  5. Each Share Holder decrypts their locally held share and constructs a ProvisionRequest.

    struct ProvisionRequest {
  share: Vec<u8>,
  approval: Approval,
}

    The share field is the Share Holder's quorum key share re-encrypted to the Node's setup Ephemeral Key. The approval is the Share Holder's signature over the manifest hash, plus the Share Holder member information. This approval proves that the intended Share Holder intentionally provisioned a share for this manifest.

  6. The Node processes each ProvisionRequest by performing the following steps in provision:

    1. Fetch the local Manifest Envelope written during BootStandardRequest.

    2. Verify that the approval signature is valid over the manifest hash.

    3. Verify that the approver belongs to the manifest's Share Set.

    4. Record the Share Set approval in the local Manifest Envelope for auditability.

    5. Decrypt the posted share with the Node's setup Ephemeral Key.

    6. Add the decrypted share to the provisioning state. Each recorded Share Set approval corresponds to one posted encrypted share; the Node does not separately accept a batch of Share Set approvals before shares are posted. Note the approvals are primarily for audit trail purposes.

    7. If fewer than manifest.share_set.threshold shares have been posted, return ProvisionResponse { reconstructed: false }.

    8. Once manifest.share_set.threshold shares have been posted, reconstruct the Quorum Key with Shamir reconstruction. The true cryptographic reconstruction threshold is determined by how the Shamir shares were originally split. manifest.share_set.threshold represents the minimum number of Share Set approvals required before the Node attempts reconstruction, so it should be greater than or equal to the Shamir reconstruction threshold.

    9. Check that the reconstructed Quorum Key public key matches manifest.namespace.quorum_key. This is the effective cryptographic check for provisioning: if the posted shares cannot reconstruct the intended key, provisioning fails.

    10. Rotate to the precommitted live Ephemeral Key so app-level uses do not reuse the setup key that protected share provisioning.

    11. Write the Quorum Key to the filesystem, at which point the Node will automatically pivot to running the Pivot App.

    12. Return ProvisionResponse { reconstructed: true }.

      struct ProvisionResponse {
  reconstructed: bool,
}

  7. The reaper observes that the Manifest Envelope, Pivot App binary, and Quorum Key all exist, then spawns the Pivot App using the manifest's pivot configuration, passing the command line arguments and env vars from the manifest.