diff --git a/docs/configuration/templated-ipxe-operating-systems.md b/docs/configuration/templated-ipxe-operating-systems.md new file mode 100644 index 0000000000..5459360314 --- /dev/null +++ b/docs/configuration/templated-ipxe-operating-systems.md @@ -0,0 +1,493 @@ +# Templated iPXE Operating Systems + +Templated iPXE Operating Systems let NICo reuse a validated iPXE script +template while supplying the parameters and boot artifacts that vary between +operating-system definitions. They replace most uses of raw `ipxeScript` and `OS Images`, +which will be deprecated in the REST API. + +This guide explains the template and Operating System resources, ownership and +site rules, synchronization between NICo REST and NICo Core, and the workflow +for creating and using a templated Operating System. + +## How the Resources Fit Together + +The feature has three layers: + +1. **iPXE template** -- a read-only script blueprint built into NICo Core. It + declares required parameters, reserved parameters, and required artifacts. +2. **Templated iPXE Operating System** -- a reusable definition that references + one template and supplies its parameter values, artifact URLs, user data, + and boot policy. +3. **Instance** -- references the Operating System by UUID. NICo Core combines + the definition with reserved values and renders + the final iPXE script when the machine boots. + +```mermaid +flowchart LR + CoreTemplate["Public template in NICo Core"] + RestTemplate["Read-only template catalog in NICo REST"] + OS["Templated iPXE Operating System"] + Instance["Instance with operatingSystemId"] + Script["Rendered iPXE script"] + + CoreTemplate -->|"site inventory"| RestTemplate + RestTemplate -->|"ipxeTemplateId"| OS + OS -->|"synchronized to one Site"| CoreTemplate + OS --> Instance + CoreTemplate --> Script + Instance --> Script +``` + +Templates and Operating Systems are different resources. A template describes +the placeholders and script structure. An Operating System provides concrete +values for one use of that template. + +The examples use `nicocli` for the REST API and `nico-admin-cli` for direct +Core administration. To run the REST calls with `curl`, set: + +```bash +export NICO_API_URL="https://nico.example.com" +export NICO_ORG="" +export NICO_TOKEN="" +``` + +## Template Discovery and Availability + +iPXE templates are currently compiled into NICo Core and are read-only. Only +templates with `Public` visibility are synchronized to NICo REST. + +REST stores one global template record for each stable Core UUID and tracks +which Sites currently report it. Consequently: + +- A template can be listed only at Sites where it is available. +- The same template UUID reported by multiple Sites is represented once. +- Removing the last Site association removes the template from the REST + catalog. +- Templates cannot be created, updated, or deleted through the REST API. + +List the templates available at the target Site: + +```bash +nicocli ipxe-template list \ + --site-id \ + --all \ + --output table +``` + +The equivalent REST request is: + +```bash +curl --fail-with-body --silent --show-error --get \ + --header "Authorization: Bearer ${NICO_TOKEN}" \ + --data-urlencode "siteId=" \ + "${NICO_API_URL}/v2/org/${NICO_ORG}/nico/ipxe-template" +``` + +Inspect the selected template: + +```bash +nicocli ipxe-template get +``` + +Or retrieve it directly through REST: + +```bash +curl --fail-with-body --silent --show-error \ + --header "Authorization: Bearer ${NICO_TOKEN}" \ + "${NICO_API_URL}/v2/org/${NICO_ORG}/nico/ipxe-template/" +``` + +Before creating an Operating System, note these fields: + +| Field | Meaning | +|---|---| +| `id` | Stable UUID to send as `ipxeTemplateId` | +| `requiredParams` | Parameter names the Operating System must provide | +| `reservedParams` | Values supplied by NICo Core at render time; do not provide them | +| `requiredArtifacts` | Artifact names the Operating System must provide | +| `visibility` | Only `Public` templates are available through REST | + +## Ownership and Authorization + +Ownership is inferred from the caller's organization and role. The deprecated +`tenantId` and `infrastructureProviderId` request fields should not be used. + +| Caller | Create behavior | Read behavior | Update and delete | +|---|---|---|---| +| Provider Admin | May create only Templated iPXE Operating Systems at a Registered Site owned by its provider. The result is provider-owned. | Sees its provider-owned definitions. | May modify its provider-owned definitions. | +| Tenant Admin | May create a tenant-owned Templated iPXE Operating System at a Registered Site accessible to the tenant. | Sees tenant-owned definitions plus provider-owned definitions associated with an accessible Site. | May modify only tenant-owned definitions. | +| Dual-role user | Provider ownership takes priority when creating a Templated iPXE Operating System. | Sees the union of its provider and tenant views. | Authorization follows the resource owner. | +| Provider Viewer | Cannot create or modify Operating Systems. | Can discover templates available at provider Sites. | None. | + +Visibility does not grant mutation rights. A Tenant can select a visible +provider-owned definition when creating or updating an Instance at an +associated Site, but only the Provider can update or delete that definition. + +## Site Rules + +A Templated iPXE Operating System created through REST must specify exactly one +Site: + +```json +{ + "siteIds": [""] +} +``` + +Although `siteIds` is an array for compatibility with the existing Operating +System API, zero or multiple values are rejected for this type. + +The Site must: + +- Exist and be in `Registered` status. +- Be owned by the Provider for a provider-owned definition, or be accessible + to the Tenant for a tenant-owned definition. +- Report the referenced iPXE template. + +The Site association is fixed at creation and cannot be updated. + +## Create a Templated iPXE Operating System + +The following example uses the built-in `kernel-initrd` template. That template +requires the `kernel_params` parameter and the `kernel` and `initrd` artifacts. +Always inspect the template returned by your deployment instead of assuming +that its requirements match this example. + +Save the request body so it can be used with either client: + +```bash +cat > templated-os.json <<'EOF' +{ + "name": "ubuntu-24.04-autoinstall", + "description": "Ubuntu 24.04 automated installation", + "siteIds": [""], + "ipxeTemplateId": "", + "ipxeTemplateParameters": [ + { + "name": "kernel_params", + "value": "ip=dhcp autoinstall" + } + ], + "ipxeTemplateArtifacts": [ + { + "name": "kernel", + "url": "https://artifacts.example.com/ubuntu/vmlinuz", + "cacheStrategy": "CacheAsNeeded" + }, + { + "name": "initrd", + "url": "https://artifacts.example.com/ubuntu/initrd", + "cacheStrategy": "CacheAsNeeded" + } + ], + "userData": "#cloud-config\nhostname: ubuntu-worker\n", + "phoneHomeEnabled": false, + "allowOverride": true +} +EOF +``` + +Create it with `nicocli`: + +```bash +nicocli operating-system create --data-file templated-os.json +``` + +Or create it directly through REST: + +```bash +curl --fail-with-body --silent --show-error \ + --request POST \ + --header "Authorization: Bearer ${NICO_TOKEN}" \ + --header "Content-Type: application/json" \ + --data @templated-os.json \ + "${NICO_API_URL}/v2/org/${NICO_ORG}/nico/operating-system" +``` + +The request type is inferred from `ipxeTemplateId`; do not send a separate +`type` value. `ipxeTemplateId` is mutually exclusive with `ipxeScript` and +`imageUrl`. + +### Parameters + +Each parameter has a `name` and `value`. + +- Supply every name in `requiredParams` with a non-empty value. +- Do not supply names in `reservedParams`; NICo Core provides those values. +- Extra parameters are accepted only when the template supports its extra + placeholder. +- Parameter names must be unique within the Operating System definition. + +### Artifacts + +Each artifact requires `name` and `url`. Optional fields are: + +- `sha` -- expected SHA-256 checksum. +- `authType` -- `Basic` or `Bearer`. +- `authToken` -- credential used to retrieve the artifact. +- `cacheStrategy` -- one of the strategies below. + +| Strategy | URL used at boot | +|---|---| +| `CacheAsNeeded` | Prefer the Site's cached URL; fall back to the supplied URL. | +| `RemoteOnly` | Always use the supplied URL. | +| `LocalOnly` | Use the supplied URL directly; it is already Site-local. | +| `CachedOnly` | Require a cached URL at the Site; rendering cannot proceed until one exists. | + +Artifact `authToken` values are accepted on create and update but are +structurally omitted from REST API responses. Treat them as write-only +credentials. + +The per-Site `cachedUrl` value is managed by NICo Core and is not stored or +returned by NICo REST. Core's cache-management gRPC operations are the only +supported way to set or clear it. A `CachedOnly` artifact without a cached URL +keeps the Core definition from becoming ready. + +After a cache service has downloaded an artifact, a Site administrator can +record its Site-local URL directly in Core: + +```bash +nico-admin-cli operating-system set-cached-url \ + --set kernel=https://cache.example.com/ubuntu/vmlinuz \ + +``` + +Repeat `--set` for multiple artifacts. Use `--set =` to clear a cached +URL. + +## Create a Site-Local Definition in Core + +The REST API is the preferred interface for normal operations. A Site +administrator can also create a definition directly in Core with +`nico-admin-cli`. For example, the built-in `qcow-image` template requires the +`image_url` parameter: + +```bash +nico-admin-cli operating-system create \ + --name site-qcow-image \ + --ipxe-template-id ea756ddd-add3-5e42-a202-44bfc2d5aac2 \ + --param image_url=https://artifacts.example.com/images/ubuntu.qcow2 +``` + +The optional `--org` flag controls how REST assigns ownership when it discovers +the definition: + +- Omit `--org` to create a provider-owned definition for the reporting Site. +- Set `--org ` to create a tenant-owned definition. + REST must be able to resolve that organization to an existing Tenant. +- An explicitly empty organization is invalid. + +The discovered definition has one Site association and therefore participates +in bidirectional synchronization. + +## Verify Synchronization + +Capture the returned Operating System UUID and inspect it: + +```bash +nicocli operating-system get +``` + +The equivalent REST request is: + +```bash +curl --fail-with-body --silent --show-error \ + --header "Authorization: Bearer ${NICO_TOKEN}" \ + "${NICO_API_URL}/v2/org/${NICO_ORG}/nico/operating-system/" +``` + +This response contains the definition, aggregate `status`, and `statusHistory`. +The get-by-ID response does not currently expand Site associations for +Templated iPXE definitions. Use the Site-filtered list response to inspect the +association: + +```bash +nicocli operating-system list \ + --site-id \ + --type TemplatedIpxe \ + --all +``` + +The REST equivalent is: + +```bash +curl --fail-with-body --silent --show-error --get \ + --header "Authorization: Bearer ${NICO_TOKEN}" \ + --data-urlencode "siteId=" \ + --data-urlencode "type=TemplatedIpxe" \ + "${NICO_API_URL}/v2/org/${NICO_ORG}/nico/operating-system" +``` + +Wait for the target `siteAssociations` entry to become `Synced` before +selecting the Operating System for an Instance. + +Common association states are: + +| State | Meaning | +|---|---| +| `Syncing` | REST is sending or re-sending the definition to the Site. | +| `Synced` | The Site accepted the definition. | +| `Error` | The Site proxy operation or association bookkeeping failed; inspect status history. | +| `Deleting` | Site cleanup is in progress. | + +Updates temporarily return the association to `Syncing`. Instance create and +update reject a Templated iPXE Operating System unless its association with the +Instance's Site is `Synced`. + +## Use the Operating System for an Instance + +For either a tenant-owned definition or a visible provider-owned definition, +pass its UUID as `operatingSystemId`. The Instance's VPC and the Operating +System must resolve to the same Site, and the Site association must be +`Synced`. + +```bash +nicocli instance create \ + --name ubuntu-worker-01 \ + --tenant-id \ + --vpc-id \ + --instance-type-id \ + --operating-system-id +``` + +The equivalent REST request is: + +```bash +curl --fail-with-body --silent --show-error \ + --request POST \ + --header "Authorization: Bearer ${NICO_TOKEN}" \ + --header "Content-Type: application/json" \ + --data '{ + "name": "ubuntu-worker-01", + "tenantId": "", + "vpcId": "", + "instanceTypeId": "", + "operatingSystemId": "" + }' \ + "${NICO_API_URL}/v2/org/${NICO_ORG}/nico/instance" +``` + +NICo sends the Operating System UUID to Core rather than expanding the template +in REST. Core retrieves the synchronized definition, supplies reserved +machine/Site values, validates the definition hash, resolves artifact URLs, +and renders the final iPXE script. + +If `allowOverride` is enabled, Instance request fields can override supported +Operating System settings such as user data. See +[Tenant Management](tenant_management.md#launching-an-instance) for the full +Instance workflow. + +## Update a Definition + +Update only the fields that should change. Omitted fields retain their current +values. Supplying a parameter or artifact array replaces the complete +corresponding list; an explicit empty array clears it if the resulting +definition remains valid. Save the update body: + +```bash +cat > templated-os-update.json <<'EOF' +{ + "description": "Ubuntu 24.04 automated installation, revision 2", + "ipxeTemplateParameters": [ + { + "name": "kernel_params", + "value": "ip=dhcp autoinstall console=ttyS0,115200" + } + ] +} +EOF +``` + +Apply it with `nicocli`: + +```bash +nicocli operating-system update \ + --data-file templated-os-update.json \ + +``` + +Or apply it directly through REST: + +```bash +curl --fail-with-body --silent --show-error \ + --request PATCH \ + --header "Authorization: Bearer ${NICO_TOKEN}" \ + --header "Content-Type: application/json" \ + --data @templated-os-update.json \ + "${NICO_API_URL}/v2/org/${NICO_ORG}/nico/operating-system/" +``` + +The Operating System type and Site association cannot be changed. To target a +different Site, create another Operating System definition. + +## Delete a Definition + +```bash +nicocli operating-system delete +``` + +The REST equivalent is: + +```bash +curl --fail-with-body --silent --show-error \ + --request DELETE \ + --header "Authorization: Bearer ${NICO_TOKEN}" \ + "${NICO_API_URL}/v2/org/${NICO_ORG}/nico/operating-system/" +``` + +REST marks the definition and its Site association as deleting, asks Core to +remove the Site copy, and records an actionable error state if Site cleanup +fails. Deletion is rejected while an Instance references the definition. + +## Synchronization and Source of Truth + +Templates and Operating Systems use different synchronization rules: + +- **Templates:** one-way from Core to REST. REST aggregates the Public + templates reported by authorized Sites. +- **Operating Systems with one Site association:** definition changes are + bidirectional. REST compares update timestamps during inventory + reconciliation. +- **Operating Systems with multiple Site associations:** REST is the source of + truth so divergent Site definitions cannot overwrite one another. + +REST currently creates a Templated iPXE Operating System with exactly one Site, +but the multi-Site rule protects existing or administratively-created records. +Reconciliation-by-absence applies only to single-Site definitions: if Core no +longer reports one at that Site, REST soft-deletes it. A multi-Site definition +is not deleted merely because one Site omits it. + +When Core reports a previously unknown Operating System: + +- A present `tenant_organization_id` resolves it as tenant-owned. +- An omitted `tenant_organization_id` resolves it as provider-owned using the + reporting Site's Infrastructure Provider. + +## Troubleshooting + +| Symptom | Cause and resolution | +|---|---| +| No templates are returned for the Site | Confirm the Site is Registered, the template is Public, Core reports it, and the caller is authorized for the Site. | +| `exactly one siteId must be specified` | Send one UUID in `siteIds`; do not omit it or send multiple Sites. | +| Template is not available at the target Site | List templates with `--site-id ` and use an ID from that response. | +| Missing or reserved parameter error | Compare the request with `requiredParams` and `reservedParams` from `ipxe-template get`. | +| Missing artifact error | Supply every name in `requiredArtifacts`; names are case-sensitive. | +| Operating System is not synchronized to the Instance's Site | Wait for the Site association to become `Synced`, or create the definition for the Instance's Site. | +| Definition remains non-ready with `CachedOnly` | Ensure the Site's cache service populated every required cached URL, or use another cache strategy. | +| `authToken` is absent from GET responses | This is intentional redaction. Submit a replacement token on update when credentials rotate. | +| Tenant cannot select a provider-owned definition | Confirm the definition is associated and `Synced` with the Instance's Site and that the Tenant has access to that Site. | + +Use `nicocli --debug` to inspect REST requests and responses, and inspect the +Operating System's `statusHistory` and `siteAssociations` for synchronization +failures. + +## Related Documentation + +- [Tenant Management](tenant_management.md) -- Tenant setup and Instance + provisioning. +- [Phone-home](tenant_management.md#phone-home) -- Readiness behavior controlled + by `phoneHomeEnabled`. +- [nico-admin-cli Operating System reference](../manuals/nico-admin-cli/commands/operating-system/operating-system.md) + -- Direct Core gRPC administration. +- [nico-admin-cli iPXE template reference](../manuals/nico-admin-cli/commands/ipxe-template/ipxe-template.md) + -- Inspect templates directly in Core. diff --git a/docs/configuration/tenant_management.md b/docs/configuration/tenant_management.md index 1f2dcbb6d6..31f7d70bbc 100644 --- a/docs/configuration/tenant_management.md +++ b/docs/configuration/tenant_management.md @@ -520,7 +520,7 @@ An instance in NICo is a bare-metal machine assigned to a tenant within a VPC. C | `--vpc-id` | yes | Parent VPC | | `--machine-id` | no | Pin to a specific machine (requires `targetedInstanceCreation: true` on the tenant) | | `--instance-type-id` | no | Pick from the pool of machines of this type (alternative to `--machine-id`) | -| `--operating-system-id` | no | OS for PXE provisioning | +| `--operating-system-id` | no | OS for PXE provisioning; see [Templated iPXE Operating Systems](templated-ipxe-operating-systems.md) | | `--allow-unhealthy-machine` | no | Override health checks | | `--ipxe-script` | no | Custom iPXE script | | `--user-data` | no | cloud-init style user data | @@ -961,6 +961,7 @@ Flag-first ordering -- always put flags before positional args. ## Related Documentation - [Network Isolation](network-isolation.md) -- Per-plane tenant isolation (Ethernet, InfiniBand, NVLink) +- [Templated iPXE Operating Systems](templated-ipxe-operating-systems.md) -- Reusable template-based boot definitions and Site synchronization - [Organization & Permissions](org-permissions.md) -- IdP-managed roles and user setup - [Quick Start Guide](../getting-started/quick-start.md) -- NICo deployment and Day Zero walkthrough - [VPC Routing Profiles](../manuals/vpc/vpc_routing_profiles.md) -- Profile configuration and behavior diff --git a/docs/index.yml b/docs/index.yml index 90eacad6be..865e2a9f07 100644 --- a/docs/index.yml +++ b/docs/index.yml @@ -140,6 +140,8 @@ navigation: path: manuals/networking/infiniband_partitioning.md - page: Tenant Management path: configuration/tenant_management.md + - page: Templated iPXE Operating Systems + path: configuration/templated-ipxe-operating-systems.md - page: Organization & Permissions path: configuration/org-permissions.md - page: Machine Identity diff --git a/docs/manuals/nico-admin-cli/commands/ipxe-template/ipxe-template-show.md b/docs/manuals/nico-admin-cli/commands/ipxe-template/ipxe-template-show.md index 76149f2f71..f0baa25615 100644 --- a/docs/manuals/nico-admin-cli/commands/ipxe-template/ipxe-template-show.md +++ b/docs/manuals/nico-admin-cli/commands/ipxe-template/ipxe-template-show.md @@ -16,6 +16,10 @@ name). Show iPXE templates (all, or one by name). +For the relationship between templates, Operating System definitions, Sites, +and Instances, see +[Templated iPXE Operating Systems](../../../../configuration/templated-ipxe-operating-systems.md). + ## OPTIONS **--extended** @@ -50,4 +54,4 @@ nico-admin-cli ipxe-template show 12345678-1234-5678-90ab-cdef01234567 --- -**See also:** [Tenant commands](../../tenant.md) · [CLI reference index](../../README.md) +**See also:** [Tenant commands](../../tenant.md) · [CLI reference index](../../index.md) diff --git a/docs/manuals/nico-admin-cli/commands/operating-system/operating-system-create.md b/docs/manuals/nico-admin-cli/commands/operating-system/operating-system-create.md index 1d8a2dcf18..baad441d7c 100644 --- a/docs/manuals/nico-admin-cli/commands/operating-system/operating-system-create.md +++ b/docs/manuals/nico-admin-cli/commands/operating-system/operating-system-create.md @@ -10,7 +10,7 @@ definition. ## SYNOPSIS **nico-admin-cli operating-system create** \<**-n**\|**--name**\> -\<**-o**\|**--org**\> \[**--id**\] \[**-d**\|**--description**\] +\[**-o**\|**--org**\] \[**--id**\] \[**-d**\|**--description**\] \[**--is-active**\] \[**--allow-override**\] \[**--phone-home-enabled**\] \[**--user-data**\] \[**--ipxe-script**\] \[**--ipxe-template-id**\] \[**--param**\] \[**--extended**\] @@ -26,7 +26,8 @@ Create a new operating system definition. Name of the operating system definition. **-o**, **--org** *\* -Organization identifier for this OS definition. +Optional Tenant organization identifier for this OS definition. Omit it for a +provider-owned definition. An explicitly empty value is invalid. **--id** *\* Optional UUID for the new OS definition (default: server-generated). @@ -63,7 +64,9 @@ Optional cloud-init / user-data script. Raw iPXE boot script (mutually exclusive with --ipxe-template-id). **--ipxe-template-id** *\* -ID of the iPXE template to use (mutually exclusive with --ipxe-script). +ID of the iPXE template to use (mutually exclusive with --ipxe-script). See +[Templated iPXE Operating Systems](../../../../configuration/templated-ipxe-operating-systems.md) +for template requirements, artifact configuration, and synchronization rules. **--param** *\* iPXE parameter in KEY=VALUE format. May be repeated. @@ -91,8 +94,9 @@ Print help (see a summary with -h) ## Examples ```sh -nico-admin-cli operating-system create --name ubuntu-22.04 --org fds34511233a -nico-admin-cli operating-system create --name ubuntu-22.04 --org fds34511233a --description "Ubuntu 22.04 base" --is-active false --allow-override +nico-admin-cli operating-system create --name provider-ubuntu-22.04 +nico-admin-cli operating-system create --name tenant-ubuntu-22.04 --org fds34511233a +nico-admin-cli operating-system create --name tenant-ubuntu-22.04 --org fds34511233a --description "Ubuntu 22.04 base" --is-active false --allow-override ``` --- diff --git a/docs/manuals/nico-admin-cli/commands/operating-system/operating-system-set-cached-url.md b/docs/manuals/nico-admin-cli/commands/operating-system/operating-system-set-cached-url.md index 7c5d146495..325b3c6d34 100644 --- a/docs/manuals/nico-admin-cli/commands/operating-system/operating-system-set-cached-url.md +++ b/docs/manuals/nico-admin-cli/commands/operating-system/operating-system-set-cached-url.md @@ -16,6 +16,10 @@ on OS artifacts. Set or clear cached_url on OS artifacts. +See +[Templated iPXE Operating Systems](../../../../configuration/templated-ipxe-operating-systems.md) +for cache strategies, readiness behavior, and the complete workflow. + ## OPTIONS **--set** *\* @@ -54,4 +58,4 @@ nico-admin-cli operating-system set-cached-url 12345678-1234-5678-90ab-cdef01234 --- -**See also:** [Tenant commands](../../tenant.md) · [CLI reference index](../../README.md) +**See also:** [Tenant commands](../../tenant.md) · [CLI reference index](../../index.md) diff --git a/docs/manuals/nico-admin-cli/commands/operating-system/operating-system-update.md b/docs/manuals/nico-admin-cli/commands/operating-system/operating-system-update.md index 24cbb3b1f5..c1e60c5337 100644 --- a/docs/manuals/nico-admin-cli/commands/operating-system/operating-system-update.md +++ b/docs/manuals/nico-admin-cli/commands/operating-system/operating-system-update.md @@ -71,7 +71,9 @@ Update the cloud-init / user-data script. Update the raw iPXE boot script. **--ipxe-template-id** *\* -Update the iPXE template ID. +Update the iPXE template ID. See +[Templated iPXE Operating Systems](../../../../configuration/templated-ipxe-operating-systems.md) +for template requirements, artifact configuration, and synchronization rules. **--param** \[*\...*\] Replace all iPXE parameters with these KEY=VALUE pairs. May be repeated.