From 6b625398fc70013a5b865f3a59844e9fcdb79325 Mon Sep 17 00:00:00 2001 From: Runlin Xiao Date: Wed, 10 Jun 2026 22:23:14 -0700 Subject: [PATCH] docs: document storageContainerName for explicit ClusterStorageContainer selection Documents the new spec.predictor.storageContainerName field added in kserve/kserve#5314, and regenerates the CRD API reference from master. Signed-off-by: Runlin Xiao --- .../storage-containers/storage-containers.md | 37 +- docs/reference/crd-api.mdx | 463 +++++++++++++++++- 2 files changed, 486 insertions(+), 14 deletions(-) diff --git a/docs/model-serving/storage/storage-containers/storage-containers.md b/docs/model-serving/storage/storage-containers/storage-containers.md index b26f8dce6..a3eecd689 100644 --- a/docs/model-serving/storage/storage-containers/storage-containers.md +++ b/docs/model-serving/storage/storage-containers/storage-containers.md @@ -46,7 +46,7 @@ In a _ClusterStorageContainer_ spec, you can specify credentials for cloud stora :::warning -If a storage URI is supported by two or more _ClusterStorageContainer_ CRs, there is no guarantee which one will be used. **Please make sure that the URI format is only supported by one ClusterStorageContainer CR**. +If a storage URI is supported by two or more _ClusterStorageContainer_ CRs and `storageContainerName` is not set, there is no guarantee which one will be used. Set [`spec.predictor.storageContainerName`](#explicitly-selecting-a-storage-container) on the `InferenceService` to explicitly select the one to use. ::: @@ -162,6 +162,41 @@ The respective secret should be created in the same namespace as the `InferenceS kubectl create secret generic hf-secret --from-literal=HF_TOKEN= ``` +## Explicitly Selecting a Storage Container + +By default, KServe selects a `ClusterStorageContainer` by matching the storage URI against each CR's `supportedUriFormats`. When more than one CR supports the same URI format, you can set `storageContainerName` on the predictor spec to explicitly select which one to use, similar to `storageClassName` on a PersistentVolumeClaim. + +For example, both the `default` and the `hf-hub` ClusterStorageContainers above support the `hf://` URI format. To make sure the `hf-hub` container (which carries the Hugging Face credentials) is used, reference it by name: + +```yaml +apiVersion: serving.kserve.io/v1beta1 +kind: InferenceService +metadata: + name: huggingface-llama3 +spec: + predictor: + storageContainerName: hf-hub + model: + modelFormat: + name: huggingface + storageUri: hf://meta-llama/meta-llama-3-8b-instruct +``` + +When `storageContainerName` is set, there is no fallback to auto-matching. The referenced `ClusterStorageContainer` must: + +- exist in the cluster, +- not be disabled, +- have `workloadType: initContainer` (the default), +- support the model's `storageUri` in its `supportedUriFormats`. + +If any of these checks fail, the deployment fails with an error instead of falling back to another storage container. + +:::note + +`storageContainerName` is only available on the predictor spec. Explainers and transformers always use URI-based auto-matching, and `LLMInferenceService` does not support this field yet. + +::: + ## Spec Attributes Spec attributes are in [API Reference](../../../reference/crd-api.mdx#clusterstoragecontainer) doc. diff --git a/docs/reference/crd-api.mdx b/docs/reference/crd-api.mdx index 8b696fe59..71ee50162 100644 --- a/docs/reference/crd-api.mdx +++ b/docs/reference/crd-api.mdx @@ -948,8 +948,11 @@ TrainedModelList contains a list of TrainedModel - [ModelStatus](#modelstatus) - [NamespacedName](#namespacedname) - [NodeStatus](#nodestatus) +- [ObservedGateway](#observedgateway) +- [ObservedSchedulerStatus](#observedschedulerstatus) - [ParallelismSpec](#parallelismspec) - [RouterSpec](#routerspec) +- [RouterStatus](#routerstatus) - [ScaleMetric](#scalemetric) - [ScalingSpec](#scalingspec) - [SchedulerConfigSpec](#schedulerconfigspec) @@ -1867,6 +1870,11 @@ LLMInferenceServiceStatus defines the observed state of LLMInferenceService. type="[Addressable](#addressable) array" required="false" description="Addresses is a list of addresses for different protocols (HTTP and HTTPS)
If Addresses is present, Address must be ignored by clients." +/> @@ -1925,7 +1933,22 @@ LoRASpec defines the configuration for LoRA adapters. name="adapters" type="[LLMModelSpec](#llmmodelspec) array" required="false" - description="Adapters is the static specification for one or more LoRA adapters.
Each adapter is defined by its own ModelSpec.
This type is recursive https://github.com/kubernetes-sigs/controller-tools/issues/585" + description="Adapters specifies one or more LoRA adapters to load alongside the base model.
Supported URI schemes: hf://, s3://, pvc://.
Each adapter must have a unique name that differs from the base model name.
This type is recursive https://github.com/kubernetes-sigs/controller-tools/issues/585" +/> @@ -1999,6 +2022,11 @@ LocalModelCacheSpec type="[NamespacedName](#namespacedname) array" required="true" description="Inference services using this local model" +/> @@ -2377,6 +2405,80 @@ NodeStatus enum required="false" description="" /> +#### ObservedGateway + + + +**Appears in:** +- [RouterStatus](#routerstatus) + +ObservedGateway is a Gateway reference with the listeners and HTTPRoutes +bound to this service through it. Used in status to record observed routing topology. + +##### Fields + + + + + + +#### ObservedSchedulerStatus + + + +**Appears in:** +- [RouterStatus](#routerstatus) + +ObservedSchedulerStatus records the scheduler-related resources observed +during the last successful routing reconciliation. + +##### Fields + + + + + + #### ParallelismSpec @@ -2461,6 +2563,34 @@ It supports Kubernetes Ingress and the Gateway API. The fields are mutually excl /> +#### RouterStatus + + + +**Appears in:** +- [LLMInferenceServiceStatus](#llminferenceservicestatus) + +RouterStatus records the networking resources observed during the last +successful routing reconciliation. Nil when routing is not configured or +the service is stopped. + +##### Fields + + + + + + #### ScaleMetric **Underlying type:** string @@ -2544,13 +2674,13 @@ When scaling is configured, the controller creates and manages autoscaling resou SchedulerSpec defines the Inference Gateway extension configuration. The SchedulerSpec configures the connection from the Gateway to the model deployment leveraging the LLM optimized -request Scheduler, also known as the Endpoint Picker (EPP) which determines the exact pod that should handle the -request and responds back to Envoy with the target pod, Envoy will then forward the request to the chosen pod. +request Scheduler, also known as the Endpoint Picker (EPP). The EPP determines the exact pod that should handle the +request and responds back to the Gateway with the target pod. The Gateway will then forward the request to the chosen pod. The Scheduler is only effective when having multiple inference pod replicas. -Step 1: Gateway (Envoy) <-- ExtProc --> EPP (select the optimal replica to handle the request) -Step 2: Gateway (Envoy) <-- forward request --> Inference Pod X +Step 1 (endpoint selection): Gateway <-- ExtProc --> EPP (select the optimal replica to handle the request) +Step 2 (endpoint routing): Gateway <-- forward request/response --> Inference Pod X ##### Fields @@ -3433,6 +3563,8 @@ LLMInferenceServiceList is the list type for LLMInferenceService. ### Available Types - [ActuatorSpec](#actuatorspec) +- [AppliedConfigRef](#appliedconfigref) +- [AppliedConfigSource](#appliedconfigsource) - [GatewayObjectReference](#gatewayobjectreference) - [GatewayRoutesSpec](#gatewayroutesspec) - [GatewaySpec](#gatewayspec) @@ -3447,15 +3579,21 @@ LLMInferenceServiceList is the list type for LLMInferenceService. - [LLMInferenceServiceValidator](#llminferenceservicevalidator) - [LLMModelSpec](#llmmodelspec) - [LoRASpec](#loraspec) +- [ModelSourcedAddressStatus](#modelsourcedaddressstatus) +- [ObservedGateway](#observedgateway) +- [ObservedSchedulerStatus](#observedschedulerstatus) - [ParallelismSpec](#parallelismspec) - [RouterSpec](#routerspec) +- [RouterStatus](#routerstatus) - [ScalingSpec](#scalingspec) - [SchedulerConfigSpec](#schedulerconfigspec) - [SchedulerSpec](#schedulerspec) +- [SourcedAddress](#sourcedaddress) - [StorageInitializerSpec](#storageinitializerspec) - [UntypedObjectReference](#untypedobjectreference) - [WVASpec](#wvaspec) - [WorkloadSpec](#workloadspec) +- [WorkloadStatus](#workloadstatus) ### Type Definitions #### ActuatorSpec @@ -3485,6 +3623,62 @@ Exactly one of HPA or KEDA must be specified. /> +#### AppliedConfigRef + + + +**Appears in:** +- [LLMInferenceServiceStatus](#llminferenceservicestatus) + +AppliedConfigRef identifies an LLMInferenceServiceConfig resource that contributed +to the final merged configuration during reconciliation. + +##### Fields + + + + + + +#### AppliedConfigSource + +**Underlying type:** string + +**Appears in:** +- [AppliedConfigRef](#appliedconfigref) + +AppliedConfigSource identifies how a configuration was selected for merging. + + + +##### Possible Values + + #### GatewayObjectReference @@ -3827,12 +4021,27 @@ LLMInferenceServiceStatus defines the observed state of LLMInferenceService. name="address" type="[Addressable](#addressable)" required="false" - description="Address is a single Addressable address.
If Addresses is present, Address will be ignored by clients." + description="Deprecated: Address is retained for CRD schema compatibility.
It is never populated; use Addresses instead." /> @@ -3886,7 +4095,117 @@ LoRASpec defines the configuration for LoRA adapters. name="adapters" type="[LLMModelSpec](#llmmodelspec) array" required="false" - description="Adapters is the static specification for one or more LoRA adapters.
Each adapter is defined by its own ModelSpec.
This type is recursive https://github.com/kubernetes-sigs/controller-tools/issues/585" + description="Adapters is a list of LoRA (Low-Rank Adaptation) adapters to attach to the base model.
Each adapter is specified by name and URI (supports hf://, s3://, and pvc:// schemes).
The controller automatically downloads adapters and configures the runtime to use them.
This type is recursive https://github.com/kubernetes-sigs/controller-tools/issues/585" +/> + + +#### ModelSourcedAddressStatus + + + +**Appears in:** +- [SourcedAddress](#sourcedaddress) + + + +##### Fields + + + + + + +#### ObservedGateway + + + +**Appears in:** +- [RouterStatus](#routerstatus) + +ObservedGateway is a Gateway reference with the listeners and HTTPRoutes +bound to this service through it. Used in status to record observed routing topology. + +##### Fields + + + + + + +#### ObservedSchedulerStatus + + + +**Appears in:** +- [RouterStatus](#routerstatus) + +ObservedSchedulerStatus records the scheduler-related resources observed +during the last successful routing reconciliation. + +##### Fields + + + + @@ -3974,6 +4293,34 @@ It supports Kubernetes Ingress and the Gateway API. The fields are mutually excl /> +#### RouterStatus + + + +**Appears in:** +- [LLMInferenceServiceStatus](#llminferenceservicestatus) + +RouterStatus records the networking resources observed during the last +successful routing reconciliation. Nil when routing is not configured or +the service is stopped. + +##### Fields + + + + + + #### ScalingSpec @@ -4045,13 +4392,13 @@ When scaling is configured, the controller creates and manages autoscaling resou SchedulerSpec defines the Inference Gateway extension configuration. The SchedulerSpec configures the connection from the Gateway to the model deployment leveraging the LLM optimized -request Scheduler, also known as the Endpoint Picker (EPP) which determines the exact pod that should handle the -request and responds back to Envoy with the target pod, Envoy will then forward the request to the chosen pod. +request Scheduler, also known as the Endpoint Picker (EPP). The EPP determines the exact pod that should handle the +request and responds back to the Gateway with the target pod. The Gateway will then forward the request to the chosen pod. The Scheduler is only effective when having multiple inference pod replicas. -Step 1: Gateway (Envoy) <-- ExtProc --> EPP (select the optimal replica to handle the request) -Step 2: Gateway (Envoy) <-- forward request --> Inference Pod X +Step 1 (endpoint selection): Gateway <-- ExtProc --> EPP (select the optimal replica to handle the request) +Step 2 (endpoint routing): Gateway <-- forward request/response --> Inference Pod X ##### Fields @@ -4090,6 +4437,53 @@ Step 2: Gateway (Envoy) <-- forward request --> Inference Pod X /> +#### SourcedAddress + + + +**Appears in:** +- [LLMInferenceServiceStatus](#llminferenceservicestatus) + +SourcedAddress extends Addressable with the networking resource that +produced this address, enabling consumers to select endpoints by origin. + +##### Fields + + + + + + #### StorageInitializerSpec @@ -4235,6 +4629,44 @@ WorkloadSpec defines the configuration for a deployment workload, such as replic /> +#### WorkloadStatus + + + +**Appears in:** +- [LLMInferenceServiceStatus](#llminferenceservicestatus) + +WorkloadStatus records the workload resources observed during the last +successful reconciliation. Nil when no workload resources have been +created yet, or when the service is stopped. + +##### Fields + + + + + + ## serving.kserve.io/v1beta1 @@ -7721,6 +8153,11 @@ The following fields follow a "1-of" semantic. Users must specify exactly one sp type="[StorageUri](#storageuri) array" required="true" description="Spec for multiple storage uris." +/>