Skip to content

Extend Cache runtime framework Phase 2: support curvine poc #5762

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
xliuqq wants to merge 8 commits into
base: master
Choose a base branch
from
+2,557 −48
Open

Extend Cache runtime framework Phase 2: support curvine poc #5762

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
ecadd42
init poc, add prepare ufs
xliuqq Apr 9, 2026
a9fbb2a
update crds
xliuqq Apr 9, 2026
0c575aa
fix lint error
xliuqq Apr 9, 2026
a8e65a0
add e2e test
xliuqq Apr 9, 2026
e65c914
finish e2e test
xliuqq Apr 9, 2026
2dd767a
remove self image prefix
xliuqq Apr 9, 2026
593055d
make gen-openapi
xliuqq Apr 10, 2026
a29c0b7
make fmt
xliuqq Apr 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions api/v1alpha1/cacheruntimeclass_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,25 @@ type RuntimeComponentDefinition struct {
// Dependencies specifies the dependencies required by the component
// +optional
Dependencies RuntimeComponentDependencies `json:"dependencies,omitempty"`

// ExecutionEntries entries to support out-of-tree integration.
// +optional
ExecutionEntries *ExecutionEntries `json:"executionEntries,omitempty"`
}

type ExecutionEntries struct {
// MountUFS defines the operations for mounting UFS
MountUFS *ExecutionCommonEntry `json:"mountUFS,omitempty"`

// ReportSummary it defines the operation how to get cache status like capacity, hit ratio etc.
ReportSummary *ExecutionCommonEntry `json:"reportSummary,omitempty"`
}

type ExecutionCommonEntry struct {
Command []string `json:"command"`

// Timeout is the timeout(seconds) for the execution entry
Timeout int `json:"timeout,omitempty"`
}

// EncryptOptionComponentDependency defines the configuration for encrypt option dependency
Expand Down
75 changes: 75 additions & 0 deletions charts/fluid/fluid/crds/data.fluid.io_cacheruntimeclasses.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,31 @@ spec:
type: array
type: object
type: object
executionEntries:
properties:
mountUFS:
properties:
command:
items:
type: string
type: array
timeout:
type: integer
required:
- command
type: object
reportSummary:
properties:
command:
items:
type: string
type: array
timeout:
type: integer
required:
- command
type: object
type: object
options:
additionalProperties:
type: string
Expand Down Expand Up @@ -3453,6 +3478,31 @@ spec:
type: array
type: object
type: object
executionEntries:
properties:
mountUFS:
properties:
command:
items:
type: string
type: array
timeout:
type: integer
required:
- command
type: object
reportSummary:
properties:
command:
items:
type: string
type: array
timeout:
type: integer
required:
- command
type: object
type: object
options:
additionalProperties:
type: string
Expand Down Expand Up @@ -6843,6 +6893,31 @@ spec:
type: array
type: object
type: object
executionEntries:
properties:
mountUFS:
properties:
command:
items:
type: string
type: array
timeout:
type: integer
required:
- command
type: object
reportSummary:
properties:
command:
items:
type: string
type: array
timeout:
type: integer
required:
- command
type: object
type: object
options:
additionalProperties:
type: string
Expand Down
4 changes: 2 additions & 2 deletions charts/fluid/fluid/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,9 @@ image:
imagePullSecrets: []

# Default registry, namespace and version tag for images managed by fluid
imagePrefix: &defaultImagePrefix fluidcloudnative
imagePrefix: &defaultImagePrefix registry.cn-hangzhou.aliyuncs.com/xliu1992
# imagePrefix: &defaultImagePrefix registry.aliyuncs.com/fluid
version: &defaultVersion v1.1.0-b457855
version: &defaultVersion v1.1.0-b0bdac58
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot Apr 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The image prefix and version have been changed to a personal registry and a specific commit hash. These should be reverted to the default values before merging to ensure the chart remains portable and points to official releases.


crdUpgrade:
enabled: true
Expand Down
75 changes: 75 additions & 0 deletions config/crd/bases/data.fluid.io_cacheruntimeclasses.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,31 @@ spec:
type: array
type: object
type: object
executionEntries:
properties:
mountUFS:
properties:
command:
items:
type: string
type: array
timeout:
type: integer
required:
- command
type: object
reportSummary:
properties:
command:
items:
type: string
type: array
timeout:
type: integer
required:
- command
type: object
type: object
options:
additionalProperties:
type: string
Expand Down Expand Up @@ -3453,6 +3478,31 @@ spec:
type: array
type: object
type: object
executionEntries:
properties:
mountUFS:
properties:
command:
items:
type: string
type: array
timeout:
type: integer
required:
- command
type: object
reportSummary:
properties:
command:
items:
type: string
type: array
timeout:
type: integer
required:
- command
type: object
type: object
options:
additionalProperties:
type: string
Expand Down Expand Up @@ -6843,6 +6893,31 @@ spec:
type: array
type: object
type: object
executionEntries:
properties:
mountUFS:
properties:
command:
items:
type: string
type: array
timeout:
type: integer
required:
- command
type: object
reportSummary:
properties:
command:
items:
type: string
type: array
timeout:
type: integer
required:
- command
type: object
type: object
options:
additionalProperties:
type: string
Expand Down
66 changes: 65 additions & 1 deletion pkg/common/cacheruntime.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,11 @@

package common

import (
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

type ComponentType string

const (
Expand All @@ -24,12 +29,71 @@
CacheEngineImpl = CacheRuntime
)

const (
ComponentTypeMaster ComponentType = "master"
ComponentTypeWorker ComponentType = "worker"
ComponentTypeClient ComponentType = "client"
)

type CacheRuntimeValue struct {
// RuntimeIdentity is used to identify the runtime (name/namespace)
RuntimeIdentity RuntimeIdentity `json:"runtimeIdentity"`

Master *CacheRuntimeComponentValue `json:"master,omitempty"`
Worker *CacheRuntimeComponentValue `json:"worker,omitempty"`
Client *CacheRuntimeComponentValue `json:"client,omitempty"`
}

// CacheRuntimeComponentValue is the common value for building CacheRuntimeValue.
type CacheRuntimeComponentValue struct {
Enabled bool `json:"enabled"`
// Component name, not Runtime name
Name string
Namespace string
Enabled bool
WorkloadType metav1.TypeMeta
Replicas int32
PodTemplateSpec corev1.PodTemplateSpec
Owner *OwnerReference
ComponentType ComponentType `json:"componentType,omitempty"`

// Service name, can be not same as Component name
Service *CacheRuntimeComponentServiceConfig
}

// CacheRuntimeConfig defines the config of runtime, will be auto mounted by configmap in the component pod.
type CacheRuntimeConfig struct {
// Mounts from Dataset Spec
Mounts []MountConfig `json:"mounts,omitempty"`
// AccessModes from Dataset Spec
AccessModes []corev1.PersistentVolumeAccessMode `json:"accessModes,omitempty"`
// fuse mount path, used in Worker or Client Pod according to Topology.
TargetPath string `json:"targetPath,omitempty"`

Master *CacheRuntimeComponentConfig `json:"master,omitempty"`
Worker *CacheRuntimeComponentConfig `json:"worker,omitempty"`
Client *CacheRuntimeComponentConfig `json:"client,omitempty"`
}

// MountConfig defines the mount config about dataset Mounts
type MountConfig struct {
MountPoint string `json:"mountPoint"`
// TODO: separate encrypt options with mount files for security

Check warning on line 80 in pkg/common/cacheruntime.go

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Complete the task associated to this TODO comment. 


See more on https://sonarcloud.io/project/issues?id=fluid-cloudnative_fluid&issues=AZ1dIvpwUbbqgtr70QLG&open=AZ1dIvpwUbbqgtr70QLG&pullRequest=5762
Options map[string]string `json:"options,omitempty"`
Name string `json:"name,omitempty"`
Path string `json:"path,omitempty"`
ReadOnly bool `json:"readOnly,omitempty"`
Shared bool `json:"shared,omitempty"`
}

type CacheRuntimeComponentConfig struct {
Enabled bool `json:"enabled,omitempty"`
Name string `json:"name,omitempty"`
Options map[string]string `json:"options,omitempty"`
Replicas int32 `json:"replicas,omitempty"`

Service CacheRuntimeComponentServiceConfig `json:"service,omitempty"`
}

type CacheRuntimeComponentServiceConfig struct {
Name string `json:"name"`
}
2 changes: 2 additions & 0 deletions pkg/common/constants.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,8 @@ const (
PodRoleType = "role"
DataloadPod = "dataload-pod"
NamespaceFluidSystem = "fluid-system"

DefaultNameSpace = "default"
)

const (
Expand Down
9 changes: 9 additions & 0 deletions pkg/common/label.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,15 @@ const (
UpdateLabel OperationType = "UpdateValue"
)

// label and annotations for cacheRuntime
const (
CacheRuntimeLabelAnnotationPrefix = "cacheruntime." + LabelAnnotationPrefix

LabelCacheRuntimeName = CacheRuntimeLabelAnnotationPrefix + "name"

LabelCacheRuntimeComponentName = CacheRuntimeLabelAnnotationPrefix + "component-name"
)

// LabelToModify modifies the labelKey in operationType.
type LabelToModify struct {
labelKey string
Expand Down
5 changes: 4 additions & 1 deletion pkg/ddc/base/runtime.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -433,7 +433,7 @@
return tieredstoreInfo, nil
}

// GetRuntimeInfo gets the RuntimeInfo according to name and namespace of it
// GetRuntimeInfo gets the RuntimeInfo according to name and namespace of it, must be called after dataset bound.
func GetRuntimeInfo(reader client.Reader, name, namespace string) (runtimeInfo RuntimeInfoInterface, err error) {
dataset, err := utils.GetDataset(reader, name, namespace)
if err != nil {
Expand Down Expand Up @@ -574,6 +574,8 @@
}
runtimeInfo.SetFuseNodeSelector(cacheRuntime.Spec.Client.NodeSelector)
runtimeInfo.SetupFuseCleanPolicy(cacheRuntime.Spec.Client.CleanPolicy)
// TODO(cache runtime): is this common logic for all runtimes? If so, move to below 'SetOwnerDatasetUID' line.

Check warning on line 577 in pkg/ddc/base/runtime.go

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Complete the task associated to this TODO comment. 


See more on https://sonarcloud.io/project/issues?id=fluid-cloudnative_fluid&issues=AZ1dIvjWUbbqgtr70QK3&open=AZ1dIvjWUbbqgtr70QK3&pullRequest=5762
runtimeInfo.SetupWithDataset(dataset)
default:
err = fmt.Errorf("fail to get runtimeInfo for runtime type: %s", runtimeType)
return
Expand Down Expand Up @@ -643,6 +645,7 @@
return status, err
}
return &runtime.Status, nil
// TODO: how to handle with cache runtime? (currently used in app pod affinity scene)

Check warning on line 648 in pkg/ddc/base/runtime.go

View check run for this annotation

SonarQubeCloud / SonarCloud Code Analysis

Complete the task associated to this TODO comment. 


See more on https://sonarcloud.io/project/issues?id=fluid-cloudnative_fluid&issues=AZ1xTmdbdbsKkNdFZtMd&open=AZ1xTmdbdbsKkNdFZtMd&pullRequest=5762
default:
err = fmt.Errorf("fail to get runtimeInfo for runtime type: %s", runtimeType)
return nil, err
Expand Down
Loading
Loading