Skip to content

About

Experimental Kubernetes operator for CUBRID native HA clusters

Resources

Code of conduct

Contributing

Security policy

Stars

2 stars

Watchers

0 watching

Forks

CUBRID Lab Operator icon (Go)

CUBRID Kubernetes Operator

An experimental Kubernetes-native operator for running highly available CUBRID clusters.

Important

This project is early-stage and under active development. It is an experimental project under cubrid-lab and is not yet production-ready. Architecture, implementation and live validation are tracked separately in ROADMAP.md. It does not replace the existing official CUBRID Operator. See Relationship to the existing CUBRID Operator.

It is a Kubernetes control plane for CUBRID that coordinates CUBRID native HA with the Kubernetes lifecycle: CUBRID elects the master and performs failover, and the Operator reconciles Pods, storage, Brokers and Services around it, reports what it observes, and checks failure scenarios with real SQL.

New to the project? docs/architecture/ explains the architecture with diagrams before you read the design or the code. For operational inspection, SQL/backup/recovery boundaries and installation prerequisites, start with the pre-release operator guide. It is not yet an independently verified installation/recovery runbook (#125/#127).

Relationship to the existing CUBRID Operator

CUBRID already provides an official Kubernetes Operator, CUBRID/cubrid-operator. It deploys CUBRID HA, reports the current master, removes Pods whose Broker is not listening from Service Endpoints, schedules backups and supports scaling.

This project does not replace it and offers no migration path. It explores different design and validation questions: a primary resolved only from the agreeing answers of every member, an authenticated Instance Manager API instead of pods/exec, a separate read-write and read-only Broker tier, verified backups restored into a new cluster, and failure scenarios checked with real SQL.

docs/official-operator-comparison.md compares the two at pinned revisions, with the evidence level of each claim, the open questions and the opportunities to contribute findings upstream. Measured side-by-side scenarios are tracked in #93.

What it manages

  • CubridCluster: a standalone database or a three-member CUBRID HA cluster (one active master, two failover-capable slaves), its storage, its Broker tier and its client Services. Restore into a new cluster is a bootstrap mode of it (spec.bootstrap.recovery).
  • CubridBackup: a backup of a member to object storage.

The API version is v1alpha1 and may change between alpha revisions (docs/compatibility.md). A later build never silently reinterprets or destroys the stored data of a cluster an earlier one created, and migrates none automatically; what else it promises is in docs/governance.md. The v0.1 scope, what is implemented and what has been validated, and at which level, are in ROADMAP.md. CUBRID engine-version migration is out of scope for the first release (ADR-0009).

A minimal example (not for production)

A single-member cluster, from config/samples/database_v1alpha1_cubridcluster.yaml:

apiVersion: database.cubrid.io/v1alpha1
kind: CubridCluster
metadata:
  name: cubridcluster-sample
spec:
  version: "11.4"
  databases:
    - name: appdb
  topology:
    promotableMembers: 1
    readReplicas: 0
  highAvailability:
    enabled: false
  storage:
    data:
      size: 1Gi

Installing the CRDs and the Operator (make install, make deploy IMG=...) and the prerequisites they assume are described in section 2 of the operator guide. Use a disposable cluster: these targets are not a validated release installer.

Client endpoints

Applications connect through Brokers, never to a DB Pod directly (ADR-0002). Every CubridCluster, HA or standalone, gets two Services:

Service Port HA cluster Standalone cluster
<cluster>-rw 33000/TCP reads and writes on the current master reads and writes on the one server
<cluster>-ro 33001/TCP reads; writes refused with -581 reads of the same one server, not a replica; writes refused with -581

The standalone behavior is that of POC-33 in docs/poc/RESULTS.md. The BrokerReady, WriteEndpointReady, ReadEndpointReady and RoutingReady Conditions say whether the Brokers are available (docs/observability.md); they do not prove that a client can run SQL.

v0.1 release scope and known HA limitations

Scope decision: 2026-11-21 UTC. v0.1 is not released or production-ready. A release candidate must still pass its mandatory scenarios on the pinned CUBRID 11.4.x/amd64 three-worker reference lab and an independently reproduced runbook. The candidate freeze is targeted for 2026-12-07, followed by repeated scenarios and a 24-hour soak (Dec 08–14), independent reproduction (Dec 15–20), and a conditional release review (Dec 21–27). See ROADMAP.md and the scenario contract. These dates do not waive incomplete evidence or authorize a lab fault or release.

The following limitations are part of the proposed v0.1 experimental claim, not guarantees of zero data loss or general CUBRID HA safety:

  • Actual VM power loss (S04): the required outcome is observed role and managed-routing recovery. Recovery time and acknowledged-write loss are measured and disclosed, not judged against the earlier 30-second limit. With the pinned engine's default ha_copy_log_timeout=5, a stalled copylogdb -m sync can stop copying while the master continues to acknowledge commits. An induced copier-stall + VM-off run found 13 acknowledged writes absent from the promoted master (#1035, POC-69). Do not treat a healthy role or Ready=True as durable replication evidence.
  • Network partitions / repeated failover (S07–S08): write quarantine is best-effort isolation of the Operator-managed -rw Broker path, not database-process fencing. Each candidate run must measure time to actual closure (Brokers, Endpoints and SQL), and independently establish no acknowledged writes after that closure on new and established sessions. During earlier partition/heal trials a higher-ranked member lacked up to 73 writes already acknowledged before closure (#702). The Operator cannot prevent native heartbeat promotion or guarantee that every acknowledged write survives it; directly connected database clients are outside managed -rw isolation.
  • Graceful primary Pod deletion (S03): abrupt-failure variants remain mandatory, but the graceful variant's recovery time and the cause of its unnecessary quarantine are published as measured limitations (#938), without converting other safety failures to passes.
  • Recovery: explicitly authorized rebuild onto verified replacement storage (S10) remains mandatory. Automatic rejoin/adoption of potentially divergent retained data (S09) and automatic adoption of old HA PVCs on cluster recreation (#798) are deferred to v0.2. Preserve original volumes and follow the reviewed manual recovery procedure; an accepted rebuild request alone does not authorize deleting data.
  • API group: database.cubrid.io is not proven owned by this project (#20). The v0.1 experimental API may need a breaking CRD recreation in a later version; do not assume an in-place upgrade.

The earlier Kind regression for observing/enforcing quarantine under denied child writes (#949) has been completed; it is not proof of VM partition durability. Historical failed SQL/VM runs remain recorded and must never be relabelled as successful. The mandatory candidate matrix, observational results and current test-harness alignment gaps are tracked in the scenario contract.

Security limitations

  • The DBA password is empty unless the cluster names one. A standalone or an HA cluster can take its DBA password from a Secret with spec.dbaPasswordSecretRef (docs/operations/dba-password.md). Every other database is created with an empty DBA password (#801), so every client the Broker NetworkPolicy admits can connect to it as dba. Changing the password later is not supported, and on an HA cluster a changed Secret value is not reported.
  • No traffic is encrypted: not between clients and Brokers, Brokers and the database, members, or the Operator and the Instance Manager. The Instance Manager requires a bearer token.
  • NetworkPolicies are enforced only by a network plugin that implements them, and they are not authentication. The namespace is the trust boundary: restrict who can create Pods in the cluster's and the Operator's namespaces.

Do not expose this configuration to untrusted networks or production data. docs/operations/network-access.md has the policies and their limits.

Operating a cluster

These pages hold the details that used to be in this README:

  • Manager watch namespace and scoped RBAC: --watch-namespace, what the manager caches, and the Role a restricted candidate needs.
  • Network access: the two ingress NetworkPolicies per cluster, spec.networkPolicy, and their limits.
  • Member placement: spec.placement.memberSpread, its default for new and older clusters, and the MemberPlacement Condition.
  • Voluntary disruption: the PodDisruptionBudgets and what a node drain can take.
  • DBA password: giving a standalone or an HA cluster its DBA password from a Secret, what the cluster reports, and what is not supported.

Contributing

We welcome bug reports, reproducible installation feedback, documentation, tests and focused fixes. Start with CONTRIBUTING.md, search issues and open PRs, and confirm the scope on an issue before coding. Core API, HA safety and recovery changes require design agreement.

Documentation

License

Project-authored source and documentation are licensed under Apache License 2.0, unless a file states otherwise. Dependencies, container images and third-party material retain their own terms. The license does not grant rights to CUBRID trademarks. See the third-party inventory.

About

Experimental Kubernetes operator for CUBRID native HA clusters

Resources

Code of conduct

Contributing

Security policy

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages