Lifecycle reference¶

Details behind Cluster lifecycle. Chart: lifecycle/.

Install keys¶

Defaults are in lifecycle/values.yaml. The schema rejects unknown keys, and rejects hive or siteconfig keys unless engine selects them.

install:
  platform: Agent                    # (1)!
  agentNamespace: hardware-inventory
  credential:
    namespace: hcp-credentials
    name: hostinventory
  services:
    APIServer:
      type: LoadBalancer
      loadBalancer:
        hostname: api-int-hcp-agent.dc1.example.com
    OAuthServer:
      type: Route                    # (2)!
    Konnectivity:
      type: Route
    Ignition:
      type: Route
  hostedCluster:
    spec:
      kubeAPIServerDNSName: api-hcp-agent.dc1.example.com
  nodePools:
    compute:                         # (3)!
      replicas: 3
      platform:
        agent:
          agentLabelSelector:
            matchLabels:
              example.com/class: fc830
  manifests:
    - apiVersion: metallb.io/v1beta1 # (4)!
      kind: IPAddressPool
      metadata:
        name: hcp-agent-apiserver
        namespace: metallb-system
      spec:
        addresses:
          - 10.20.9.13/32

# values/platforms/aws.yaml, trimmed
install:
  engine: hive
  credential:
    namespace: hive-credentials
    name: aws                       # (1)!
  hive:
    installConfig:
      controlPlane:
        name: master
        replicas: 3
        platform:
          aws:
            type: m6i.xlarge
      networking:
        networkType: OVNKubernetes

# fleet/hive-aws.yaml
install:
  baseDomain: aws.example.com
  hive:
    platform:
      aws:
        region: us-east-2           # (2)!

# values/platforms/baremetal.yaml
install:
  engine: siteconfig
  credential:
    namespace: bm-credentials
    name: hostinventory
  siteconfig:
    bmcCredential:
      namespace: bm-credentials
      name: bmc
    clusterInstance:
      networkType: OVNKubernetes

# fleet/sno-edge.yaml
install:
  siteconfig:
    clusterInstance:
      clusterType: SNO
      machineNetwork:
        - cidr: 10.20.30.0/24
    nodes:
      sno-edge.dc1.example.com:     # (1)!
        role: master
        bmcAddress: redfish-virtualmedia://10.20.0.11/redfish/v1/Systems/1
        bootMACAddress: "00:11:22:33:44:55"

Generated objects¶

<cluster> is the fleet file name. Everything not listed uses the operator's defaults: set it in hostedCluster, hive.clusterDeployment or siteconfig.clusterInstance.

Hive and SiteConfig reject release image changes, and SiteConfig any spec change before install completes. The chart reads those fields back from the live objects, so a version bump or credential rotation never touches them. Those clusters upgrade through upgrade.

Policy chains¶

The hub gets present-lifecycle-<cluster> or absent-lifecycle-<cluster>, picked by state. Each template waits for the one before it to be Compliant.

flowchart LR
  subgraph present
    pg[guard] --> pc[credentials] --> pm[manifests] --> pcl[cluster] --> pr[ready]
  end
  subgraph absent
    ag[guard] --> ad[detach] --> adt[detached] --> ax[destroy] --> axd[destroyed] --> acl[cleanup]
  end

Templates are named <state>-<step>-<cluster>, e.g. present-guard-hcp-kubevirt. siteconfig has no detach or detached.

present-lifecycle-<cluster>:

absent-lifecycle-<cluster>:

• An enforced delete is Compliant once the API accepts it, so every destroy step has an inform wait after it.
• Create templates are musthave: operator-filled fields are not compared, so the Policy never fights HyperShift, Hive or SiteConfig.
• The credential is never deleted.

Day-2 handoff¶

1. The ManagedCluster appears. hypershift: the hypershift-addon creates it once the HostedCluster is available. hive, siteconfig: ACM imports the one the chart or SiteConfig created.
2. It joins global, the GitOpsCluster creates its Argo CD secret, and ACM adds the clusterID label (Install PolicyStack).
3. The Day-2 ApplicationSet pairs the secret with the same fleet file and creates <element>-<cluster> Applications. It ignores hub, state and install.

Labels in hostedCluster.metadata.labels are copied to the ManagedCluster once, at creation.

Ownership¶

Guard and detach act only on a ManagedCluster carrying its engine's owner annotations:

A hand-made ManagedCluster needs these, or the guard stops both chains.

Immutable fields¶

The operators reject changes to these, among others:

Changing one does not change the cluster: present-cluster-<cluster> goes NonCompliant with the API server's message. Revert the value.

• NodePool upgradeType or arch: add a new pool, remove the old one.
• Failed SiteConfig install: set state: absent, then build again with the fix.

Removing a NodePool or host¶

musthave never deletes. Removing an entry only stops managing it.

oc delete nodepool <cluster>-<pool> -n clusters

install:
  nodePools:
    workers:
      replicas: 0

oc edit clusterinstance <cluster> -n <cluster>

Troubleshooting¶

Start with the Application for render errors (missing version, credential or required key), then the Policy:

oc get applications.argoproj.io lifecycle-<cluster> -n openshift-gitops -o jsonpath='{.status.conditions}'

oc annotate clusterdeployment <cluster> -n <cluster> hive.openshift.io/protected-delete-

oc get clusterinstance <cluster> -n <cluster> -o jsonpath='{.status.conditions}'

oc get klusterlet klusterlet-<cluster> -o jsonpath='{.metadata.finalizers}'

for m in $(oc get machines.cluster.x-k8s.io -n clusters-<cluster> -o name); do
  oc annotate "$m" -n clusters-<cluster> pre-terminate.delete.hook.machine.cluster.x-k8s.io/agentmachine-
done

Adding an engine¶

An engine <e> is a component of lifecycle/. templates/policy.yaml renders whichever component install.engine names, so the ApplicationSet and fleet keys do not change.

1. Add converters/<e>/: objects to create, the root object by identity alone (for destroy), and its credentials. lookup back every field the engine rejects changes to. End the cleanup converter with include "lifecycle.cleanupManifests".
2. Add stack.<e> to values.yaml with present and absent Policies following the chains. No Policy-level remediationAction or pruneObjectBehavior: the first overrides the inform gates, the second lets a Policy delete a cluster.
3. Add <e> to the install.engine enum in values.schema.json, with its keys. POLICY090 fails when the enum and stack keys differ (Rules).
4. Add an owner case to lifecycle.ownerAnnotations, a present and an absent fleet file to tools/validator/testdata/clusters/, and an example to Cluster lifecycle.

An engine that upgrades from the hub maps version onto its objects, like hypershift. One that only installs uses version at create and leaves upgrades to upgrade (Upgrades).