Clusters and Applications¶

Two ApplicationSets in openshift-gitops on the hub turn fleet files into Argo CD Applications:

Fleet files¶

One file per cluster: fleet/<cluster>.yaml, named exactly like its ManagedCluster. The hub uses fleet/hubs/<hubName>.yaml (The hub).

# fleet/prod-east-1.yaml
revision: main
channel: stable-4.20
valueFiles: # (1)!
  - environments/prod.yaml
  - datacenters/dc1.yaml
  - platforms/aws.yaml

# fleet/hcp-kubevirt.yaml
revision: main
hub: acm-dc1 # (1)!
version: 4.20.8
valueFiles:
  - environments/nonprod.yaml
  - platforms/kubevirt.yaml

The validator enforces: no other keys, every valueFiles entry exists, and a hub file sets no hub, state or install. version and channel go only in fleet files (Upgrades). policystack ignores the lifecycle keys.

Fleet files are read only from the default branch (HEAD). fleet/ on other branches does nothing. Changing the default branch re-points every cluster, and a cluster with no file there loses its Applications.

Changes land within about 6 minutes (3 minute generator requeue plus 3 minute branch cache). An ApplicationSet webhook makes them immediate, except a fix for a failed generation, which waits for the 3 minute retry.

Upstream ships no live fleet files. Examples are the validator fixtures in tools/validator/testdata/clusters/.

# config.example.com/environment.1: prod
# config.example.com/datacenter.2: dc1
valueFiles:
  - environments/prod.yaml
  - datacenters/dc1.yaml

PolicyStack reads five labels. You only ever set the last one.

Revision¶

A branch (slashes work), tag or commit SHA. The Render Diff PR comment shows each cluster's Policies at its revision before and after merge.

oc get applicationset policystack -n openshift-gitops \
  -o jsonpath='{.status.conditions[?(@.type=="ErrorOccurred")].message}'

Pin tags, SHAs or protected branches. A deleted pinned branch freezes the fleet, and GitHub deletes PR branches on merge when automatic head branch deletion is on.

The hub¶

The hub's ManagedCluster has local-cluster, so it reads fleet/hubs/<hubName>.yaml. hubName is set once at helm install, no default (Install PolicyStack).

# fleet/hubs/acm-dc1.yaml
revision: main
valueFiles:
  - environments/prod.yaml
  - datacenters/dc1.yaml
  - platforms/baremetal.yaml
  - acm/acm-dc1.yaml # (1)!

1. Hubs in one datacenter share a layer by listing it.

hubName replaces the cluster name everywhere on the hub:

Every hub reading the repo, DR hubs included, sets its own hubName. The chart fails without it rather than apply another hub's config. Check it with helm get values policystack -n openshift-gitops.

A Global Hub managed hub imported in default mode has no local-cluster; the global hub reads its file instead (Global Hub).

Protection¶

Other labels are ignored, and the five above are guarded by RBAC on ManagedClusters and openshift-gitops Secrets, so oc label cannot change what a cluster runs. Whoever merges to fleet/ decides, including destroying clusters with state: absent.

Add a CODEOWNERS entry and a default branch ruleset requiring code owner review and blocking force push and deletion:

# .github/CODEOWNERS
/fleet/ @<org>/<team>

To gate only production, own /fleet/prod-*.yaml and /fleet/hubs/ instead.

Generators¶

policystack, rendered with hubName=acm-dc1:

generators:
  - matrix:
      generators:
        - matrix:
            generators:
              - clusters: # (1)!
                  selector:
                    matchExpressions:
                      - key: clusterID
                        operator: Exists
              - git: # (2)!
                  repoURL: 'https://github.com/PolicyStack/PolicyStack.git'
                  revision: HEAD
                  pathParamPrefix: fleet # (3)!
                  files:
                  - path: 'fleet/{{if index .metadata.labels "local-cluster"}}hubs/acm-dc1{{else if eq (index .metadata.labels "global-hub.open-cluster-management.io/deploy-mode") "default"}}hubs/{{.name}}{{else}}{{.name}}{{end}}.yaml'
        - git: # (4)!
            repoURL: 'https://github.com/PolicyStack/PolicyStack.git'
            revision: '{{.revision}}'
            directories:
            - path: 'stack/*'

1. Every cluster secret with clusterID. The GitOpsCluster creates one per managed cluster.
2. The cluster's fleet file at HEAD. Its keys become parameters. No file, no Applications.
3. Stops the file's path from shadowing the element directory's path in the matrix.
4. Every stack/* directory at the cluster's revision. One Application per cluster and directory.

An element added on a branch reaches only clusters pinned to a revision containing it. Disabled elements still get an Application that renders nothing.

Lifecycle generator¶

policystack-lifecycle, rendered with hubName=acm-dc1:

generators:
  - matrix: # (1)!
      generators:
        - clusters:
            selector:
              matchExpressions:
                - key: local-cluster
                  operator: Exists
        - git: &fleet
            repoURL: 'https://github.com/PolicyStack/PolicyStack.git'
            revision: HEAD
            files:
            - path: 'fleet/*.yaml'
            - path: 'fleet/*/*.yaml' # (2)!
              exclude: true
          selector:
            matchExpressions:
              - key: hub
                operator: In
                values:
                  - "acm-dc1"
  - matrix: # (3)!
      generators:
        - git: *fleet
          selector:
            matchExpressions:
              - key: hub
                operator: Exists
        - clusters:
            selector:
              matchLabels:
                apps.open-cluster-management.io/cluster-name: '{{ if regexMatch "^[a-z0-9]([-a-z0-9]{0,61}[a-z0-9])?$" (toString .hub) }}{{ .hub }}{{ end }}' # (4)!
                global-hub.open-cluster-management.io/deploy-mode: default

1. Fleet files with hub: acm-dc1, paired with this hub's own cluster secret. A hub that does not manage itself builds nothing.
2. Argo CD's * crosses /. This drops hubs/ and other subdirectories.
3. Fleet files naming a managed hub imported in default mode (Global Hub). Fleet files come first because Argo CD templates a child generator's fields but not its selector.
4. A hub that is not a DNS label matches nothing instead of failing the whole ApplicationSet.

Files without hub match neither, so imported clusters get no lifecycle Application. The cluster secret's .name is the ManagedCluster the Policies are placed on.

Applications¶

The Application name is the Helm release name, which every object name derives from (Naming limits). Argo CD deploys only to the hub; ACM delivers policies to clusters (Placement). Objects land in policyNamespace; the destination namespace is unused. allowEmpty lets a disabled element prune to zero objects (Disabling an element).

Both ApplicationSets set preserveResourcesOnDeletion: false: deleting an Application deletes its hub objects (Removing an element). Deleting a lifecycle Application leaves the cluster running unmanaged (Abandon).

From policy-library 3.1.0, copyPolicyMetadata: false keeps Argo CD tracking metadata off ACM's per-cluster Policy copies, so Argo CD never prunes them. Clusters pinned to an older chart still copy it.

Render cache¶

Each Application lists what its render reads in manifest-generate-paths. For compliance-prod-east-1:

argocd.argoproj.io/manifest-generate-paths: '.;../../values.yaml;../../values/environments/prod.yaml;../../values/datacenters/dc1.yaml;../../values/platforms/aws.yaml;../../values/clusters/prod-east-1.yaml'

A new commit re-renders only Applications whose listed paths changed. Fleet file changes alter the spec and always re-render. Without a webhook this needs Argo CD 2.11 (OpenShift GitOps 1.13)+; older versions re-render every Application on every commit.

Injected values¶

Passed in valuesObject, which ranks above every values file (Order). For prod-east-1:

channel: "stable-4.20" # (1)!
selectedName: prod-east-1 # (2)!
selectedId: <clusterID>
selector:
  matchExpressions:
    name: # (3)!
      key: name
      operator: In
      values:
        - "prod-east-1"

1. version and channel, only when the fleet file sets them.
2. hubName on the hub.
3. policy-library turns this into each Placement's selector. On the hub it holds the ManagedCluster name, not hubName.

Nothing else is injected. An element that needs the environment reads a key set in environments/<env>.yaml.

A lifecycle Application passes the fleet file's keys instead, so the fleet file's install overrides the cascade (Values):

hub: "acm-dc1"
state: "present"   # default
version: "4.20.8"  # channel too, when set
install: {}        # default
selector:
  matchExpressions:
    hub: # (1)!
      key: name
      operator: In
      values:
        - "local-cluster"

1. The ManagedCluster of the hub running the lifecycle Policies.

The install chart¶

The install chart installs both ApplicationSets, the GitOpsCluster and RBAC (Install PolicyStack).

Upgrade it as in Upgrading. Preview the render:

helm template policystack ./install --set hubName=acm-dc1 --set gitRepo=https://github.com/PolicyStack/PolicyStack.git