# The DFE Control-Plane Layer

> The **control-plane layer** is the small set of repos that own the DFE control
> plane - configuration, RBAC, schema, the operator UI, and the deploy target.
> When a doc says "the control-plane layer" it means exactly these repos. Anything
> that interacts directly, or one level removed, with the dfe-engine API or what
> it does is reasoning about this layer.

## The layer

```mermaid
flowchart TB
    subgraph CP["Control-plane layer"]
        engine["dfe-engine<br/>config control plane + sole authz PDP<br/>the FastAPI API + the dfe CLI"]
        schemas["dfe-schemas<br/>schema SSoT (event header, DDL, profiles)"]
        ui["dfe-ui<br/>operator console (window over the API)"]
        infra["dfe-infra<br/>GitOps SSoT + deploy vehicle (Argo CD)"]
        deploy[("dfe-deploy<br/>per-deployment RW gitops repo<br/>we own only the INITIAL state")]
    end

    ui -->|HTTP, engine-minted JWT| engine
    engine -->|reads| schemas
    engine -->|commits config| deploy
    infra -->|deploys + syncs| deploy

    classDef cp fill:#cfe2ff,stroke:#084298;
    classDef ext fill:#e2e3e5,stroke:#41464b;
    class engine,schemas,ui,infra cp;
    class deploy ext;
```

| Repo | Role in the layer |
|---|---|
| **dfe-engine** | The config control plane + the sole authorisation PDP. Hosts the FastAPI API and the generated `dfe` CLI. Everything else in the layer keys off its API. |
| **dfe-schemas** | The schema SSoT (common event header, hunt-results DDL, profiles). The engine reads it (the version-pinned `dfe-schemas` wheel) and auto-creates from it. |
| **dfe-ui** | The operator console. A window over the engine API - it holds no authority of its own; it verifies the engine-minted JWT. |
| **dfe-infra** | The GitOps SSoT and deploy vehicle (Argo CD). Deploys the suite; never a config authority. |
| **dfe-deploy** | The per-deployment read-write gitops repo the running engine commits to. We own only its **initial** state; after handover the deployment owns it. |

## The controlling SSoTs

The layer is organised around a handful of **single sources of truth**. The SSoTs
are how everything works: each holds one kind of authoritative state, every other
component reads from it (or from a generated artefact of it), and each has exactly
ONE write path - so a change is auditable and consumers can never diverge.

| SSoT | Lives in | Consumed by | Only written by |
|---|---|---|---|
| **API surface** - `openapi.json` | dfe-engine `openapi/` | dfe CLI, dfe-ui types, contract mock, reference docs | the FastAPI routes via `scripts/generate_openapi.py` (never hand-edited) |
| **Schema** - event header, hunt-results DDL, profiles | dfe-schemas (the `dfe-schemas` wheel, pinned in `uv.lock`) | the engine (auto-creates the CH objects at startup), the CLI, dfe-ui | dfe-schemas commits |
| **Deployment shape** - Helm charts + values | each app's thin chart, built from its deployment contract, plus dfe-infra values + the deploy-repo overlays | Argo CD, the cluster | the thin charts by each app's release, the values by dfe-infra, the overlays by the engine, one `values/<service>-<instance>-values.yaml` per app instance, each write a gitops commit. Where no chart runs (Compose), `appmgmt/appconfig.py` merges an overlay's `config:` over the deployment's base config into the app's config file |
| **Deployed state** - the gitops YAML | dfe-deploy (per deployment) | Argo CD (syncs to the cluster), the engine (reads current state) | the engine API - every mutation is a git commit (governed ops); `dfe local` break-glass when the daemon is down |
| **AuthZ** - RBAC roles + scopes | dfe-engine `auth/` (YAML) | the API (`require_action`), the CLI | the engine (governed writes) |

```mermaid
flowchart LR
    subgraph SSoT["The SSoTs"]
        api[("openapi.json")]
        schema[("dfe-schemas")]
        helm[("Helm charts + values")]
        state[("gitops state")]
        rbac[("RBAC roles + scopes")]
    end

    routes["API routes"] -->|generate| api
    api --> consumers["dfe CLI / dfe-ui / docs"]
    schema --> engine["dfe-engine<br/>(auto-creates CH objects)"]
    engine -->|compiles + commits| helm
    engine -->|every mutation = a commit| state
    helm --> argo["Argo CD -> cluster"]
    state --> argo
    rbac --> engine

    classDef ssot fill:#fff3cd,stroke:#664d03,stroke-width:2px;
    class api,schema,helm,state,rbac ssot;
```

To change any of them you go through its one writer: edit the API routes and
regenerate the spec, commit to dfe-schemas, turn the engine's dials (which writes
Helm values + gitops state as commits), or edit the RBAC YAML. Never hand-edit a
generated artefact (openapi.json, compiled Helm values) - change the source and
let it regenerate. The next section walks the `openapi.json` case in full as the
worked example.

## dfe-engine, never "dfe-api"

The API is **the dfe-engine API**. `dfe-api` was the old server entry-point name
and is retired: the daemon binary is now `dfe-engine` (`dfe-engine run`), the
human CLI is `dfe`. Anything talking to the API is talking to the dfe-engine API --
do not call it "the dfe-api". Its own settings read as `DFE_ENGINE_API_*`.

## openapi.json is the API SSoT

`openapi/openapi.json` is the **single source of truth for the API surface**.
It is generated from the FastAPI app (`uv run python scripts/generate_openapi.py`)
and committed. Every consumer keys off it, never off a hand-maintained copy:

The spec is the hub; everything radiates out of it. The FastAPI routes feed it,
and every downstream consumer keys off it - never off a hand-kept copy:

```mermaid
flowchart TB
    routes["FastAPI routes<br/>(the live API definition)"]
    routes -->|"scripts/generate_openapi.py"| spec[("openapi.json<br/>THE API SSoT")]

    spec -->|"command tree at runtime<br/>(x-scalo.cli.hidden opts out)"| cli["dfe CLI"]
    spec -->|"openapi-typescript<br/>-> @dfe/dfe-engine-types"| ui["dfe-ui"]
    spec -->|"Prism"| mock["contract mock (dev + CI)"]
    spec -->|"Swagger UI"| refdocs["API reference docs"]

    classDef ssot fill:#fff3cd,stroke:#664d03,stroke-width:3px;
    classDef src fill:#e2e3e5,stroke:#41464b;
    class spec ssot;
    class routes src;
```

- The **`dfe` CLI** generates its whole command tree from the spec at runtime -
  add an endpoint and the CLI grows a command, with no CLI code. An endpoint
  opts out via `x-scalo.cli.hidden`.
- **dfe-ui** generates its TypeScript types from the same spec.

So when the API changes: change the routes, regenerate `openapi.json`, and every
consumer follows. Never edit `openapi.json` by hand or let a consumer drift from
it.

## What is NOT in this layer

- The data-plane repos (receivers, loaders, transforms, fetchers, archiver,
  hunt-runner) are downstream services the control plane configures - not part
  of it.

## Documents in this area

| Doc | Covers |
|---|---|
| [governed-ops-design.md](https://github.com/hyperi-io/dfe-engine/blob/v1.22.90/docs/control-plane/governed-ops-design.md) | GitCrud + governance layer + curated actions |
| [gitops-commit-standard.md](https://github.com/hyperi-io/dfe-engine/blob/v1.22.90/docs/control-plane/gitops-commit-standard.md) | machine-commit conventions for gitops repos |
| [oidc-rbac-architecture.md](/v/2.2.2/dfe-engine/control-plane/oidc-rbac-architecture.md) | OIDC + RBAC SSoT - auth model, the two RBAC planes, provider registry + adapters, management API, infra requirements |
| [rbac.md](/v/2.2.2/dfe-engine/control-plane/rbac.md) | auth paths, RBAC model, multi-tenant ClickHouse, stores |
| [rbac-sign-in.md](/v/2.2.2/dfe-engine/control-plane/rbac-sign-in.md) | identity resolution, the sign-in allowlist, first login and password changes |
| [rbac-groups.md](/v/2.2.2/dfe-engine/control-plane/rbac-groups.md) | IdP group sync, linking a group to an IdP group, the checks on a link |
| [rbac-engine-reads.md](/v/2.2.2/dfe-engine/control-plane/rbac-engine-reads.md) | the engine's own ClickHouse reads and how they bind to a caller's orgs |
| [idp-setup/](/v/2.2.2/dfe-engine/control-plane/idp-setup.md) -- [google.md](/v/2.2.2/dfe-engine/control-plane/idp-setup/google.md), [entra.md](/v/2.2.2/dfe-engine/control-plane/idp-setup/entra.md), [okta.md](/v/2.2.2/dfe-engine/control-plane/idp-setup/okta.md) | one-time setup for a customer's IdP admin -- sign-in client, groups, the values to send the operator |
| [rbac-vocabulary.md](/v/2.2.2/dfe-engine/control-plane/rbac-vocabulary.md) | SSoT for role + group names and terms across engine, ui, infra, deploy, schemas and test envs |
| [synthetic-data.md](/v/2.2.2/dfe-engine/control-plane/synthetic-data.md) | synthetic data generation - reference packs, lookalike scrub, standing demo streams |
| [ui-api-guide.md](https://github.com/hyperi-io/dfe-engine/blob/v1.22.90/docs/control-plane/ui-api-guide.md) | frontend integration guide (endpoint reference) |
| [type-safety-sync.md](https://github.com/hyperi-io/dfe-engine/blob/v1.22.90/docs/control-plane/type-safety-sync.md) | OpenAPI + RBAC-scope typegen sync to dfe-ui |
| [dfe-cli-design.md](https://github.com/hyperi-io/dfe-engine/blob/v1.22.90/docs/control-plane/dfe-cli-design.md) | the `dfe` client CLI design (AWS CLI v2 model) |

The system map is [../architecture.md](/v/2.2.2/dfe-engine/architecture.md).