Data plane
The close-coupled source - ingest - schema - transform path. One source definition carries routing, schema binding, and field-map bindings together; the receiver, loader, views, hunts, and query API all read the same definition, so there is no separate ETL config to drift. The system map is ../architecture.md.
Documents in this area
| Doc | Covers |
|---|---|
| source.md | Source as the top-level data abstraction |
| source-definition.md | the source YAML field by field, and what a write is validated against |
| source-registry.md | states, storage backends, CRUD, topics on deploy, the API surface |
| source-flow.md | INPUT, optional TRANSFORM, OUTPUT on the Kafka or gRPC transport; what compiles from a source; how a change reaches a pod |
| source-routing.md | the exact receiver and loader config a source compiles into |
| topic-contract.md | the Kafka topics a source implies: ensure, converge, status, remove, and the dials |
| external-pipelines.md | the customer's own Vector, Fluentd or Logstash as a source's origin or transform on Kafka |
| source-detections.md | rules, hunts and Sigma views on a source |
| schema.md | schema type system: primitives, attributes, use cases, DDL |
| schema-classes.md | engine class reference for the schema system (SchemaLoader, SchemaBuilderV2, DDL generation) |
| schema-sync.md | engine<->loader runtime schema contract |
| field-mapping.md | standards translation layer (Sigma / ECS / CIM views) |
| sigma.md | Sigma rule-provider pipeline (feeds -> catalogue -> propagation -> source-views) + query cost leaderboard |
| expressions-cel.md | CEL profile, classifier tiers, SQL transpile |
| query-api.md | query API surface, and the in-image Python client |
| hunt-runner-scaling.md | pull-based runner, deterministic-due scaling, keda-shim fail-safe |
| hunt-match-delivery.md | where a hunt match is written, who sees it, alert destinations and what the runner sends |
| hunt-schedule-smoothing.md | phase-drift load smoothing (incl. the reference derivation) |