Skip to main content

Quickstart: the demo

Raw firewall syslog in, one parsed ClickHouse table per vendor out, on one host. The fetcher pulls two live API feeds beside it: Wikipedia's recent changes, and the latest GitHub release of each public DFE component.

You need Docker Engine 26.0 or later with Compose 2.34.0 or later.

docker compose -f oci://ghcr.io/hyperi-io/dfe-docker:demo up -d

A throwaway stack that seeds its own sources and streams vendor syslog into one table per vendor, beside its fetchers. Point your browser to http://localhost:3000 and log in as admin with password dfe-first-login. The console makes you change it at first login. It takes the Docker install's ports, so stop that first, and docker compose -f oci://ghcr.io/hyperi-io/dfe-docker:demo down -v removes it all.

From a checkout​

To change the demo's rate or protocol, or to keep the stack afterwards, run it from a dfe-docker checkout. That needs Python 3, git and make as well as Docker.

git clone https://github.com/hyperi-io/dfe-docker && cd dfe-docker
make demo                       # newest release, 20 lines a second over TCP
make demo DEMO_RATE=200         # a faster stream
make demo DEMO_PROTOCOL=udp     # syslog over UDP

When no stack is running, make demo starts one and runs its self test, which prints PASS per check. Then it replays Cisco ASA and FortiGate logs as raw syslog. dfe-transform-elastic works out which device sent each line, parses it, and lands it in dfe.cisco-asa or dfe.fortinet. A line no vendor claims goes to dfe.demo-syslog.

It returns once every vendor table has gained rows, and the stream keeps going. Each vendor has about 6,000 sample lines, so it runs roughly ten minutes before a line repeats. On a stack already running the demo, run make demo-stop before make up.

When you're done:

make demo-stop    # stop the stream
make down         # stop the stack too, keep the data
make clean        # stop everything and DELETE the volumes, ClickHouse included

Next: