Crate Architecture
How the workspace crates compose into a temporal event-processing platform — a generic engine plus operator catalogs, scripting, persistence, connectors, and a browser binding. tflo is experimental and pre-1.0 — not yet published to crates.io.
Dependency layers
tflo-core is contract-only above the engine: connectors, state stores, and
scripting integrations live in separate crates so the engine stays free of direct deps
on aws-sdk-s3, rdkafka, rumqttc, or
reqwest. Operators were extracted from the engine into
tflo-ops (and the finance-specific subset into tflo-fintech)
so that domain catalogs evolve on their own cadence.
┌──────────────────────────────────────────────────────────────────┐
│ Connectors & sinks │
│ tflo-connect-kafka tflo-connect-mqtt │
│ tflo-sink-influx tflo-arrow (Parquet / Polars) │
├──────────────────────────────────────────────────────────────────┤
│ Scripting & policy filters │
│ tflo-cel tflo-rhai tflo-rego │
├──────────────────────────────────────────────────────────────────┤
│ Pattern matching (event sequences) │
│ tflo-cep (closure-based, WASM-clean) │
├──────────────────────────────────────────────────────────────────┤
│ Operator catalogs │
│ tflo-ops (generic catalog) tflo-fintech (finance plugin) │
├──────────────────────────────────────────────────────────────────┤
│ Engine │
│ tflo-core │
│ graph + windowing + keyed execution │
│ timers + dedup + metrics + async contracts │
│ typed Absent + Checkpointer + ShardRouter │
├──────────────────────────────────────────────────────────────────┤
│ Persistence │
│ tflo-state-files tflo-state-s3 │
├──────────────────────────────────────────────────────────────────┤
│ Browser │
│ tflo-wasm (ops/fintech) tflo-cep-wasm (patterns) │
└──────────────────────────────────────────────────────────────────┘ Engine
tflo-core
The temporal event-processing engine. Provides the .tflo() /
.with() / .tflo_keyed() iterator adapters, property
extraction, windowing primitives, the compiled graph runtime, the typed
Computed = Result<f64, Absent> semantics, the
Operator trait, timers, the Deduplicator, the
Metrics trait, the async state / checkpoint / shard-router contracts,
and Builder::fingerprint(). Async support via
--features async.
cargo add tflo-core
Operator catalogs
tflo-ops
The generic operator catalog. Owns every windowed aggregation (SMA, EMA, STD, VAR,
WMA, median, quantile, skewness, kurtosis, correlation, covariance), the stateful
trackers (prev, lag, cumulative_*, pct_change, zscore, rate_of_change, momentum,
peak_decline), and the event detectors (cross, glitch, runt, pulse_width,
window_detector, zone). Extracted from tflo-core in the hardening
pass so domain catalogs can evolve independently.
cargo add tflo-ops
tflo-fintech
Financial technical-analysis indicators (MACD, ADX, ATR, KAMA, Stochastic, CCI,
OBV, VWAP, Bollinger, Ichimoku) as a tflo-ops plugin. Bit-exact
golden-fixture suite validated against TA-Lib. Finance is one domain
catalog among many.
cargo add tflo-fintech
Pattern matching
tflo-cep
Closure-based event-pattern matching — "A then B within T", "A then not B
within T", bounded sequences. Six-method builder
(when / then / not_then /
within / emit plus timestamp),
Match<E> with first / last /
at(name) / named() / all(). The shared
engine::Runtime is also reused by
tflo-cep-wasm. WASM-clean by construction.
cargo add tflo-cep
tflo-cep-wasm
WebAssembly bindings for tflo-cep. Wraps engine::Runtime
with JS-callback adapters (Rc-based, since wasm32 is single-threaded). ~41 KB
optimized .wasm. Consumed by the
@tflo/browser-events
TypeScript SDK — see the
browser analytics
page for the end-to-end flow.
Persistence
tflo-state-files
File-based checkpoint store. Implements both the sync StateStore and
(under the async feature) the AsyncStateStore trait via
spawn_blocking — same backend, both interfaces.
tflo-state-s3
S3-compatible checkpoint store. Rewritten in the hardening pass as a direct
AsyncStateStore impl over a pluggable S3Client trait
(drop in aws-sdk-s3, MinIO, or a mock). Coordinates with
Checkpointer for crash-safe (snapshot → cursor) ordering.
Connectors & sinks
tflo-connect-kafka
Kafka integration. KafkaShardRouter requires an
AsyncStateStore at construction (compile-time poka-yoke against
sharded execution without durable state). Optional rdkafka-backend
feature wraps the production client. AssignmentEpoch fences events
arriving during rebalance.
tflo-connect-mqtt
MQTT adapter for the edge. MqttCursor enforces a bounded QoS-2 dedup
window (64 KiB cap) so the in-flight set cannot grow unbounded. Optional
rumqttc-backend — pure Rust, cross-compiles cleanly to ARM.
tflo-sink-influx
InfluxDB write sink. LineProtocol builder plus a
Batcher with soft flush and a hard max_buffer_bytes
cap. Pluggable InfluxHttpClient trait so HTTP client choice is the
user's.
tflo-arrow
Arrow / Parquet / Polars interop. schema_fingerprint() stamps a hash
into snapshot metadata so backfills against a structurally-different schema are
rejected on restore. parquet_io::{read,write}_batches helpers cover
replay sources and result sinks.
Scripting & policy filters
tflo-cel
Common Expression Language filter. Implement IntoCelContext for your
domain type and call .cel_filter("expr") on any iterator — no
graph step cycle required.
tflo-rhai
Rhai scripting integration. Implement IntoRhaiScope and call
.rhai_filter("expr") — or load full scripts at runtime without
recompiling.
tflo-rego
OPA / Rego policy engine integration. Load policies, then call
.rego_filter(engine, "data.<pkg>.allow") for compliance and
governance use cases.
Tooling & examples
tflo-wasm
WebAssembly bindings for the operator catalogs (tflo-ops and
tflo-fintech). Run streaming detectors and TA indicators live in
a dashboard, extension, or service worker. For browser event-pattern
matching, see the sibling tflo-cep-wasm crate above. See the
WebAssembly page.
tflo-examples
Runnable examples backing the docs and blog. The iot-portal example
stitches MQTT, Kafka, Influx, and Parquet into the end-to-end
reference deployment.
Adding crates to your project
Pick the crates you need — you don't have to pull in everything:
# Minimal: engine + operator catalog
cargo add tflo-core tflo-ops
# Add the financial technical-analysis plugin
cargo add tflo-core tflo-ops tflo-fintech
# Engine plus async stream support (Tokio)
cargo add tflo-core --features async