Warmup and NaN — the silent graph killers

You built a graph. You fed it 50 records. It produced 30 outputs. The other 20 are gone. Where did they go?

Warmup and NaN are the two most common answers — and they’re both silent by default. No error, no warning, just fewer outputs than you expect. Let’s walk through both, then give you a debugging checklist so you never chase this ghost again.

Warmup: the mandatory apprenticeship

Every operator in tflo needs a minimum number of records before it can produce a valid result. An SMA(20) needs 20 records — you can’t average what you don’t have. An RSI(14) needs 14. A deviation band needs enough to compute both SMA and standard deviation.

When you compile a graph, min_warmup is set to the maximum warmup requirement across all operators:

// From CompiledGraph source:
pub min_warmup: usize,
// Set during compile to max of all operator requirements

During warmup, step() silences everything:

pub fn step(&mut self, record: &R) -> Option<PipelineItem<C, O>>

Returns None until warmup finishes. That’s your missing 20 records.

If you want visibility into what’s happening during warmup, switch to step_with_status():

pub fn step_with_status(&mut self, record: &R) -> StepResult<C, O>

The StepResult enum tells you exactly where you are:

pub enum StepResult<C: PipelineContext, O> {
    Ready(PipelineItem<C, O>),
    WarmingUp {
        remaining: usize,
        reason: Absent,
    },
    Error(ComputeError),
}

WarmingUp { remaining, .. } — remaining is how many more records you need before this operator starts producing. Print it, log it, monitor it. The reason field carries the typed [Absent] cause: once remaining hits zero the graph is warmed up, and a still-empty output means something else (a filtered value, a divide-by-zero) — reason tells you which.

You can also inspect the graph’s warmup state without stepping:

let plan: GraphPlan = graph.graph_plan();
// plan.node_count, plan.min_warmup, plan.warmup_remaining, plan.records_seen

let summary: GraphStateSummary = graph.state_summary();
// summary.is_warmed_up, summary.records_seen, summary.min_warmup
// summary.warmup_remaining, summary.node_count, summary.output_count

state_summary() is the introspection tool. Call it any time to see where your graph stands.

NaN: one bad number poisons everything

IEEE 754 says NaN + anything = NaN. That’s not a bug — it’s the standard. But in a streaming graph, it means one bad reading propagates through your entire pipeline.

Here’s the cascade:

1. A value arrives as f64::NAN — maybe a corrupt sensor feed, maybe a division by zero upstream.
2. Your SMA node processes it: sum += NaN → SMA is now NaN.
3. Your RSI node takes SMA as input: RSI is now NaN.
4. Your cross detector compares two NaN values: no signal, no output, no alert.

Nothing crashed. Nothing logged an error. You just lost an unknown number of downstream results.

ValidationOptions is how you stop this:

// Permissive — accepts everything (default)
let options = ValidationOptions::new();

// Strict — rejects NaN, inf, unsorted timestamps, negative values
let options = ValidationOptions::strict();

strict() enables all checks:

pub const fn strict() -> Self {
    Self {
        assert_sorted: true,
        min_warmup: 1,
        reject_nan: true,
        reject_inf: true,
        error_on_nan: true,
        error_on_inf: true,
        error_on_negative: true,
        max_gap_ms: None,
    }
}

validated() enforces every option: reject_nan / reject_inf filter the offending value out of the stream, error_on_nan / error_on_inf / error_on_negative surface a typed Err(TFloError), assert_sorted and max_gap_ms police timestamps, and min_warmup suppresses early output.

Apply validation at the stream level (sortedness check) or as a pre-filter:

use tflo_core::prelude::*;
use tflo_core::validation::ValidationOptions;
use tflo_ops::prelude::*;

// Validate at the stream level
let results: Vec<TFloResult<f64>> = data
    .into_iter()
    .validated(ValidationOptions::strict(), |t| {
        t.timestamp(|x| x.ts);
        let value = t.prop(|x| x.value);
        value.sma(20_u64.secs())
    })
    .collect();

Debugging checklist

Found yourself staring at a graph that’s producing fewer outputs than it should? Run through this:

Step 1 — Check warmup. Call graph.state_summary(). If is_warmed_up is false, you know what’s happening. warmup_remaining tells you how many more records you need.

Step 2 — Check for NaN. Scan your input data. If any value is f64::NAN or f64::INFINITY, that’s likely the source. One NaN entering the graph can silence every downstream node.

Step 3 — Validate the stream. Wrap your stream with validated(ValidationOptions::strict(), ...) — it catches out-of-order timestamps, oversized timestamp gaps, and NaN/infinite/negative values, surfacing each as a typed Err(TFloError). (The standalone require_not_nan / require_finite helpers from tflo_core::validation are still available if you prefer to pre-filter.)

Step 4 — Inspect with state_summary(). Call graph.state_summary() at any point. You get records_seen, min_warmup, warmup_remaining, is_warmed_up, node_count, and output_count. It’s the fastest way to understand what your graph thinks is happening.

Most “broken graph” problems are one of these two things. Now you know how to catch both.

Source code

The full working example is available on GitHub: tflo-examples/examples/warmup-and-nan

— Matt