tflo
validation nan production safety

Taming NaN — tflo's validation system

NaN is silent. It doesn't crash your program. It just makes your outputs meaningless. Here's how tflo catches it before it poisons your pipeline.

7 min read

NaN is silent. It doesn’t crash your program. It just makes your outputs meaningless.

IEEE 754 floating point propagates NaN through every operation. One bad input — a division by zero, a missing sensor reading, a log of a negative number — and every downstream computation becomes NaN too. Your crossover detector fires Nothing. Your SMA vanishes. Your alerts go quiet.

The worst part: nothing tells you. The graph keeps running, producing NaN after NaN, and you don’t notice until someone asks why no signals fired today.

tflo’s validation system gives you explicit control over this — decide what to catch, and catch it.

Permissive by default

Out of the box, ValidationOptions::new() is permissive. It allows NaN, allows infinity, doesn’t check timestamps. This matches IEEE 754 semantics — NaN propagates naturally, and your code runs without overhead.

use tflo_core::validation::ValidationOptions;

let options = ValidationOptions::new();
// assert_sorted: false
// reject_nan: false
// reject_inf: false
// error_on_nan: false
// error_on_inf: false
// error_on_negative: false
// min_warmup: 0
// max_gap_ms: None

This is fine for development. You’re testing against clean data, you know your inputs are valid, and you don’t want the overhead of checks. But for production, you probably want more.

Strict mode

One call flips all the switches:

let options = ValidationOptions::strict();
// assert_sorted: true
// reject_nan: true
// reject_inf: true
// error_on_nan: true
// error_on_inf: true
// error_on_negative: true
// min_warmup: 1
// max_gap_ms: None

The validated() method on iterators enforces every option:

use tflo_core::prelude::*;
use tflo_core::validation::ValidationOptions;
use tflo_ops::prelude::*;

let results: Vec<TFloResult<f64>> = events.into_iter()
    .validated(ValidationOptions::strict(), |t| {
        t.timestamp(|x| x.ts);
        let value = t.prop(|x| x.value);
        value.sma(20_u64.secs())
    })
    .collect();

Note the return type: TFloResult<f64> (an alias for Result<f64, TFloError>) instead of f64. Each result is either Ok(f64) or Err(TFloError). Which checks fire depends on which options you enabled — out-of-order timestamps surface as Err(TFloError::OutOfOrderTimestamp), a NaN with error_on_nan set surfaces as Err(TFloError::NaN), and a reject_* option simply drops the offending value from the stream.

Sanitizing input with standalone validators

Use the standalone validation functions from tflo_core::validation as a separate step before feeding data into .tflo():

use tflo_core::validation::{require_finite, require_not_nan};
use tflo_core::prelude::*;
use tflo_ops::prelude::*;

let results: Vec<TFloResult<f64>> = events.into_iter()
    .filter_map(|event| {
        // Validate inputs before they reach the graph
        require_not_nan(event.value).ok()?;
        require_finite(event.value).ok()?;
        Some(event)
    })
    .tflo(|t| {
        t.timestamp(|x| x.ts);
        let value = t.prop(|x| x.value);
        value.sma(20_u64.secs())
    })
    .collect();

This gives you explicit control: bad values are filtered out before the pipeline sees them, and the graph only processes clean data.

Builder pattern for fine control

You don’t have to go all-or-nothing. ValidationOptions uses a builder pattern so you can pick exactly which checks to enable:

let options = ValidationOptions::new()
    .assert_sorted(true)          // Error on out-of-order timestamps
    .reject_nan(true)             // Filter NaN values out of the output
    .min_warmup(10);              // Suppress output for the first 10 records

let options = ValidationOptions::new()
    .error_on_nan(true)           // Error (rather than filter) on NaN
    .error_on_inf(true)           // Error on infinity
    .max_gap_ms(30_000);          // Error if consecutive timestamps are >30s apart

The individual methods — assert_sorted(), reject_nan(), error_on_nan(), reject_inf(), error_on_inf(), error_on_negative(), min_warmup(), max_gap_ms() — are all const fn, so you can build the options at compile time with zero runtime cost.

What each flag does

  • reject_nan — drops records whose output is NaN; they simply don’t appear in the result stream.
  • error_on_nan — returns Err(TFloError::NaN) on a NaN output (stronger than reject_nan; the error wins if both are set).
  • reject_inf / error_on_inf — the same filter-or-error logic for infinity.
  • error_on_negative — returns Err(TFloError::NegativeValue) on a negative output.
  • assert_sorted — checks that timestamps are monotonically non-decreasing; violations return Err(TFloError::OutOfOrderTimestamp).
  • min_warmup — suppresses output until the graph has seen this many records.
  • max_gap_ms — returns Err(TFloError::TimestampGapExceeded) when two consecutive timestamps are further apart than this threshold.

Every one of these is enforced by validated().

The error types

On the validated iterator, errors are wrapped in TFloError. The variants validated() raises:

pub enum TFloError {
    OutOfOrderTimestamp { previous, current },
    TimestampGapExceeded { previous, current, max_gap },
    NaN,
    Infinite,
    NegativeValue { reason },
    // ...
}

Which variant you get tells you which option tripped: assert_sorted → OutOfOrderTimestamp, max_gap_ms → TimestampGapExceeded, error_on_nan → NaN, error_on_inf → Infinite, error_on_negative → NegativeValue.

When to use each mode

  • Development and testing: ValidationOptions::new() — permissive, fast, no noise.
  • CI and pre-production: ValidationOptions::strict() — validated() enforces every check and surfaces each failure as a typed TFloError.
  • Production with monitoring: pick exactly the checks you want with the builder — e.g. assert_sorted(true) plus reject_nan(true) to drop bad values while erroring on ordering issues.
  • Legacy data sources: reject_nan(true).reject_inf(true) filters bad values out of the stream; the standalone require_* helpers are still available if you prefer to sanitize before the pipeline.

The point isn’t to always use strict mode. The point is to choose. Silence is the enemy of debuggable pipelines — decide what you want to catch, and catch it explicitly.

Source code

The full working example is available on GitHub: tflo-examples/examples/taming-nan

— Matt