Hebbian Robotics Launches HFlow SDK for Scalable Robotics Data Pipelines

The founders said robotics data pipelines often start as one-off scripts for video transcoding, timestamp checks, labeling, and copying recordings. As corpora grow, teams lose track of which code ran, why an episode was excluded, or whether a dataset can be reproduced. They cited quality control as the first pain point, since frozen cameras, missing topics, timestamp drift, and duplicate recordings can silently enter training data.

HFlow accepts one MCAP file per episode. MCAP is an open container format from Foxglove that keeps synchronized video, robot state, actions, and other sensor streams in a single file. The SDK exposes transformations, checks, labels, and enrichments as plain Python functions that receive an episode and return measurements, artifacts, or transformed data. During development these functions run in-process; for scheduled corpus processing, HFlow packages the same registered steps as Airflow 3 DAGs.

The output is a canonical MCAP with in-band H.264 video, grouped camera and state chunks, and provenance describing how the output was produced. Each step has an explicit behavior version. HFlow writes measurements, metadata, version stamps, and artifact locations to an append-only Parquet catalog, which teams can query with DuckDB SQL and use to produce a version-pinned manifest without reopening recordings. Critical checks can quarantine an episode, but HFlow does not delete data.

The founders described three types of teams that would use HFlow: data vendors or marketplaces collecting egocentric recordings, robotics teams collecting teleoperated demonstrations for their own models, and teams operating robots in the field. For field operations, HFlow could process incoming logs, quarantine incomplete or corrupted episodes, and query the catalog for specific robot versions, environments, or failure conditions.

HFlow is pre-v1 but the core lifecycle works end to end. It is free under the Apache-2.0 license. Users can try it by cloning the repository without an account, Docker, or robot hardware. The open-source deployment is currently a single-tenant workspace; the company has not yet built a hosted, multi-tenant control plane.