Orderflow Engine and Bindings

Orderflow is a multi-language market data and analytics engine that computes 29 categories of microstructural analytics across Rust, C, Python, and Java — from spread metrics and VPIN toxicity through fingerprint patterns, volatility signatures, Almgren-Chriss impact models, options flow, futures basis, dark pool siphon detection, and machine-learning-ready LOB features.

Version 0.5.0 hardens the execution, OMS, signal, adapter, and persistence foundations for developer-built production workflows. Execution APIs use separate Rust crates, C handles, Python classes, and Java classes, so existing analytics integrations remain stable. The execution layer provides typed order requests, FIX-style state transitions, structured risk rejection, simulated execution, bounded concurrent command workers, journals, recovery helpers, C/Python/Java bindings, a reusable low-allocation FIX codec/session foundation, a transport-injected FIX execution runtime, and a separate parent/child execution-algorithm crate. It is not a broker-certified production OMS by itself; it is the reusable foundation developers use to build one.

What's New In 0.5.0

0.5.0 is a non-breaking production-hardening release over 0.4.0. The established analytics/runtime/C ABI/binding packages move together to 0.5.0; new standalone crate surfaces remain at 0.1.0 on their own stabilization paths.

Package versions for this release:

Package family	Version	Why
Existing Rust crates: of_core, of_adapters, of_signals, of_persist, of_runtime, of_ffi_c	0.5.0	Same established API line, additive production hardening
Python binding: orderflow-gregorian09	0.5.0	Matches the native of_ffi_c package line
Java binding: orderflow-java-binding	0.5.0	Matches the native of_ffi_c package line
New Rust crates: of_analytics, of_fix, of_persist_parquet	0.1.0	New advanced analytics/FIX/cold-storage crate family, intentionally versioned from its own first public release
of_execution_core, of_execution	0.2.0	Additive execution-domain and OMS expansion after their published 0.1.0 releases
of_execution_algos	0.1.0	First publication of the execution-algorithm crate against of_execution_core 0.2.0
of_execution_adapters	0.2.0	Additive FIX execution adapter and certification capabilities

Major additions:

• execution-domain primitives: fixed-size identifiers, typed submit/cancel/amend requests, execution reports, strict order-state machine, and basic risk gates
• execution engine: route/account/symbol routing, simulated adapter, bounded event buffers, route-scoped risk accounting, journaling, metrics, and health
• concurrent OMS worker: bounded command/report queues for many producers while one native worker owns order state deterministically
• OMS helpers: command correlation, event fanout, lifecycle snapshots, durable file/WAL journals with full request-payload crash recovery, checkpoint recovery, native order-intent and parent/child lifecycle, scoped kill switches, production risk, exact position/PnL ledger, command idempotency, execution-report deduplication, reconciliation, safety policies, generalized multi-source recovery reconciliation, a deterministic 18-scenario adapter/OMS certification venue, fixed-memory percentile execution SLIs and explicit SLO evaluation, permission-ready idempotent operator commands with durable intent/outcome audit, bounded production incident bundles with SHA-256 inventory and atomic publication, allocation, telemetry, sharding, throttling, and replay simulation
• FIX foundation: borrowed tag-value codec, BodyLength(9) and CheckSum(10) validation, common tag extraction, typed FixVersion/FixMsgType helpers, static dictionary/profile validation, deterministic session sequence primitives, typed session/admin and order-entry builders, caller-owned encode buffers, and debug rendering outside the hot path
• execution adapters: a transport-injected FIX 4.2/4.4 runtime with bounded gap recovery, durable resend hooks, venue profiles, latency metrics, a deterministic wire-level certification harness, and the original fail-closed compatibility shell
• market-data operations: additive typed active-adapter mode/session status, subscription inventory, queue/loss/freshness/raw-capture diagnostics, and endpoint redaction that never returns userinfo, paths, queries, or fragments
• market-data persistence: additive checksum-linked segmented normalized WAL, explicit seal/sync policies, atomic rebuilt manifests, and cloneable nonblocking producers feeding a single bounded writer with separate record and payload-byte limits; provider-native raw evidence capture adds timestamp provenance, source/session identity, payload flags, and ownership-preserving bounded admission before normalization; runtime, C, Python, and Java hosts can opt into owned writer lifecycle, explicit failure policy, durability barriers, quality-preserving replay, and event-time backlog health
• verified cold storage: opt-in of_persist_parquet exports bounded Arrow batches into date/venue/symbol/stream partitions, preserves source, sequence, timestamp, quality, raw normalized payload, and optional derived snapshots, then reopens and verifies the file before producing retention evidence
• execution algorithms: additive parent/child order primitives, fixed-capacity decision buffers, OMS execution-event progress folding, and deterministic TWAP slice planning in of_execution_algos
• advanced analytics split: new of_analytics crate with dependency-light market-quality/TCA and liquidity/depth primitives plus feature profiles for future impact, toxicity, volatility, regime, pattern, derivatives, institutional, and ML-feature modules
• bindings: additive C ABI, Python, and Java execution APIs, including a deterministic TWAP parent handle with explicit plan/commit/event-folding semantics, plus config-driven built-in signal replay validation with typed event inputs and versioned JSON reports, without changing existing analytics or signal-evaluation APIs
• documentation: end-to-end strategy workflow, OMS architecture, OMS cookbook, low-latency design, provider adapter authoring, recovery operations, and exhaustive binding READMEs

Upgrade rule: existing analytics users can upgrade without renaming existing APIs. Execution adopters using the published of_execution 0.1.x line can upgrade to 0.2.0 additively; provider adapters should update to of_execution_adapters 0.2.0 when using that line.

The full release inventory, compatibility rules, operational constraints, and upgrade sequence are in Release 0.5.0.

Documentation

Start here:

• Strategy Cookbook — 30 exhaustive strategy examples covering every analytics concept across all four API layers (Rust, C, Python, Java), plus a full multi-concept live trading loop, an API compatibility map, and the AnalyticsConfig tuning guide.
• Building an Orderflow Strategy — complete idea-to-execution workflow: ingest, analytics, signal gating, risk, simulated OMS execution, reports, journal, replay, and promotion checklist.
• OMS Cookbook — multi-symbol routes, synchronous and concurrent execution, risk rejection, recovery, reconciliation, throttling, replay, and C/Python/Java worker examples.
• Handbook Home — primer, architecture, API reference.
• docs/README.md — full navigation.
• CONTRIBUTING.md — coding standards, compatibility rules, testing, documentation, security, and review workflow.
• docs/bindings/README.md — Python (ctypes) and Java (JNA) setup.

Analytics at a Glance

Category	Concepts	Exposed
Book and spread analytics	Spread, depth, book events, resiliency	Rust, C, Python, Java
Trade-flow analytics	Trade classification, VPIN, Kyle's lambda, Amihud, CVD	Rust, C, Python, Java
Pattern detection	19 pattern flags across footprint, DOM, delta, session, and volume profile context	Rust, C, Python, Java
Volatility and impact	Parkinson/Garman-Klass/Yang-Zhang volatility, signature plots, noise, Hasbrouck, Almgren-Chriss, spread decomposition, ACD, regime	Rust, C, Python, Java
Advanced market structure	Kinetic energy, agent-type ID, LOB features	Rust, C, Python, Java
Institutional flow	Dark pool, dark-lit correlation, institutional flow	Rust, C, Python, Java
Derivatives analytics	Options flow, open-interest analysis, futures basis, calendar spread, settlement	Rust, C, Python, Java

Execution Core at a Glance

Layer	Additive API
Rust core	of_execution_core order IDs, requests, events, state machine, risk
FIX protocol	of_fix borrowed tag-value codec, profile rules, durable sequence/resend primitives, and transport-independent session engine
Rust engine	of_execution adapter trait, bounded event buffer, simulator, journal/recovery, operator controls, and verifiable incident bundles
FIX execution adapter	of_execution_adapters::fix transport-injected session runtime, venue profile boundary, durable resend hooks, report mapper, and compatibility shell
Execution algos	of_execution_algos parent/child substrate and deterministic TWAP planner
C ABI	of_execution_engine_t, of_execution_twap_algo_t, submit/cancel/amend/poll/state/progress
Python	ExecutionEngine, TwapExecutionAlgo, typed requests/plans/progress
Java	OrderflowExecutionEngine, TwapExecutionAlgo, typed requests/plans/progress

Low-latency-sensitive paths use typed structs and caller-owned event buffers, not JSON payloads. JSON remains for analytics snapshots and diagnostics.

Every analytics type is configurable via the 22-field AnalyticsConfig struct and queryable through the same buffer-negotiation C ABI pattern.

Quick Build

cargo build --all-features
cargo test --all-features

Build C ABI for bindings, then test FFI exports:

cargo build -p of_ffi_c --features tickbar
python3 tools/check_api_manifest.py
python3 tools/generate_binding_signatures.py --check
python3 tools/test_generate_binding_signatures.py
python3 tools/check_binding_parity.py
python3 tools/generate_api_inventory.py --check
tools/check_ffi_exports.sh

Bindings Quickstart

Python

from orderflow import Engine, EngineConfig, Symbol

with Engine(EngineConfig()) as engine:
    engine.start()
    engine.subscribe(Symbol("CME", "ESM6", 10))
    engine.poll_once()

    # Read any of the 29 analytics
    print(engine.analytics_snapshot(Symbol("CME", "ESM6", 10)))
    print(engine.pattern_snapshot(Symbol("CME", "ESM6", 10)))
    print(engine.lob_features(Symbol("CME", "ESM6", 10), 0.0, 0.0, 0.0))

Execution simulation:

from orderflow import (
    ConcurrentExecutionEngine, ExecutionEngine, ExecutionOrderType, ExecutionSide, ExecutionTimeInForce,
    OrderRequest, RiskLimits, RouteConfig,
)

limits = RiskLimits(False, 100, 1_000_000, 10, 10_000_000, 0)
routes = [
    RouteConfig("SIM", "ACC", "SIM", "ES", True, limits),
    RouteConfig("SIM", "ACC", "SIM", "NQ", True, limits),
]

with ExecutionEngine(routes) as execution:
    events = execution.submit_order(OrderRequest(
        "C1", "ACC", "SIM", "STRAT", "SIM", "ES",
        ExecutionSide.BUY, ExecutionOrderType.LIMIT, ExecutionTimeInForce.DAY,
        10, 5000,
    ))
    print(events[-1].order_status)

with ConcurrentExecutionEngine(routes) as execution:
    sequence = execution.submit_order(OrderRequest(
        "C2", "ACC", "SIM", "STRAT", "SIM", "NQ",
        ExecutionSide.BUY, ExecutionOrderType.LIMIT, ExecutionTimeInForce.DAY,
        10, 17000,
    ))
    report = execution.try_recv_report()

Java

try (OrderflowEngine engine = new OrderflowEngine()) {
    engine.start();
    Symbol sym = new Symbol("CME", "ESM6", (short) 10);
    engine.subscribe(sym, StreamKind.ANALYTICS);

    System.out.println(engine.analyticsSnapshot(sym));
    System.out.println(engine.volatilitySnapshot(sym));
}

Execution simulation:

RiskLimits limits = new RiskLimits(false, 100, 1_000_000, 10, 10_000_000, 0);
List<RouteConfig> routes = List.of(
    new RouteConfig("SIM", "ACC", "SIM", "ES", true, limits),
    new RouteConfig("SIM", "ACC", "SIM", "NQ", true, limits)
);

try (OrderflowExecutionEngine execution = new OrderflowExecutionEngine(null, routes)) {
    execution.start();
    execution.submitOrder(new OrderRequest(
        "C1", "ACC", "SIM", "STRAT", "SIM", "ES",
        ExecutionSide.BUY, ExecutionOrderType.LIMIT, ExecutionTimeInForce.DAY,
        10, 5000, 0, 1, 2
    ));
}

try (ConcurrentOrderflowExecutionEngine execution =
         new ConcurrentOrderflowExecutionEngine(null, routes)) {
    long sequence = execution.submitOrder(new OrderRequest(
        "C2", "ACC", "SIM", "STRAT", "SIM", "NQ",
        ExecutionSide.BUY, ExecutionOrderType.LIMIT, ExecutionTimeInForce.DAY,
        10, 17000, 0, 1, 3
    ));
    Optional<ExecutionCommandReport> report = execution.tryRecvReport();
}

Tooling

Replay utility:

cargo run -p replay_cli -- data              # list venues
cargo run -p replay_cli -- data CME          # list symbols
cargo run -p replay_cli -- data CME ESM6 100 200  # replay range