Benchmarks

How fast is Ibex?

Three benchmarks: Ibex's own in-memory DataFrame suite, the independent h2o db-benchmark and PDS-H, Polars' TPC-H-derived benchmark. H2O and PDS-H pin every engine to the same cores; the in-memory suite uses separate identical instances with default engine thread budgets. Each section names the machine and the commit it was measured on. Lower is better.

Ratios are Ibex time divided by the other engine's time; below 1 means Ibex is faster. Geometric means weight questions equally in ratio space; totals compare the sum of query times and give longer queries more weight. Both summaries compress individual wins and losses. See the per-query tables and the run details and reproduction commands.

In-memory DataFrame queries

Ibex against Polars 2.0.0 and DuckDB 2.0.0 on the same columnar queries, from 1M to 50M rows. The table shows six execution variants measured in this run; blank cells mean that a variant was not measured for that query and size. Times are means of five iterations.

Published run: October 24, 2026, Ibex 950fafc1; separate identical r7i.2xlarge instances per engine, 64 GiB, 8 vCPUs on 4 physical cores with SMT on. Engines use their default thread budgets; the -st columns use one thread. Polars and DuckDB are both version 2.0.0. One warmup and five timed iterations. Reproduce this configuration.

Pick a row count and a metric. Each row is colour-scaled green→red against the best engine in that row — fastest for time, lightest for memory — and the best cell is outlined. Hover any cell for the exact time and peak memory.

What this measures. Each engine loads the data once into its own in-memory format, untimed, and every query is timed from there until its full result is materialised. DuckDB hands its result over as an Arrow table: measured against writing it into a DuckDB table, that is the faster of the two overall. Ibex's CSV reader dictionary-encodes string columns automatically, so a group-by on symbol runs on integer codes. Polars is given the same encoding (its string columns are cast to Categorical); DuckDB groups on plain strings by default. Tables are read once and queried many times, so that is a real advantage.

Answer checks. Deterministic queries are compared by complete output on small fixtures before timing. Strict answer digests check every measured query and size. This run had zero mismatches; checks run outside the timed sections.

Across the full matrix. Ratios are Ibex time divided by the named engine's time; below 1 means Ibex is faster. Geometric means weight each shared query/size cell equally; total ratios compare summed times.

Rows
Show
Columns
All columns

Threads. Every engine gets the same box: 8 vCPU, which is 4 physical cores with 2 hardware threads each — EC2 counts hyperthreads as vCPUs, so these are 4-core numbers, not 8. Ibex parallelises some operators by default, so ibex is not a single-core number; ibex-st is the same build run with IBEX_CORES=1, which makes ibex vs ibex-st Ibex's own threading gain. polars-st and duckdb-st are the equivalent single-thread runs, so ibex-st against those is the per-core comparison.

Memory. The memory metrics show absolute peak resident set size (RSS) during a query's timed iterations — the footprint to run the op, including the already-resident input table — measured per process from the kernel's high-water mark. Multi-threaded engines trade memory for the parallel speed-ups noted above.

Scope. This run includes Ibex, Polars and DuckDB only. The TimeFrame constructor and two scalar builtins without matching reference cases are Ibex-only and excluded from paired summaries. Every displayed cell comes from this run; no older result is carried forward. Each engine materialises its full result. tf rolling EWMA uses the bounded window definition in Ibex, Polars and DuckDB. Variants absent from this AWS fleet, including other engines, do not appear in this comparison.

The code behind these numbers. Every query's exact code, in every engine, is on the Methodology & code page. Directly extracted from the harness source.

Independent Python API. The mean-by-symbol-and-day query translates directly to import ibex as pl. See the example and fresh in-memory measurements, including prepared categorical keys for both engines and first-use costs. These development measurements are separate from the historical table above.

h2o db-benchmark

The h2oai db-benchmark questions, as maintained by DuckDB Labs (duckdblabs/db-benchmark, pinned at 3b074bc319): ten group-by questions on 10M rows at three key cardinalities (K=100, 10 and 2, the last with 5M groups in id3), with 5% nulls and pre-sorted variants, and five joins. None of these queries or data shapes were written alongside Ibex. Pick a data set and a core count.

Published run: October 24, 2026, Ibex 950fafc1; r7i.4xlarge, 128 GiB, SMT off, pinned to 1 and 8 physical cores. One warmup and five timed iterations per question.

Result provenance: sources and engine versions are listed above. Accepted artifacts retain the exact Release executable and plugin fingerprints, runtime source hashes, and validation evidence. Reproduction instructions.

What this measures. The data comes from the upstream R generators, unmodified, and every engine reads the same CSV. Polars and DuckDB use the schemas of the upstream solutions: Polars runs its streaming engine with Int32 ids and string keys cast to Categorical; DuckDB uses ENUM keys where upstream does and a FLOAT v3. Ibex declares the same string keys Categorical and otherwise picks its own types (Int64 and Float64, which move more bytes than the upstream Int32/Float32). Loading is not timed. This harness uses resident DataFrames rather than upstream's Polars IPC scans, and aligns all-null sums across engines. Each engine reuses one process per dataset and releases preceding results outside timing. Warmup counts and all samples are retained in the TSV; the table shows the fastest fully materialised result. Every answer is checked across engines with a mandatory bounded full-output pass and key-bound fingerprints of every output column at the measured scale.

Publication requires every question to pass for all three engines. The summary rows show the geometric mean of the per-question ratios, and the ratio of the totals. Below 1 means Ibex is faster. The harness is benchmarking/h2o/run.py.

PDS-H (TPC-H)

Polars' TPC-H-derived benchmark (pola-rs/polars-benchmark): 22 decision-support queries over a star schema, read from Parquet. Polars (streaming engine) and DuckDB run upstream's own query code; Ibex runs its translations of the same queries. Pick a core count.

What this measures. The Parquet files come from upstream's generator, never from Ibex, and each query reads its inputs from them, as upstream PDS-H does. Every Ibex answer is diffed against Polars at the benchmarked scale before anything is timed. Times are the mean of 5 timed iterations after 1 warmup.

Published run: October 24, 2026, r7i.4xlarge, 128 GiB, SMT off; each engine pinned to the same 1, 2 or 8 physical cores. Ibex 950fafc1, upstream revision 401908a307f133a2dffbe3f5e0e0fb2b50ea310b, Polars 2.0.0, DuckDB 2.0.0. The archive retains per-run manifests for audit. Reproduce this configuration.

The summary rows compare Ibex with each engine: the geometric mean of the per-query ratios, and the ratio of the totals. Below 1 means Ibex is faster. The queries are in benchmarking/tpch/queries.