Skip to content

DO NOT MERGE: benchmark Arrow #12008 on #26816 - #26964

Draft
Rich-T-kid wants to merge 10 commits into
apache:mainfrom
Rich-T-kid:codex/bench-arrow-12008
Draft

DO NOT MERGE: benchmark Arrow #12008 on #26816#26964
Rich-T-kid wants to merge 10 commits into
apache:mainfrom
Rich-T-kid:codex/bench-arrow-12008

Conversation

@Rich-T-kid

@Rich-T-kid Rich-T-kid commented Oct 31, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Benchmark-only branch; do not merge.

Rationale for this change

Measure apache/arrow-rs#12008 end to end on the exact head of #26816 (fe597f220a0597120beb06632d80be828ec0bbb4).

What changes are included in this PR?

Two dependency-only commits make a controlled comparison:

  • e3c8ee1a8412c0e0f9c0144e72e9a4a147ee40c6 pins the parent of Arrow Fix license header #12008 (bf0cde345b14e5abdf3e8abb57f55d43da48b49f).
  • 55a2794bc3465c88ff4bda8acd33820fc7c74f60 advances only to Arrow Fix license header #12008 (90cbf31cd8dbacb5ae12566c5206624961461f90).

What is the testing strategy for this PR?

Run ClickBench with dictionary decoding enabled for both refs and compare the per-query timings. dfbench resolves and compiles against both exact Arrow revisions locally.

The full DataFusion clippy suite is not expected to pass on this temporary branch because Arrow 60.1 deprecates an API still used by this DataFusion revision.

Are there any user-facing changes?

No. This PR exists only to run benchmarks.

@github-actions github-actions Bot added common Related to common crate physical-plan Changes to the physical-plan crate labels Oct 31, 2026
@Rich-T-kid

Rich-T-kid commented Oct 31, 2026

Copy link
Copy Markdown
Contributor Author

run benchmark clickbench_partitioned

env:
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
  DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"
baseline:
  ref: "e3c8ee1a8412c0e0f9c0144e72e9a4a147ee40c6"
changed:
  ref: "55a2794bc3465c88ff4bda8acd33820fc7c74f60"

@adriangbot

adriangbot commented Oct 31, 2026

Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c6539694262-4207-f2x2b 6.12.105+ #1 SMP Sat Sep 19 07:55:25 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu) 
Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing 55a2794 (55a2794) to e3c8ee1 diff

Run configuration 
run benchmark clickbench_partitioned
env:
  DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
baseline:
  ref: "e3c8ee1a8412c0e0f9c0144e72e9a4a147ee40c6"
changed:
  ref: "55a2794bc3465c88ff4bda8acd33820fc7c74f60"

Results will be posted here when complete

File an issue against this benchmark runner

@adriangbot

adriangbot commented Oct 31, 2026

Copy link
Copy Markdown

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing 55a2794 (55a2794) to e3c8ee1 diff

Run configuration 
run benchmark clickbench_partitioned
env:
  DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
baseline:
  ref: "e3c8ee1a8412c0e0f9c0144e72e9a4a147ee40c6"
changed:
  ref: "55a2794bc3465c88ff4bda8acd33820fc7c74f60"
CPU Details (lscpu) 
Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected
Details 

Comparing HEAD and codex_bench-arrow-12008
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃       HEAD ┃ codex_bench-arrow-12008 ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0  │    1.14 ms │                 1.14 ms │     no change │
│ QQuery 1  │   11.24 ms │                11.17 ms │     no change │
│ QQuery 2  │   36.12 ms │                35.25 ms │     no change │
│ QQuery 3  │   31.16 ms │                31.03 ms │     no change │
│ QQuery 4  │  238.41 ms │               235.74 ms │     no change │
│ QQuery 5  │ 1035.92 ms │               826.53 ms │ +1.25x faster │
│ QQuery 6  │    1.18 ms │                 1.20 ms │     no change │
│ QQuery 7  │   12.49 ms │                12.29 ms │     no change │
│ QQuery 8  │  330.59 ms │               333.26 ms │     no change │
│ QQuery 9  │  473.28 ms │               475.32 ms │     no change │
│ QQuery 10 │  200.46 ms │               201.34 ms │     no change │
│ QQuery 11 │  212.45 ms │               212.22 ms │     no change │
│ QQuery 12 │ 1062.14 ms │               844.74 ms │ +1.26x faster │
│ QQuery 13 │ 1551.81 ms │              1298.19 ms │ +1.20x faster │
│ QQuery 14 │ 1160.91 ms │               913.46 ms │ +1.27x faster │
│ QQuery 15 │  287.44 ms │               293.15 ms │     no change │
│ QQuery 16 │ 2175.27 ms │              1379.22 ms │ +1.58x faster │
│ QQuery 17 │ 2172.82 ms │              1386.31 ms │ +1.57x faster │
│ QQuery 18 │ 3502.00 ms │              2033.89 ms │ +1.72x faster │
│ QQuery 19 │   27.24 ms │                27.04 ms │     no change │
│ QQuery 20 │ 1064.26 ms │              1043.90 ms │     no change │
│ QQuery 21 │ 1380.43 ms │              1363.03 ms │     no change │
│ QQuery 22 │ 3218.06 ms │              3243.78 ms │     no change │
│ QQuery 23 │ 9194.90 ms │              9326.06 ms │     no change │
│ QQuery 24 │   95.27 ms │                96.64 ms │     no change │
│ QQuery 25 │  353.51 ms │               352.57 ms │     no change │
│ QQuery 26 │   96.74 ms │                96.58 ms │     no change │
│ QQuery 27 │ 1137.80 ms │              1133.42 ms │     no change │
│ QQuery 28 │ 2579.69 ms │              2518.29 ms │     no change │
│ QQuery 29 │   39.11 ms │                38.62 ms │     no change │
│ QQuery 30 │  545.68 ms │               542.48 ms │     no change │
│ QQuery 31 │  535.49 ms │               540.99 ms │     no change │
│ QQuery 32 │  984.25 ms │               990.21 ms │     no change │
│ QQuery 33 │ 6963.34 ms │              3857.07 ms │ +1.81x faster │
│ QQuery 34 │ 7235.37 ms │              3948.00 ms │ +1.83x faster │
│ QQuery 35 │  294.41 ms │               295.18 ms │     no change │
│ QQuery 36 │  128.33 ms │               128.46 ms │     no change │
│ QQuery 37 │   97.00 ms │                97.08 ms │     no change │
│ QQuery 38 │   83.99 ms │                81.48 ms │     no change │
│ QQuery 39 │  272.65 ms │               290.41 ms │  1.07x slower │
│ QQuery 40 │   13.69 ms │                13.72 ms │     no change │
│ QQuery 41 │   13.16 ms │                13.54 ms │     no change │
│ QQuery 42 │   12.75 ms │                12.66 ms │     no change │
└───────────┴────────────┴─────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 50863.92ms │
│ Total Time (codex_bench-arrow-12008)   │ 40576.64ms │
│ Average Time (HEAD)                    │  1182.88ms │
│ Average Time (codex_bench-arrow-12008) │   943.64ms │
│ Queries Faster                         │          9 │
│ Queries Slower                         │          1 │
│ Queries with No Change                 │         33 │
│ Queries with Failure                   │          0 │
└────────────────────────────────────────┴────────────┘

Distribution per query (min / mean ±stddev / max):

Comparing HEAD and codex_bench-arrow-12008
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃                                   HEAD ┃                codex_bench-arrow-12008 ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0  │           1.14 / 4.14 ±5.94 / 16.01 ms │           1.14 / 4.17 ±5.98 / 16.14 ms │     no change │
│ QQuery 1  │         11.24 / 11.31 ±0.07 / 11.40 ms │         11.17 / 11.31 ±0.11 / 11.51 ms │     no change │
│ QQuery 2  │         36.12 / 36.38 ±0.19 / 36.66 ms │         35.25 / 35.54 ±0.16 / 35.70 ms │     no change │
│ QQuery 3  │         31.16 / 31.90 ±0.75 / 33.32 ms │         31.03 / 31.46 ±0.25 / 31.71 ms │     no change │
│ QQuery 4  │      238.41 / 242.76 ±3.91 / 248.74 ms │      235.74 / 241.59 ±4.59 / 249.40 ms │     no change │
│ QQuery 5  │  1035.92 / 1063.75 ±21.07 / 1091.95 ms │     826.53 / 852.46 ±15.06 / 873.10 ms │ +1.25x faster │
│ QQuery 6  │            1.18 / 1.35 ±0.23 / 1.81 ms │            1.20 / 1.36 ±0.23 / 1.82 ms │     no change │
│ QQuery 7  │         12.49 / 12.55 ±0.05 / 12.63 ms │         12.29 / 12.46 ±0.11 / 12.61 ms │     no change │
│ QQuery 8  │      330.59 / 337.76 ±8.79 / 355.07 ms │     333.26 / 343.05 ±11.50 / 360.47 ms │     no change │
│ QQuery 9  │     473.28 / 484.15 ±13.18 / 507.79 ms │     475.32 / 494.10 ±13.79 / 509.65 ms │     no change │
│ QQuery 10 │      200.46 / 202.56 ±1.63 / 205.48 ms │      201.34 / 202.28 ±0.66 / 202.97 ms │     no change │
│ QQuery 11 │      212.45 / 214.03 ±1.05 / 215.20 ms │      212.22 / 213.35 ±0.92 / 214.63 ms │     no change │
│ QQuery 12 │  1062.14 / 1087.81 ±17.54 / 1110.44 ms │     844.74 / 863.10 ±14.04 / 887.59 ms │ +1.26x faster │
│ QQuery 13 │  1551.81 / 1587.90 ±20.99 / 1611.25 ms │  1298.19 / 1359.39 ±58.90 / 1437.77 ms │ +1.17x faster │
│ QQuery 14 │  1160.91 / 1189.30 ±21.87 / 1228.50 ms │     913.46 / 940.32 ±24.32 / 980.02 ms │ +1.26x faster │
│ QQuery 15 │      287.44 / 295.24 ±9.31 / 312.94 ms │      293.15 / 298.45 ±5.05 / 306.90 ms │     no change │
│ QQuery 16 │  2175.27 / 2217.59 ±44.16 / 2297.71 ms │  1379.22 / 1418.97 ±36.52 / 1481.37 ms │ +1.56x faster │
│ QQuery 17 │  2172.82 / 2271.74 ±59.39 / 2358.05 ms │  1386.31 / 1422.01 ±27.55 / 1462.05 ms │ +1.60x faster │
│ QQuery 18 │ 3502.00 / 3720.13 ±143.90 / 3897.19 ms │  2033.89 / 2085.44 ±38.86 / 2145.73 ms │ +1.78x faster │
│ QQuery 19 │         27.24 / 29.14 ±2.80 / 34.60 ms │         27.04 / 32.39 ±9.91 / 52.20 ms │  1.11x slower │
│ QQuery 20 │  1064.26 / 1082.05 ±16.44 / 1111.59 ms │  1043.90 / 1069.31 ±22.57 / 1111.69 ms │     no change │
│ QQuery 21 │  1380.43 / 1398.17 ±15.31 / 1425.62 ms │   1363.03 / 1369.27 ±6.20 / 1377.57 ms │     no change │
│ QQuery 22 │  3218.06 / 3275.40 ±36.22 / 3318.76 ms │  3243.78 / 3265.71 ±15.44 / 3289.05 ms │     no change │
│ QQuery 23 │  9194.90 / 9352.63 ±92.66 / 9470.94 ms │  9326.06 / 9427.37 ±74.12 / 9553.00 ms │     no change │
│ QQuery 24 │      95.27 / 133.19 ±71.80 / 276.75 ms │      96.64 / 106.69 ±12.99 / 131.65 ms │ +1.25x faster │
│ QQuery 25 │      353.51 / 358.16 ±3.76 / 363.02 ms │      352.57 / 356.38 ±3.45 / 362.17 ms │     no change │
│ QQuery 26 │         96.74 / 97.84 ±0.76 / 99.05 ms │         96.58 / 97.89 ±1.41 / 99.62 ms │     no change │
│ QQuery 27 │  1137.80 / 1151.83 ±10.79 / 1167.36 ms │   1133.42 / 1144.21 ±8.32 / 1156.22 ms │     no change │
│ QQuery 28 │  2579.69 / 2602.96 ±17.15 / 2631.16 ms │  2518.29 / 2537.85 ±17.55 / 2563.40 ms │     no change │
│ QQuery 29 │         39.11 / 39.39 ±0.21 / 39.72 ms │         38.62 / 42.95 ±7.39 / 57.66 ms │  1.09x slower │
│ QQuery 30 │     545.68 / 558.63 ±12.80 / 580.51 ms │      542.48 / 551.14 ±7.26 / 562.41 ms │     no change │
│ QQuery 31 │     535.49 / 553.36 ±12.38 / 573.94 ms │      540.99 / 551.53 ±6.97 / 560.92 ms │     no change │
│ QQuery 32 │   984.25 / 1048.39 ±35.95 / 1087.58 ms │   990.21 / 1031.59 ±28.56 / 1070.11 ms │     no change │
│ QQuery 33 │ 6963.34 / 7277.45 ±178.86 / 7501.21 ms │ 3857.07 / 4124.49 ±273.44 / 4644.68 ms │ +1.76x faster │
│ QQuery 34 │ 7235.37 / 7482.38 ±135.96 / 7623.27 ms │  3948.00 / 4062.55 ±82.45 / 4146.92 ms │ +1.84x faster │
│ QQuery 35 │     294.41 / 310.86 ±13.08 / 333.35 ms │     295.18 / 328.42 ±37.17 / 387.89 ms │  1.06x slower │
│ QQuery 36 │     128.33 / 149.14 ±15.87 / 167.27 ms │      128.46 / 135.94 ±6.51 / 146.16 ms │ +1.10x faster │
│ QQuery 37 │        97.00 / 98.70 ±2.43 / 103.47 ms │       97.08 / 101.83 ±3.84 / 106.46 ms │     no change │
│ QQuery 38 │         83.99 / 86.91 ±2.66 / 91.61 ms │         81.48 / 84.87 ±2.74 / 89.15 ms │     no change │
│ QQuery 39 │     272.65 / 290.00 ±10.24 / 303.95 ms │      290.41 / 295.78 ±4.12 / 300.83 ms │     no change │
│ QQuery 40 │         13.69 / 14.05 ±0.57 / 15.17 ms │         13.72 / 14.85 ±1.10 / 16.20 ms │  1.06x slower │
│ QQuery 41 │         13.16 / 14.13 ±1.26 / 16.59 ms │         13.54 / 15.29 ±2.16 / 19.11 ms │  1.08x slower │
│ QQuery 42 │         12.75 / 13.97 ±1.99 / 17.94 ms │         12.66 / 12.85 ±0.15 / 13.11 ms │ +1.09x faster │
└───────────┴────────────────────────────────────────┴────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 52431.07ms │
│ Total Time (codex_bench-arrow-12008)   │ 41591.97ms │
│ Average Time (HEAD)                    │  1219.33ms │
│ Average Time (codex_bench-arrow-12008) │   967.26ms │
│ Queries Faster                         │         12 │
│ Queries Slower                         │          5 │
│ Queries with No Change                 │         26 │
│ Queries with Failure                   │          0 │
└────────────────────────────────────────┴────────────┘

Memory Pool Peaks

Peak MemoryPool reservation per query — what DataFusion's accounting believes it reserved. Recorded only for benchmarks that write a results JSON (the dfbench suites, and the suites bench.sh runs through the Criterion SQL harness since #25644), and only under a memory limit: DATAFUSION_RUNTIME_MEMORY_LIMIT, or one the suite sets itself.

Base: e3c8ee1a8412c0e0f9c0144e72e9a4a147ee40c6 | Changed: 55a2794bc3465c88ff4bda8acd33820fc7c74f60

clickbench_partitioned

Query Base Changed Change
Query 0 0 B 0 B 0.0%
Query 1 104 B 104 B +0.0%
Query 2 936 B 936 B +0.0%
Query 3 312 B 312 B +0.0%
Query 4 844.6 MiB 836.3 MiB -1.0%
Query 5 1.9 GiB 1.8 GiB -1.6%
Query 6 0 B 0 B 0.0%
Query 7 40.2 MiB 40.0 MiB -0.4%
Query 8 944.0 MiB 967.2 MiB +2.5%
Query 9 594.2 MiB 552.0 MiB -7.1%
Query 10 72.6 MiB 73.3 MiB +1.1%
Query 11 77.0 MiB 77.5 MiB +0.6%
Query 12 1.9 GiB 1.9 GiB +2.1%
Query 13 1.9 GiB 2.7 GiB +38.4%
Query 14 2.0 GiB 2.0 GiB +2.9%
Query 15 1.2 GiB 1.2 GiB +2.4%
Query 16 3.5 GiB 3.4 GiB -3.9%
Query 17 3.6 GiB 3.3 GiB -7.6%
Query 18 7.6 GiB 7.2 GiB -5.4%
Query 19 0 B 0 B 0.0%
Query 20 104 B 104 B +0.0%
Query 21 1.4 MiB 1.1 MiB -20.1%
Query 22 2.6 MiB 2.4 MiB -6.6%
Query 23 12.1 MiB 12.8 MiB +6.1%
Query 24 6.8 MiB 5.5 MiB -19.6%
Query 25 8.1 MiB 8.1 MiB +0.0%
Query 26 7.0 MiB 7.9 MiB +12.9%
Query 27 2.4 MiB 2.5 MiB +1.8%
Query 28 1.9 GiB 2.0 GiB +2.0%
Query 29 624 B 624 B +0.0%
Query 30 658.6 MiB 653.2 MiB -0.8%
Query 31 1.5 GiB 1.5 GiB -1.5%
Query 32 7.2 GiB 7.2 GiB -0.0%
Query 33 16.0 GiB 17.1 GiB +6.7%
Query 34 12.7 GiB 14.2 GiB +12.0%
Query 35 657.0 MiB 665.0 MiB +1.2%
Query 36 474.1 MiB 474.1 MiB +0.0%
Query 37 8.1 MiB 8.1 MiB -0.0%
Query 38 5.3 MiB 5.2 MiB -2.1%
Query 39 900.1 MiB 909.9 MiB +1.1%
Query 40 2.3 MiB 2.3 MiB +3.3%
Query 41 3.1 MiB 3.1 MiB -0.0%
Query 42 1.7 MiB 1.7 MiB +0.4%

Pool accounting vs. process RSS

Max pool peak is the largest reservation any single query in the run reached; peak RSS covers the whole invocation, including data loading and allocator retention, and the two high-water marks need not coincide in time. The gap is therefore an upper bound on what the pool did not account for, not a measurement of it.

Benchmark Side Max pool peak Peak RSS Gap RSS / pool
clickbench_partitioned base (e3c8ee1a8412c0e0f9c0144e72e9a4a147ee40c6) 16.0 GiB 15.5 GiB −534.0 MiB 1.0×
clickbench_partitioned changed (55a2794bc3465c88ff4bda8acd33820fc7c74f60) 17.1 GiB 15.7 GiB −1.3 GiB 0.9×
Resource Usage

clickbench_partitioned — base (merge-base)

Metric Value
Wall time 265.1s
Peak memory 15.5 GiB
Avg memory 6.3 GiB
CPU user 2106.4s
CPU sys 167.1s
Peak spill 0 B

clickbench_partitioned — branch

Metric Value
Wall time 210.0s
Peak memory 15.7 GiB
Avg memory 5.9 GiB
CPU user 1981.0s
CPU sys 137.2s
Peak spill 0 B

File an issue against this benchmark runner

@Rich-T-kid

Rich-T-kid commented Oct 31, 2026

Copy link
Copy Markdown
Contributor Author

run benchmark clickbench_partitioned

env:
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
baseline:
  ref: "55a2794bc3465c88ff4bda8acd33820fc7c74f60"
  env:
    DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "false"
changed:
  ref: "55a2794bc3465c88ff4bda8acd33820fc7c74f60"
  env:
    DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"

@adriangbot

adriangbot commented Oct 31, 2026

Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c6540702538-4208-n995k 6.12.105+ #1 SMP Sat Sep 19 07:55:25 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu) 
Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing 55a2794 (55a2794) to 55a2794 diff

Run configuration 
run benchmark clickbench_partitioned
env:
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
baseline:
  ref: "55a2794bc3465c88ff4bda8acd33820fc7c74f60"
  env:
    DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "false"
changed:
  ref: "55a2794bc3465c88ff4bda8acd33820fc7c74f60"
  env:
    DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"

Results will be posted here when complete

File an issue against this benchmark runner

@adriangbot

adriangbot commented Oct 31, 2026

Copy link
Copy Markdown

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing 55a2794 (55a2794) to 55a2794 diff

Run configuration 
run benchmark clickbench_partitioned
env:
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
baseline:
  ref: "55a2794bc3465c88ff4bda8acd33820fc7c74f60"
  env:
    DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "false"
changed:
  ref: "55a2794bc3465c88ff4bda8acd33820fc7c74f60"
  env:
    DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"
CPU Details (lscpu) 
Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected
Details 

Comparing HEAD and codex_bench-arrow-12008
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃       HEAD ┃ codex_bench-arrow-12008 ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0  │    1.19 ms │                 1.15 ms │     no change │
│ QQuery 1  │   11.50 ms │                11.05 ms │     no change │
│ QQuery 2  │   35.62 ms │                34.62 ms │     no change │
│ QQuery 3  │   31.15 ms │                30.62 ms │     no change │
│ QQuery 4  │  235.76 ms │               235.64 ms │     no change │
│ QQuery 5  │  271.23 ms │               855.39 ms │  3.15x slower │
│ QQuery 6  │    1.24 ms │                 1.19 ms │     no change │
│ QQuery 7  │   12.90 ms │                12.31 ms │     no change │
│ QQuery 8  │  324.92 ms │               335.87 ms │     no change │
│ QQuery 9  │  460.72 ms │               475.66 ms │     no change │
│ QQuery 10 │   63.73 ms │               199.55 ms │  3.13x slower │
│ QQuery 11 │   75.13 ms │               210.31 ms │  2.80x slower │
│ QQuery 12 │  258.34 ms │               849.87 ms │  3.29x slower │
│ QQuery 13 │  357.14 ms │              1289.29 ms │  3.61x slower │
│ QQuery 14 │  278.00 ms │               938.11 ms │  3.37x slower │
│ QQuery 15 │  281.79 ms │               294.14 ms │     no change │
│ QQuery 16 │  614.85 ms │              1420.80 ms │  2.31x slower │
│ QQuery 17 │  615.94 ms │              1400.66 ms │  2.27x slower │
│ QQuery 18 │ 1200.53 ms │              2050.32 ms │  1.71x slower │
│ QQuery 19 │   29.36 ms │                27.48 ms │ +1.07x faster │
│ QQuery 20 │  519.17 ms │              1058.37 ms │  2.04x slower │
│ QQuery 21 │  509.00 ms │              1372.98 ms │  2.70x slower │
│ QQuery 22 │  964.99 ms │              3283.89 ms │  3.40x slower │
│ QQuery 23 │ 3053.69 ms │              9289.90 ms │  3.04x slower │
│ QQuery 24 │   39.94 ms │                96.13 ms │  2.41x slower │
│ QQuery 25 │  105.18 ms │               354.01 ms │  3.37x slower │
│ QQuery 26 │   40.71 ms │                97.11 ms │  2.39x slower │
│ QQuery 27 │  508.05 ms │              1133.62 ms │  2.23x slower │
│ QQuery 28 │ 2881.58 ms │              2543.52 ms │ +1.13x faster │
│ QQuery 29 │   39.24 ms │                38.25 ms │     no change │
│ QQuery 30 │  310.39 ms │               538.88 ms │  1.74x slower │
│ QQuery 31 │  287.32 ms │               531.16 ms │  1.85x slower │
│ QQuery 32 │  939.99 ms │              1009.13 ms │  1.07x slower │
│ QQuery 33 │ 1444.42 ms │              3851.07 ms │  2.67x slower │
│ QQuery 34 │ 1475.97 ms │              3976.54 ms │  2.69x slower │
│ QQuery 35 │  291.94 ms │               298.02 ms │     no change │
│ QQuery 36 │   66.41 ms │               125.51 ms │  1.89x slower │
│ QQuery 37 │   35.35 ms │                95.65 ms │  2.71x slower │
│ QQuery 38 │   40.18 ms │                80.85 ms │  2.01x slower │
│ QQuery 39 │  145.76 ms │               284.13 ms │  1.95x slower │
│ QQuery 40 │   14.69 ms │                13.59 ms │ +1.08x faster │
│ QQuery 41 │   13.79 ms │                12.96 ms │ +1.06x faster │
│ QQuery 42 │   12.70 ms │                12.47 ms │     no change │
└───────────┴────────────┴─────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 18901.50ms │
│ Total Time (codex_bench-arrow-12008)   │ 40771.76ms │
│ Average Time (HEAD)                    │   439.57ms │
│ Average Time (codex_bench-arrow-12008) │   948.18ms │
│ Queries Faster                         │          4 │
│ Queries Slower                         │         26 │
│ Queries with No Change                 │         13 │
│ Queries with Failure                   │          0 │
└────────────────────────────────────────┴────────────┘

Distribution per query (min / mean ±stddev / max):

Comparing HEAD and codex_bench-arrow-12008
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃                                  HEAD ┃                codex_bench-arrow-12008 ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0  │          1.19 / 4.16 ±5.86 / 15.88 ms │           1.15 / 4.21 ±6.01 / 16.24 ms │     no change │
│ QQuery 1  │        11.50 / 11.65 ±0.13 / 11.84 ms │         11.05 / 11.26 ±0.13 / 11.42 ms │     no change │
│ QQuery 2  │        35.62 / 35.99 ±0.30 / 36.51 ms │         34.62 / 35.14 ±0.52 / 36.03 ms │     no change │
│ QQuery 3  │        31.15 / 32.20 ±0.64 / 32.90 ms │         30.62 / 30.88 ±0.20 / 31.21 ms │     no change │
│ QQuery 4  │     235.76 / 239.50 ±3.23 / 244.23 ms │      235.64 / 241.36 ±6.03 / 250.84 ms │     no change │
│ QQuery 5  │     271.23 / 276.31 ±5.61 / 287.20 ms │      855.39 / 863.35 ±6.55 / 875.27 ms │  3.12x slower │
│ QQuery 6  │           1.24 / 1.40 ±0.24 / 1.87 ms │            1.19 / 1.36 ±0.24 / 1.82 ms │     no change │
│ QQuery 7  │        12.90 / 12.98 ±0.06 / 13.06 ms │         12.31 / 12.39 ±0.06 / 12.50 ms │     no change │
│ QQuery 8  │     324.92 / 331.23 ±3.65 / 334.45 ms │      335.87 / 340.69 ±4.03 / 345.95 ms │     no change │
│ QQuery 9  │     460.72 / 467.31 ±5.34 / 475.35 ms │     475.66 / 484.57 ±10.26 / 503.61 ms │     no change │
│ QQuery 10 │        63.73 / 65.86 ±3.51 / 72.83 ms │      199.55 / 200.35 ±0.68 / 201.58 ms │  3.04x slower │
│ QQuery 11 │        75.13 / 76.44 ±1.34 / 79.00 ms │      210.31 / 211.05 ±0.53 / 211.93 ms │  2.76x slower │
│ QQuery 12 │     258.34 / 262.82 ±3.30 / 267.85 ms │     849.87 / 871.57 ±18.57 / 903.51 ms │  3.32x slower │
│ QQuery 13 │    357.14 / 372.80 ±12.70 / 391.04 ms │  1289.29 / 1369.29 ±77.31 / 1466.46 ms │  3.67x slower │
│ QQuery 14 │     278.00 / 281.11 ±2.48 / 284.48 ms │     938.11 / 956.76 ±14.33 / 973.81 ms │  3.40x slower │
│ QQuery 15 │     281.79 / 289.72 ±6.72 / 297.52 ms │      294.14 / 301.69 ±6.36 / 311.42 ms │     no change │
│ QQuery 16 │    614.85 / 628.29 ±14.64 / 655.83 ms │  1420.80 / 1444.95 ±17.06 / 1467.22 ms │  2.30x slower │
│ QQuery 17 │     615.94 / 627.14 ±6.83 / 633.80 ms │  1400.66 / 1444.48 ±24.50 / 1470.22 ms │  2.30x slower │
│ QQuery 18 │ 1200.53 / 1229.81 ±15.89 / 1247.52 ms │  2050.32 / 2114.05 ±64.89 / 2239.23 ms │  1.72x slower │
│ QQuery 19 │       29.36 / 37.60 ±10.43 / 56.41 ms │        27.48 / 41.11 ±14.31 / 67.21 ms │  1.09x slower │
│ QQuery 20 │     519.17 / 528.23 ±7.21 / 538.97 ms │   1058.37 / 1067.73 ±6.31 / 1077.70 ms │  2.02x slower │
│ QQuery 21 │     509.00 / 513.49 ±7.34 / 528.06 ms │   1372.98 / 1385.26 ±6.52 / 1391.01 ms │  2.70x slower │
│ QQuery 22 │   964.99 / 983.25 ±13.32 / 1005.63 ms │  3283.89 / 3328.85 ±31.19 / 3372.46 ms │  3.39x slower │
│ QQuery 23 │ 3053.69 / 3088.30 ±28.51 / 3130.05 ms │ 9289.90 / 9527.58 ±123.22 / 9618.58 ms │  3.09x slower │
│ QQuery 24 │        39.94 / 40.43 ±0.56 / 41.35 ms │      96.13 / 126.22 ±56.87 / 239.91 ms │  3.12x slower │
│ QQuery 25 │     105.18 / 113.07 ±9.30 / 124.82 ms │      354.01 / 358.31 ±3.84 / 364.11 ms │  3.17x slower │
│ QQuery 26 │        40.71 / 43.13 ±2.66 / 47.85 ms │        97.11 / 98.52 ±1.40 / 101.06 ms │  2.28x slower │
│ QQuery 27 │     508.05 / 514.22 ±4.40 / 521.14 ms │  1133.62 / 1158.49 ±17.35 / 1185.42 ms │  2.25x slower │
│ QQuery 28 │ 2881.58 / 2922.67 ±23.35 / 2949.00 ms │  2543.52 / 2582.08 ±40.25 / 2654.95 ms │ +1.13x faster │
│ QQuery 29 │       39.24 / 49.11 ±13.92 / 75.68 ms │        38.25 / 52.50 ±21.66 / 94.70 ms │  1.07x slower │
│ QQuery 30 │     310.39 / 315.08 ±3.29 / 319.62 ms │     538.88 / 547.95 ±10.57 / 568.40 ms │  1.74x slower │
│ QQuery 31 │     287.32 / 292.09 ±5.97 / 303.48 ms │     531.16 / 555.24 ±19.29 / 584.87 ms │  1.90x slower │
│ QQuery 32 │    939.99 / 965.31 ±16.43 / 982.67 ms │  1009.13 / 1086.80 ±52.64 / 1155.01 ms │  1.13x slower │
│ QQuery 33 │ 1444.42 / 1474.59 ±19.02 / 1495.81 ms │  3851.07 / 4016.51 ±90.64 / 4110.54 ms │  2.72x slower │
│ QQuery 34 │  1475.97 / 1488.10 ±7.86 / 1497.62 ms │ 3976.54 / 4066.06 ±118.93 / 4297.66 ms │  2.73x slower │
│ QQuery 35 │    291.94 / 313.40 ±22.91 / 345.70 ms │     298.02 / 326.80 ±45.64 / 417.53 ms │     no change │
│ QQuery 36 │        66.41 / 71.61 ±6.11 / 83.16 ms │      125.51 / 139.33 ±7.26 / 145.90 ms │  1.95x slower │
│ QQuery 37 │        35.35 / 36.72 ±1.17 / 38.29 ms │       95.65 / 102.94 ±6.11 / 113.91 ms │  2.80x slower │
│ QQuery 38 │        40.18 / 44.49 ±3.42 / 50.63 ms │         80.85 / 85.18 ±4.93 / 94.32 ms │  1.91x slower │
│ QQuery 39 │     145.76 / 160.21 ±8.42 / 171.44 ms │      284.13 / 295.45 ±7.64 / 304.42 ms │  1.84x slower │
│ QQuery 40 │        14.69 / 18.26 ±4.63 / 26.24 ms │         13.59 / 14.11 ±0.39 / 14.67 ms │ +1.29x faster │
│ QQuery 41 │        13.79 / 14.38 ±0.44 / 15.14 ms │         12.96 / 13.89 ±1.18 / 16.20 ms │     no change │
│ QQuery 42 │        12.70 / 13.22 ±0.28 / 13.45 ms │         12.47 / 12.78 ±0.22 / 13.13 ms │     no change │
└───────────┴───────────────────────────────────────┴────────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 19287.66ms │
│ Total Time (codex_bench-arrow-12008)   │ 41929.12ms │
│ Average Time (HEAD)                    │   448.55ms │
│ Average Time (codex_bench-arrow-12008) │   975.10ms │
│ Queries Faster                         │          2 │
│ Queries Slower                         │         28 │
│ Queries with No Change                 │         13 │
│ Queries with Failure                   │          0 │
└────────────────────────────────────────┴────────────┘

Memory Pool Peaks

Peak MemoryPool reservation per query — what DataFusion's accounting believes it reserved. Recorded only for benchmarks that write a results JSON (the dfbench suites, and the suites bench.sh runs through the Criterion SQL harness since #25644), and only under a memory limit: DATAFUSION_RUNTIME_MEMORY_LIMIT, or one the suite sets itself.

Base: 55a2794bc3465c88ff4bda8acd33820fc7c74f60 | Changed: 55a2794bc3465c88ff4bda8acd33820fc7c74f60

clickbench_partitioned

Query Base Changed Change
Query 0 0 B 0 B 0.0%
Query 1 104 B 104 B +0.0%
Query 2 936 B 936 B +0.0%
Query 3 312 B 312 B +0.0%
Query 4 812.2 MiB 830.2 MiB +2.2%
Query 5 1.2 GiB 1.8 GiB +47.6%
Query 6 0 B 0 B 0.0%
Query 7 40.3 MiB 60.3 MiB +49.8%
Query 8 946.2 MiB 946.8 MiB +0.1%
Query 9 593.9 MiB 635.2 MiB +7.0%
Query 10 117.2 MiB 73.3 MiB -37.5%
Query 11 117.5 MiB 76.7 MiB -34.7%
Query 12 1.4 GiB 1.9 GiB +40.8%
Query 13 1.6 GiB 2.3 GiB +42.9%
Query 14 1.3 GiB 2.0 GiB +48.0%
Query 15 1.2 GiB 1.2 GiB -1.7%
Query 16 3.3 GiB 3.4 GiB +2.6%
Query 17 3.4 GiB 3.6 GiB +5.0%
Query 18 8.4 GiB 7.4 GiB -12.2%
Query 19 0 B 0 B 0.0%
Query 20 104 B 104 B +0.0%
Query 21 4.0 MiB 1.5 MiB -61.6%
Query 22 3.4 MiB 2.5 MiB -25.8%
Query 23 36.4 MiB 12.6 MiB -65.3%
Query 24 59.3 MiB 6.7 MiB -88.7%
Query 25 167.0 MiB 8.1 MiB -95.1%
Query 26 61.3 MiB 7.5 MiB -87.8%
Query 27 2.5 MiB 2.7 MiB +8.9%
Query 28 1.7 GiB 1.9 GiB +11.6%
Query 29 624 B 624 B +0.0%
Query 30 688.4 MiB 646.3 MiB -6.1%
Query 31 1.5 GiB 1.5 GiB +0.0%
Query 32 7.1 GiB 7.1 GiB +0.3%
Query 33 8.2 GiB 13.8 GiB +68.1%
Query 34 8.5 GiB 14.8 GiB +74.2%
Query 35 678.7 MiB 660.9 MiB -2.6%
Query 36 117.4 MiB 474.1 MiB +303.9%
Query 37 7.3 MiB 8.1 MiB +11.6%
Query 38 5.2 MiB 5.3 MiB +2.8%
Query 39 298.2 MiB 909.9 MiB +205.2%
Query 40 2.3 MiB 2.3 MiB -1.0%
Query 41 2.9 MiB 3.1 MiB +9.7%
Query 42 1.7 MiB 1.5 MiB -12.9%

Pool accounting vs. process RSS

Max pool peak is the largest reservation any single query in the run reached; peak RSS covers the whole invocation, including data loading and allocator retention, and the two high-water marks need not coincide in time. The gap is therefore an upper bound on what the pool did not account for, not a measurement of it.

Benchmark Side Max pool peak Peak RSS Gap RSS / pool
clickbench_partitioned base (55a2794bc3465c88ff4bda8acd33820fc7c74f60) 8.5 GiB 11.3 GiB 2.8 GiB 1.3×
clickbench_partitioned changed (55a2794bc3465c88ff4bda8acd33820fc7c74f60) 14.8 GiB 13.2 GiB −1.6 GiB 0.9×
Resource Usage

clickbench_partitioned — base (merge-base)

Metric Value
Wall time 100.0s
Peak memory 11.3 GiB
Avg memory 4.4 GiB
CPU user 989.2s
CPU sys 67.5s
Peak spill 0 B

clickbench_partitioned — branch

Metric Value
Wall time 210.1s
Peak memory 13.2 GiB
Avg memory 5.7 GiB
CPU user 1994.7s
CPU sys 141.8s
Peak spill 284.4 MiB

File an issue against this benchmark runner

@Rich-T-kid Rich-T-kid mentioned this pull request Oct 31, 2026
Draft
@Rich-T-kid

Rich-T-kid commented Oct 31, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor Author

run benchmark clickbench_partitioned

env:
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
  DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"
baseline:
  ref: "55a2794bc3465c88ff4bda8acd33820fc7c74f60"
changed:
  ref: "4b73f22a53c3cdec4c44da9eb5ae736ec4ac4bfb"

@github-actions github-actions Bot added documentation Improvements or additions to documentation sqllogictest SQL Logic Tests (.slt) datasource Changes to the datasource crate labels Oct 31, 2026
@adriangbot

adriangbot commented Oct 31, 2026

Copy link
Copy Markdown

Benchmark for this request failed before finishing (Kubernetes reason: BackoffLimitExceeded).

Benchmarks requested: clickbench_partitioned

Runner log (last 40 lines) 
2026-10-31T14:36:15.678035Z  INFO runner starting benchmark runner bench_type=Datafusion, pr_url=https://github.com/apache/datafusion/pull/26964, benchmarks=clickbench_partitioned
2026-10-31T14:36:15.756077Z  INFO benchmark_controller::runner::bench_datafusion === Cloning PR branch ===
2026-10-31T14:36:15.756099Z  INFO benchmark_controller::runner::shell running command cmd=git, args=["clone", "--depth=200", "https://github.com/apache/datafusion.git", "/workspace/datafusion-branch"], cwd="/"
2026-10-31T14:36:20.764736Z  INFO benchmark_controller::runner::shell running command cmd=git, args=["fetch", "origin", "refs/pull/26964/head:codex/bench-arrow-12008", "main"], cwd="/workspace/datafusion-branch"
2026-10-31T14:36:25.766835Z  INFO benchmark_controller::runner::shell running command cmd=git, args=["checkout", "codex/bench-arrow-12008"], cwd="/workspace/datafusion-branch"
2026-10-31T14:36:30.769791Z  INFO benchmark_controller::runner::shell running command cmd=git, args=["merge-base", "HEAD", "origin/main"], cwd="/workspace/datafusion-branch"
2026-10-31T14:36:35.775241Z  INFO benchmark_controller::runner::bench_datafusion === Checking out custom changed ref === changed_ref=4b73f22a5e19db468646bd8fb99dde1a277d770b
2026-10-31T14:36:35.775256Z  INFO benchmark_controller::runner::shell running command cmd=git, args=["fetch", "origin", "refs/pull/26964/head"], cwd="/workspace/datafusion-branch"
2026-10-31T14:36:40.777546Z  INFO benchmark_controller::runner::shell running command cmd=git, args=["fetch", "origin"], cwd="/workspace/datafusion-branch"
2026-10-31T14:36:45.779082Z  INFO benchmark_controller::runner::shell running command cmd=git, args=["-c", "advice.detachedHead=false", "checkout", "4b73f22a5e19db468646bd8fb99dde1a277d770b"], cwd="/workspace/datafusion-branch"
2026-10-31T14:36:50.781345Z ERROR runner benchmark failed error=git checkout: git -c advice.detachedHead=false checkout 4b73f22a5e19db468646bd8fb99dde1a277d770b exited with code 128
stdout:

stderr:
fatal: reference is not a tree: 4b73f22a5e19db468646bd8fb99dde1a277d770b
Kubernetes message 
Job has reached the specified backoff limit

File an issue against this benchmark runner

@Rich-T-kid

Rich-T-kid commented Oct 31, 2026

Copy link
Copy Markdown
Contributor Author

run benchmark clickbench_partitioned

env:
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
  DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"
baseline:
  ref: "55a2794bc3465c88ff4bda8acd33820fc7c74f60"
changed:
  ref: "4b73f22a53c3cdec4c44da9eb5ae736ec4ac4bfb"

@adriangbot

adriangbot commented Oct 31, 2026

Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c6545220892-4211-vmwwk 6.12.105+ #1 SMP Sat Sep 19 07:55:25 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu) 
Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing 4b73f22 (4b73f22) to 55a2794 diff

Run configuration 
run benchmark clickbench_partitioned
env:
  DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
baseline:
  ref: "55a2794bc3465c88ff4bda8acd33820fc7c74f60"
changed:
  ref: "4b73f22a53c3cdec4c44da9eb5ae736ec4ac4bfb"

Results will be posted here when complete

File an issue against this benchmark runner

@adriangbot

adriangbot commented Oct 31, 2026

Copy link
Copy Markdown

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing 4b73f22 (4b73f22) to 55a2794 diff

Run configuration 
run benchmark clickbench_partitioned
env:
  DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
baseline:
  ref: "55a2794bc3465c88ff4bda8acd33820fc7c74f60"
changed:
  ref: "4b73f22a53c3cdec4c44da9eb5ae736ec4ac4bfb"
CPU Details (lscpu) 
Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected
Details 

Comparing HEAD and codex_bench-arrow-12008
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃       HEAD ┃ codex_bench-arrow-12008 ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0  │    1.15 ms │                 1.16 ms │     no change │
│ QQuery 1  │   11.00 ms │                11.56 ms │  1.05x slower │
│ QQuery 2  │   34.76 ms │                35.00 ms │     no change │
│ QQuery 3  │   30.42 ms │                30.90 ms │     no change │
│ QQuery 4  │  233.28 ms │               234.60 ms │     no change │
│ QQuery 5  │  837.38 ms │               272.77 ms │ +3.07x faster │
│ QQuery 6  │    1.17 ms │                 1.23 ms │  1.05x slower │
│ QQuery 7  │   12.27 ms │                12.85 ms │     no change │
│ QQuery 8  │  329.40 ms │               325.01 ms │     no change │
│ QQuery 9  │  467.87 ms │               465.02 ms │     no change │
│ QQuery 10 │  198.64 ms │               201.10 ms │     no change │
│ QQuery 11 │  209.32 ms │               212.84 ms │     no change │
│ QQuery 12 │  858.85 ms │               259.39 ms │ +3.31x faster │
│ QQuery 13 │ 1324.34 ms │               359.72 ms │ +3.68x faster │
│ QQuery 14 │  912.34 ms │               283.16 ms │ +3.22x faster │
│ QQuery 15 │  285.47 ms │               283.44 ms │     no change │
│ QQuery 16 │ 1370.63 ms │               612.03 ms │ +2.24x faster │
│ QQuery 17 │ 1370.10 ms │               624.66 ms │ +2.19x faster │
│ QQuery 18 │ 2010.07 ms │              1223.49 ms │ +1.64x faster │
│ QQuery 19 │   26.77 ms │                27.22 ms │     no change │
│ QQuery 20 │ 1050.40 ms │               514.71 ms │ +2.04x faster │
│ QQuery 21 │ 1354.57 ms │               506.21 ms │ +2.68x faster │
│ QQuery 22 │ 3256.32 ms │               967.07 ms │ +3.37x faster │
│ QQuery 23 │ 9229.71 ms │              6119.19 ms │ +1.51x faster │
│ QQuery 24 │   96.28 ms │                40.91 ms │ +2.35x faster │
│ QQuery 25 │  350.17 ms │               105.72 ms │ +3.31x faster │
│ QQuery 26 │   95.88 ms │                41.26 ms │ +2.32x faster │
│ QQuery 27 │ 1136.97 ms │               508.22 ms │ +2.24x faster │
│ QQuery 28 │ 2539.96 ms │              2856.77 ms │  1.12x slower │
│ QQuery 29 │   38.06 ms │                38.88 ms │     no change │
│ QQuery 30 │  550.34 ms │               305.52 ms │ +1.80x faster │
│ QQuery 31 │  532.43 ms │               273.47 ms │ +1.95x faster │
│ QQuery 32 │ 1096.22 ms │               955.33 ms │ +1.15x faster │
│ QQuery 33 │ 3935.94 ms │              1479.12 ms │ +2.66x faster │
│ QQuery 34 │ 3970.33 ms │              1468.01 ms │ +2.70x faster │
│ QQuery 35 │  304.82 ms │               290.80 ms │     no change │
│ QQuery 36 │  130.83 ms │                67.40 ms │ +1.94x faster │
│ QQuery 37 │   96.74 ms │                34.89 ms │ +2.77x faster │
│ QQuery 38 │   81.46 ms │                40.47 ms │ +2.01x faster │
│ QQuery 39 │  279.78 ms │               132.76 ms │ +2.11x faster │
│ QQuery 40 │   13.81 ms │                14.12 ms │     no change │
│ QQuery 41 │   13.06 ms │                13.12 ms │     no change │
│ QQuery 42 │   12.64 ms │                12.92 ms │     no change │
└───────────┴────────────┴─────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 40691.96ms │
│ Total Time (codex_bench-arrow-12008)   │ 22264.05ms │
│ Average Time (HEAD)                    │   946.32ms │
│ Average Time (codex_bench-arrow-12008) │   517.77ms │
│ Queries Faster                         │         24 │
│ Queries Slower                         │          3 │
│ Queries with No Change                 │         16 │
│ Queries with Failure                   │          0 │
└────────────────────────────────────────┴────────────┘

Distribution per query (min / mean ±stddev / max):

Comparing HEAD and codex_bench-arrow-12008
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃                                   HEAD ┃               codex_bench-arrow-12008 ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0  │           1.15 / 4.17 ±5.96 / 16.09 ms │          1.16 / 4.09 ±5.77 / 15.64 ms │     no change │
│ QQuery 1  │         11.00 / 11.20 ±0.15 / 11.40 ms │        11.56 / 11.65 ±0.09 / 11.82 ms │     no change │
│ QQuery 2  │         34.76 / 34.93 ±0.18 / 35.25 ms │        35.00 / 35.26 ±0.31 / 35.80 ms │     no change │
│ QQuery 3  │         30.42 / 31.47 ±1.05 / 32.87 ms │        30.90 / 32.58 ±1.34 / 34.66 ms │     no change │
│ QQuery 4  │      233.28 / 236.70 ±4.74 / 245.64 ms │     234.60 / 238.20 ±3.90 / 245.00 ms │     no change │
│ QQuery 5  │      837.38 / 848.41 ±9.28 / 864.87 ms │     272.77 / 275.79 ±2.80 / 280.96 ms │ +3.08x faster │
│ QQuery 6  │            1.17 / 1.33 ±0.23 / 1.79 ms │           1.23 / 1.40 ±0.24 / 1.88 ms │  1.06x slower │
│ QQuery 7  │         12.27 / 12.35 ±0.07 / 12.46 ms │        12.85 / 12.98 ±0.08 / 13.09 ms │  1.05x slower │
│ QQuery 8  │      329.40 / 334.83 ±3.42 / 339.35 ms │     325.01 / 330.79 ±4.29 / 337.63 ms │     no change │
│ QQuery 9  │      467.87 / 472.39 ±5.08 / 482.15 ms │     465.02 / 474.67 ±8.07 / 487.40 ms │     no change │
│ QQuery 10 │      198.64 / 200.24 ±1.92 / 203.98 ms │     201.10 / 204.99 ±5.81 / 216.50 ms │     no change │
│ QQuery 11 │      209.32 / 210.73 ±1.08 / 212.41 ms │     212.84 / 213.89 ±0.97 / 215.18 ms │     no change │
│ QQuery 12 │      858.85 / 862.74 ±4.24 / 869.85 ms │     259.39 / 270.71 ±8.71 / 281.08 ms │ +3.19x faster │
│ QQuery 13 │  1324.34 / 1361.29 ±41.76 / 1431.55 ms │     359.72 / 372.63 ±9.92 / 389.34 ms │ +3.65x faster │
│ QQuery 14 │     912.34 / 942.13 ±25.67 / 982.70 ms │     283.16 / 284.77 ±1.22 / 286.20 ms │ +3.31x faster │
│ QQuery 15 │      285.47 / 292.55 ±5.97 / 300.50 ms │     283.44 / 285.76 ±3.00 / 291.57 ms │     no change │
│ QQuery 16 │  1370.63 / 1408.98 ±27.61 / 1441.17 ms │     612.03 / 628.65 ±9.92 / 642.96 ms │ +2.24x faster │
│ QQuery 17 │  1370.10 / 1392.89 ±25.81 / 1440.93 ms │     624.66 / 632.98 ±5.24 / 640.72 ms │ +2.20x faster │
│ QQuery 18 │  2010.07 / 2071.55 ±49.36 / 2155.50 ms │ 1223.49 / 1248.60 ±27.92 / 1303.11 ms │ +1.66x faster │
│ QQuery 19 │        26.77 / 37.75 ±13.00 / 54.86 ms │        27.22 / 29.10 ±2.85 / 34.77 ms │ +1.30x faster │
│ QQuery 20 │  1050.40 / 1064.20 ±11.21 / 1082.29 ms │     514.71 / 523.52 ±6.66 / 534.27 ms │ +2.03x faster │
│ QQuery 21 │   1354.57 / 1368.09 ±8.92 / 1380.05 ms │     506.21 / 512.77 ±4.38 / 519.41 ms │ +2.67x faster │
│ QQuery 22 │   3256.32 / 3264.49 ±9.83 / 3281.52 ms │     967.07 / 982.79 ±8.91 / 990.25 ms │ +3.32x faster │
│ QQuery 23 │ 9229.71 / 9465.67 ±149.27 / 9698.92 ms │ 6119.19 / 6168.67 ±27.89 / 6199.16 ms │ +1.53x faster │
│ QQuery 24 │        96.28 / 99.99 ±5.39 / 110.67 ms │        40.91 / 43.31 ±1.56 / 45.82 ms │ +2.31x faster │
│ QQuery 25 │      350.17 / 354.00 ±4.19 / 361.66 ms │     105.72 / 107.33 ±1.76 / 109.62 ms │ +3.30x faster │
│ QQuery 26 │         95.88 / 96.93 ±0.89 / 98.24 ms │        41.26 / 42.72 ±1.86 / 46.33 ms │ +2.27x faster │
│ QQuery 27 │  1136.97 / 1151.00 ±16.44 / 1183.10 ms │    508.22 / 523.25 ±15.38 / 552.00 ms │ +2.20x faster │
│ QQuery 28 │  2539.96 / 2567.15 ±19.19 / 2593.14 ms │ 2856.77 / 2900.17 ±30.05 / 2944.45 ms │  1.13x slower │
│ QQuery 29 │         38.06 / 38.38 ±0.28 / 38.88 ms │      38.88 / 54.85 ±23.71 / 100.80 ms │  1.43x slower │
│ QQuery 30 │      550.34 / 562.15 ±8.33 / 575.09 ms │     305.52 / 316.94 ±8.74 / 328.25 ms │ +1.77x faster │
│ QQuery 31 │     532.43 / 558.12 ±15.15 / 570.19 ms │     273.47 / 289.97 ±8.86 / 299.67 ms │ +1.92x faster │
│ QQuery 32 │  1096.22 / 1160.90 ±55.10 / 1256.46 ms │  955.33 / 1003.36 ±29.58 / 1036.73 ms │ +1.16x faster │
│ QQuery 33 │  3935.94 / 4054.48 ±78.15 / 4140.20 ms │ 1479.12 / 1504.45 ±32.54 / 1567.76 ms │ +2.69x faster │
│ QQuery 34 │  3970.33 / 4010.24 ±49.51 / 4105.93 ms │ 1468.01 / 1483.71 ±12.30 / 1502.20 ms │ +2.70x faster │
│ QQuery 35 │      304.82 / 315.34 ±8.33 / 328.01 ms │    290.80 / 347.04 ±70.46 / 470.42 ms │  1.10x slower │
│ QQuery 36 │      130.83 / 134.17 ±3.68 / 140.91 ms │        67.40 / 73.47 ±9.30 / 91.92 ms │ +1.83x faster │
│ QQuery 37 │       96.74 / 104.16 ±4.89 / 110.69 ms │        34.89 / 37.72 ±3.12 / 43.09 ms │ +2.76x faster │
│ QQuery 38 │         81.46 / 84.64 ±2.37 / 88.69 ms │       40.47 / 48.66 ±10.53 / 69.35 ms │ +1.74x faster │
│ QQuery 39 │     279.78 / 294.05 ±10.75 / 306.15 ms │    132.76 / 147.93 ±11.22 / 164.22 ms │ +1.99x faster │
│ QQuery 40 │         13.81 / 17.71 ±4.67 / 24.37 ms │        14.12 / 14.78 ±0.73 / 16.10 ms │ +1.20x faster │
│ QQuery 41 │         13.06 / 13.42 ±0.30 / 13.88 ms │        13.12 / 13.57 ±0.41 / 14.32 ms │     no change │
│ QQuery 42 │         12.64 / 13.06 ±0.25 / 13.36 ms │        12.92 / 14.11 ±1.88 / 17.83 ms │  1.08x slower │
└───────────┴────────────────────────────────────────┴───────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 41560.96ms │
│ Total Time (codex_bench-arrow-12008)   │ 22744.58ms │
│ Average Time (HEAD)                    │   966.53ms │
│ Average Time (codex_bench-arrow-12008) │   528.94ms │
│ Queries Faster                         │         26 │
│ Queries Slower                         │          6 │
│ Queries with No Change                 │         11 │
│ Queries with Failure                   │          0 │
└────────────────────────────────────────┴────────────┘

Memory Pool Peaks

Peak MemoryPool reservation per query — what DataFusion's accounting believes it reserved. Recorded only for benchmarks that write a results JSON (the dfbench suites, and the suites bench.sh runs through the Criterion SQL harness since #25644), and only under a memory limit: DATAFUSION_RUNTIME_MEMORY_LIMIT, or one the suite sets itself.

Base: 55a2794bc3465c88ff4bda8acd33820fc7c74f60 | Changed: 4b73f22a53c3cdec4c44da9eb5ae736ec4ac4bfb

clickbench_partitioned

Query Base Changed Change
Query 0 0 B 0 B 0.0%
Query 1 104 B 104 B +0.0%
Query 2 936 B 936 B +0.0%
Query 3 312 B 312 B +0.0%
Query 4 842.2 MiB 825.9 MiB -1.9%
Query 5 1.8 GiB 1.2 GiB -35.1%
Query 6 0 B 0 B 0.0%
Query 7 50.0 MiB 50.2 MiB +0.3%
Query 8 941.3 MiB 966.9 MiB +2.7%
Query 9 594.2 MiB 552.0 MiB -7.1%
Query 10 73.4 MiB 73.3 MiB -0.1%
Query 11 78.4 MiB 76.7 MiB -2.1%
Query 12 2.0 GiB 1.3 GiB -31.8%
Query 13 2.4 GiB 1.6 GiB -35.2%
Query 14 1.9 GiB 1.3 GiB -32.9%
Query 15 1.2 GiB 1.2 GiB +2.7%
Query 16 3.4 GiB 3.3 GiB -3.1%
Query 17 3.5 GiB 3.4 GiB -3.4%
Query 18 7.5 GiB 8.4 GiB +12.2%
Query 19 0 B 0 B 0.0%
Query 20 104 B 104 B +0.0%
Query 21 1.3 MiB 3.7 MiB +180.7%
Query 22 2.7 MiB 3.3 MiB +22.3%
Query 23 12.8 MiB 10.1 MiB -21.0%
Query 24 6.4 MiB 61.6 MiB +859.6%
Query 25 8.1 MiB 178.0 MiB +2087.3%
Query 26 7.8 MiB 62.3 MiB +697.3%
Query 27 2.5 MiB 2.5 MiB +0.0%
Query 28 2.2 GiB 1.8 GiB -20.6%
Query 29 624 B 624 B +0.0%
Query 30 682.5 MiB 681.7 MiB -0.1%
Query 31 1.5 GiB 1.5 GiB -3.1%
Query 32 7.1 GiB 7.1 GiB +0.1%
Query 33 12.5 GiB 8.3 GiB -32.9%
Query 34 14.8 GiB 7.9 GiB -46.8%
Query 35 659.6 MiB 664.8 MiB +0.8%
Query 36 504.7 MiB 109.5 MiB -78.3%
Query 37 8.4 MiB 7.3 MiB -13.0%
Query 38 5.3 MiB 5.1 MiB -3.5%
Query 39 910.4 MiB 297.0 MiB -67.4%
Query 40 2.5 MiB 2.2 MiB -12.6%
Query 41 3.1 MiB 3.1 MiB +0.0%
Query 42 1.6 MiB 1.7 MiB +6.2%

Pool accounting vs. process RSS

Max pool peak is the largest reservation any single query in the run reached; peak RSS covers the whole invocation, including data loading and allocator retention, and the two high-water marks need not coincide in time. The gap is therefore an upper bound on what the pool did not account for, not a measurement of it.

Benchmark Side Max pool peak Peak RSS Gap RSS / pool
clickbench_partitioned base (55a2794bc3465c88ff4bda8acd33820fc7c74f60) 14.8 GiB 15.8 GiB 1.1 GiB 1.1×
clickbench_partitioned changed (4b73f22a53c3cdec4c44da9eb5ae736ec4ac4bfb) 8.4 GiB 13.4 GiB 5.0 GiB 1.6×
Resource Usage

clickbench_partitioned — base (merge-base)

Metric Value
Wall time 210.0s
Peak memory 15.8 GiB
Avg memory 6.0 GiB
CPU user 1977.4s
CPU sys 138.0s
Peak spill 267.9 MiB

clickbench_partitioned — branch

Metric Value
Wall time 115.0s
Peak memory 13.4 GiB
Avg memory 4.3 GiB
CPU user 1175.2s
CPU sys 74.6s
Peak spill 0 B

File an issue against this benchmark runner

@Rich-T-kid Rich-T-kid mentioned this pull request Oct 31, 2026
Open
@Rich-T-kid

Rich-T-kid commented Oct 31, 2026

Copy link
Copy Markdown
Contributor Author

run benchmark clickbench_partitioned

env:
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
baseline:
  ref: "4b73f22a53c3cdec4c44da9eb5ae736ec4ac4bfb"
  env:
    DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "false"
changed:
  ref: "4b73f22a53c3cdec4c44da9eb5ae736ec4ac4bfb"
  env:
    DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"

@adriangbot

adriangbot commented Oct 31, 2026

Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c6545483252-4212-lxs86 6.12.105+ #1 SMP Sat Sep 19 07:55:25 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu) 
Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing 4b73f22 (4b73f22) to 4b73f22 diff

Run configuration 
run benchmark clickbench_partitioned
env:
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
baseline:
  ref: "4b73f22a53c3cdec4c44da9eb5ae736ec4ac4bfb"
  env:
    DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "false"
changed:
  ref: "4b73f22a53c3cdec4c44da9eb5ae736ec4ac4bfb"
  env:
    DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"

Results will be posted here when complete

File an issue against this benchmark runner

@adriangbot

adriangbot commented Oct 31, 2026

Copy link
Copy Markdown

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing 4b73f22 (4b73f22) to 4b73f22 diff

Run configuration 
run benchmark clickbench_partitioned
env:
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
baseline:
  ref: "4b73f22a53c3cdec4c44da9eb5ae736ec4ac4bfb"
  env:
    DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "false"
changed:
  ref: "4b73f22a53c3cdec4c44da9eb5ae736ec4ac4bfb"
  env:
    DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"
CPU Details (lscpu) 
Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected
Details 

Comparing HEAD and codex_bench-arrow-12008
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query     ┃       HEAD ┃ codex_bench-arrow-12008 ┃       Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ QQuery 0  │    1.19 ms │                 1.20 ms │    no change │
│ QQuery 1  │   11.40 ms │                11.60 ms │    no change │
│ QQuery 2  │   35.06 ms │                35.10 ms │    no change │
│ QQuery 3  │   31.21 ms │                30.89 ms │    no change │
│ QQuery 4  │  232.81 ms │               235.34 ms │    no change │
│ QQuery 5  │  270.81 ms │               271.23 ms │    no change │
│ QQuery 6  │    1.23 ms │                 1.24 ms │    no change │
│ QQuery 7  │   12.73 ms │                12.87 ms │    no change │
│ QQuery 8  │  321.21 ms │               325.52 ms │    no change │
│ QQuery 9  │  460.99 ms │               470.53 ms │    no change │
│ QQuery 10 │   63.44 ms │               200.18 ms │ 3.16x slower │
│ QQuery 11 │   74.92 ms │               210.87 ms │ 2.81x slower │
│ QQuery 12 │  258.80 ms │               259.94 ms │    no change │
│ QQuery 13 │  357.47 ms │               367.98 ms │    no change │
│ QQuery 14 │  272.92 ms │               280.30 ms │    no change │
│ QQuery 15 │  284.51 ms │               281.42 ms │    no change │
│ QQuery 16 │  619.08 ms │               610.07 ms │    no change │
│ QQuery 17 │  626.06 ms │               618.47 ms │    no change │
│ QQuery 18 │ 1202.54 ms │              1196.28 ms │    no change │
│ QQuery 19 │   27.62 ms │                26.95 ms │    no change │
│ QQuery 20 │  516.55 ms │               519.92 ms │    no change │
│ QQuery 21 │  508.75 ms │               509.36 ms │    no change │
│ QQuery 22 │  972.04 ms │               970.13 ms │    no change │
│ QQuery 23 │ 3004.19 ms │              6110.31 ms │ 2.03x slower │
│ QQuery 24 │   39.84 ms │                40.06 ms │    no change │
│ QQuery 25 │  104.60 ms │               104.48 ms │    no change │
│ QQuery 26 │   40.16 ms │                40.36 ms │    no change │
│ QQuery 27 │  511.97 ms │               514.54 ms │    no change │
│ QQuery 28 │ 2842.69 ms │              2871.02 ms │    no change │
│ QQuery 29 │   38.85 ms │                39.04 ms │    no change │
│ QQuery 30 │  299.99 ms │               303.22 ms │    no change │
│ QQuery 31 │  281.68 ms │               278.58 ms │    no change │
│ QQuery 32 │  923.67 ms │               937.01 ms │    no change │
│ QQuery 33 │ 1432.79 ms │              1447.98 ms │    no change │
│ QQuery 34 │ 1467.62 ms │              1450.91 ms │    no change │
│ QQuery 35 │  287.99 ms │               293.90 ms │    no change │
│ QQuery 36 │   66.54 ms │                66.63 ms │    no change │
│ QQuery 37 │   34.85 ms │                35.22 ms │    no change │
│ QQuery 38 │   41.16 ms │                42.34 ms │    no change │
│ QQuery 39 │  137.32 ms │               148.44 ms │ 1.08x slower │
│ QQuery 40 │   13.96 ms │                13.98 ms │    no change │
│ QQuery 41 │   13.41 ms │                13.31 ms │    no change │
│ QQuery 42 │   12.94 ms │                12.93 ms │    no change │
└───────────┴────────────┴─────────────────────────┴──────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 18759.56ms │
│ Total Time (codex_bench-arrow-12008)   │ 22211.66ms │
│ Average Time (HEAD)                    │   436.27ms │
│ Average Time (codex_bench-arrow-12008) │   516.55ms │
│ Queries Faster                         │          0 │
│ Queries Slower                         │          4 │
│ Queries with No Change                 │         39 │
│ Queries with Failure                   │          0 │
└────────────────────────────────────────┴────────────┘

Distribution per query (min / mean ±stddev / max):

Comparing HEAD and codex_bench-arrow-12008
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃                                  HEAD ┃               codex_bench-arrow-12008 ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0  │          1.19 / 4.13 ±5.81 / 15.75 ms │          1.20 / 4.16 ±5.87 / 15.90 ms │     no change │
│ QQuery 1  │        11.40 / 11.65 ±0.15 / 11.88 ms │        11.60 / 11.79 ±0.17 / 12.12 ms │     no change │
│ QQuery 2  │        35.06 / 35.22 ±0.10 / 35.33 ms │        35.10 / 35.37 ±0.30 / 35.93 ms │     no change │
│ QQuery 3  │        31.21 / 32.08 ±0.74 / 33.03 ms │        30.89 / 31.59 ±0.75 / 32.97 ms │     no change │
│ QQuery 4  │     232.81 / 235.99 ±1.62 / 237.33 ms │     235.34 / 238.75 ±2.57 / 242.06 ms │     no change │
│ QQuery 5  │     270.81 / 273.81 ±2.69 / 278.33 ms │     271.23 / 274.77 ±2.00 / 276.67 ms │     no change │
│ QQuery 6  │           1.23 / 1.39 ±0.23 / 1.83 ms │           1.24 / 1.45 ±0.22 / 1.85 ms │     no change │
│ QQuery 7  │        12.73 / 12.90 ±0.10 / 13.01 ms │        12.87 / 12.95 ±0.05 / 13.00 ms │     no change │
│ QQuery 8  │     321.21 / 328.68 ±6.41 / 340.06 ms │     325.52 / 330.81 ±5.44 / 340.82 ms │     no change │
│ QQuery 9  │     460.99 / 467.14 ±6.11 / 478.15 ms │     470.53 / 479.80 ±7.78 / 488.60 ms │     no change │
│ QQuery 10 │        63.44 / 64.26 ±0.74 / 65.51 ms │     200.18 / 203.66 ±5.75 / 215.05 ms │  3.17x slower │
│ QQuery 11 │        74.92 / 77.37 ±3.93 / 85.17 ms │     210.87 / 211.85 ±0.71 / 212.82 ms │  2.74x slower │
│ QQuery 12 │     258.80 / 262.93 ±3.99 / 269.68 ms │     259.94 / 269.82 ±6.96 / 277.56 ms │     no change │
│ QQuery 13 │     357.47 / 361.57 ±2.25 / 364.12 ms │     367.98 / 374.40 ±6.98 / 387.74 ms │     no change │
│ QQuery 14 │     272.92 / 280.80 ±7.86 / 294.87 ms │     280.30 / 287.95 ±7.69 / 299.49 ms │     no change │
│ QQuery 15 │     284.51 / 291.56 ±6.48 / 301.57 ms │     281.42 / 284.48 ±3.76 / 291.85 ms │     no change │
│ QQuery 16 │     619.08 / 625.00 ±6.39 / 636.69 ms │    610.07 / 625.64 ±12.12 / 643.96 ms │     no change │
│ QQuery 17 │     626.06 / 633.73 ±5.11 / 641.55 ms │     618.47 / 630.09 ±6.70 / 637.67 ms │     no change │
│ QQuery 18 │ 1202.54 / 1228.96 ±20.73 / 1259.49 ms │ 1196.28 / 1232.40 ±23.68 / 1269.12 ms │     no change │
│ QQuery 19 │        27.62 / 32.39 ±6.71 / 45.10 ms │       26.95 / 38.67 ±21.89 / 82.43 ms │  1.19x slower │
│ QQuery 20 │    516.55 / 524.02 ±11.00 / 545.90 ms │     519.92 / 526.52 ±4.84 / 534.13 ms │     no change │
│ QQuery 21 │     508.75 / 513.17 ±5.54 / 523.98 ms │     509.36 / 516.06 ±5.69 / 523.13 ms │     no change │
│ QQuery 22 │     972.04 / 983.94 ±7.00 / 993.88 ms │     970.13 / 986.13 ±9.52 / 998.60 ms │     no change │
│ QQuery 23 │ 3004.19 / 3037.54 ±18.78 / 3059.94 ms │ 6110.31 / 6165.90 ±48.27 / 6253.54 ms │  2.03x slower │
│ QQuery 24 │        39.84 / 40.40 ±0.37 / 40.88 ms │        40.06 / 42.46 ±3.30 / 48.98 ms │  1.05x slower │
│ QQuery 25 │     104.60 / 110.08 ±5.48 / 120.33 ms │     104.48 / 106.32 ±2.22 / 110.43 ms │     no change │
│ QQuery 26 │        40.16 / 40.73 ±0.52 / 41.35 ms │        40.36 / 41.01 ±0.53 / 41.65 ms │     no change │
│ QQuery 27 │     511.97 / 519.84 ±6.13 / 530.21 ms │    514.54 / 525.22 ±16.19 / 557.41 ms │     no change │
│ QQuery 28 │ 2842.69 / 2874.85 ±19.98 / 2898.29 ms │ 2871.02 / 2888.09 ±11.35 / 2906.33 ms │     no change │
│ QQuery 29 │       38.85 / 50.42 ±16.15 / 80.42 ms │       39.04 / 52.33 ±15.58 / 71.44 ms │     no change │
│ QQuery 30 │     299.99 / 313.42 ±9.40 / 328.49 ms │     303.22 / 313.06 ±7.59 / 324.92 ms │     no change │
│ QQuery 31 │    281.68 / 295.56 ±12.65 / 319.32 ms │    278.58 / 295.04 ±15.72 / 324.49 ms │     no change │
│ QQuery 32 │    923.67 / 938.29 ±15.98 / 963.82 ms │   937.01 / 998.79 ±56.48 / 1097.73 ms │  1.06x slower │
│ QQuery 33 │ 1432.79 / 1464.34 ±24.76 / 1507.23 ms │ 1447.98 / 1467.82 ±12.64 / 1481.09 ms │     no change │
│ QQuery 34 │ 1467.62 / 1514.08 ±50.13 / 1611.51 ms │ 1450.91 / 1490.17 ±27.28 / 1526.94 ms │     no change │
│ QQuery 35 │    287.99 / 348.47 ±65.60 / 471.40 ms │    293.90 / 314.12 ±23.62 / 357.07 ms │ +1.11x faster │
│ QQuery 36 │        66.54 / 70.97 ±4.08 / 77.19 ms │        66.63 / 70.53 ±3.04 / 75.25 ms │     no change │
│ QQuery 37 │        34.85 / 37.40 ±3.03 / 43.19 ms │        35.22 / 38.22 ±4.80 / 47.79 ms │     no change │
│ QQuery 38 │        41.16 / 43.90 ±1.94 / 46.55 ms │        42.34 / 46.28 ±2.57 / 48.50 ms │  1.05x slower │
│ QQuery 39 │    137.32 / 152.02 ±10.80 / 169.62 ms │     148.44 / 155.72 ±7.64 / 170.48 ms │     no change │
│ QQuery 40 │        13.96 / 16.04 ±2.81 / 21.47 ms │        13.98 / 14.25 ±0.28 / 14.74 ms │ +1.13x faster │
│ QQuery 41 │        13.41 / 13.58 ±0.17 / 13.87 ms │        13.31 / 17.17 ±4.91 / 25.31 ms │  1.26x slower │
│ QQuery 42 │        12.94 / 13.19 ±0.18 / 13.40 ms │        12.93 / 14.51 ±2.42 / 19.30 ms │  1.10x slower │
└───────────┴───────────────────────────────────────┴───────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 19177.81ms │
│ Total Time (codex_bench-arrow-12008)   │ 22666.10ms │
│ Average Time (HEAD)                    │   446.00ms │
│ Average Time (codex_bench-arrow-12008) │   527.12ms │
│ Queries Faster                         │          2 │
│ Queries Slower                         │          9 │
│ Queries with No Change                 │         32 │
│ Queries with Failure                   │          0 │
└────────────────────────────────────────┴────────────┘

Memory Pool Peaks

Peak MemoryPool reservation per query — what DataFusion's accounting believes it reserved. Recorded only for benchmarks that write a results JSON (the dfbench suites, and the suites bench.sh runs through the Criterion SQL harness since #25644), and only under a memory limit: DATAFUSION_RUNTIME_MEMORY_LIMIT, or one the suite sets itself.

Base: 4b73f22a53c3cdec4c44da9eb5ae736ec4ac4bfb | Changed: 4b73f22a53c3cdec4c44da9eb5ae736ec4ac4bfb

clickbench_partitioned

Query Base Changed Change
Query 0 0 B 0 B 0.0%
Query 1 104 B 104 B +0.0%
Query 2 936 B 936 B +0.0%
Query 3 312 B 312 B +0.0%
Query 4 827.2 MiB 842.4 MiB +1.8%
Query 5 1.2 GiB 1.3 GiB +5.5%
Query 6 0 B 0 B 0.0%
Query 7 50.2 MiB 50.2 MiB +0.0%
Query 8 966.6 MiB 925.3 MiB -4.3%
Query 9 594.2 MiB 594.2 MiB +0.0%
Query 10 109.5 MiB 73.6 MiB -32.8%
Query 11 125.5 MiB 81.5 MiB -35.0%
Query 12 1.3 GiB 1.3 GiB -0.4%
Query 13 1.6 GiB 1.6 GiB -1.0%
Query 14 1.3 GiB 1.3 GiB -2.8%
Query 15 1.3 GiB 1.2 GiB -2.6%
Query 16 3.4 GiB 3.4 GiB +0.2%
Query 17 3.5 GiB 3.4 GiB -0.5%
Query 18 8.6 GiB 8.4 GiB -2.7%
Query 19 0 B 0 B 0.0%
Query 20 104 B 104 B +0.0%
Query 21 3.7 MiB 3.7 MiB +0.0%
Query 22 3.3 MiB 3.7 MiB +10.6%
Query 23 28.6 MiB 10.1 MiB -64.6%
Query 24 61.6 MiB 60.3 MiB -2.0%
Query 25 177.1 MiB 167.4 MiB -5.5%
Query 26 61.3 MiB 59.5 MiB -2.9%
Query 27 2.5 MiB 2.3 MiB -7.6%
Query 28 1.8 GiB 1.8 GiB +2.6%
Query 29 624 B 624 B +0.0%
Query 30 678.7 MiB 678.8 MiB +0.0%
Query 31 1.5 GiB 1.6 GiB +6.5%
Query 32 7.1 GiB 7.1 GiB +0.1%
Query 33 7.9 GiB 7.4 GiB -5.2%
Query 34 7.9 GiB 7.8 GiB -0.5%
Query 35 659.8 MiB 656.2 MiB -0.5%
Query 36 124.4 MiB 122.9 MiB -1.2%
Query 37 8.4 MiB 6.6 MiB -21.2%
Query 38 5.0 MiB 5.4 MiB +6.6%
Query 39 296.4 MiB 297.8 MiB +0.5%
Query 40 2.1 MiB 2.3 MiB +5.9%
Query 41 3.1 MiB 3.1 MiB +0.0%
Query 42 1.6 MiB 1.7 MiB +6.6%

Pool accounting vs. process RSS

Max pool peak is the largest reservation any single query in the run reached; peak RSS covers the whole invocation, including data loading and allocator retention, and the two high-water marks need not coincide in time. The gap is therefore an upper bound on what the pool did not account for, not a measurement of it.

Benchmark Side Max pool peak Peak RSS Gap RSS / pool
clickbench_partitioned base (4b73f22a53c3cdec4c44da9eb5ae736ec4ac4bfb) 8.6 GiB 11.9 GiB 3.3 GiB 1.4×
clickbench_partitioned changed (4b73f22a53c3cdec4c44da9eb5ae736ec4ac4bfb) 8.4 GiB 12.7 GiB 4.3 GiB 1.5×
Resource Usage

clickbench_partitioned — base (merge-base)

Metric Value
Wall time 100.0s
Peak memory 11.9 GiB
Avg memory 4.6 GiB
CPU user 984.1s
CPU sys 68.5s
Peak spill 0 B

clickbench_partitioned — branch

Metric Value
Wall time 115.0s
Peak memory 12.7 GiB
Avg memory 4.2 GiB
CPU user 1174.0s
CPU sys 74.2s
Peak spill 0 B

File an issue against this benchmark runner

@Rich-T-kid

Rich-T-kid commented Oct 31, 2026

Copy link
Copy Markdown
Contributor Author

run benchmark clickbench_partitioned

env:
  DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
baseline:
  ref: "4b73f22a53c3cdec4c44da9eb5ae736ec4ac4bfb"
changed:
  ref: "9e040a92df90fe7ad6348413ee2c37d1c457a068"

@adriangbot

adriangbot commented Oct 31, 2026

Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c6548737280-4213-wmqjg 6.12.105+ #1 SMP Sat Sep 19 07:55:25 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu) 
Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing 9e040a9 (9e040a9) to 4b73f22 diff

Run configuration 
run benchmark clickbench_partitioned
env:
  DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
baseline:
  ref: "4b73f22a53c3cdec4c44da9eb5ae736ec4ac4bfb"
changed:
  ref: "9e040a92df90fe7ad6348413ee2c37d1c457a068"

Results will be posted here when complete

File an issue against this benchmark runner

@Rich-T-kid Rich-T-kid mentioned this pull request Oct 31, 2026
Draft
@adriangbot

adriangbot commented Oct 31, 2026

Copy link
Copy Markdown

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing 9e040a9 (9e040a9) to 4b73f22 diff

Run configuration 
run benchmark clickbench_partitioned
env:
  DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
baseline:
  ref: "4b73f22a53c3cdec4c44da9eb5ae736ec4ac4bfb"
changed:
  ref: "9e040a92df90fe7ad6348413ee2c37d1c457a068"
CPU Details (lscpu) 
Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected
Details 

Comparing HEAD and codex_bench-arrow-12008
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃       HEAD ┃ codex_bench-arrow-12008 ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0  │    1.16 ms │                 1.19 ms │     no change │
│ QQuery 1  │   11.61 ms │                11.86 ms │     no change │
│ QQuery 2  │   34.82 ms │                35.22 ms │     no change │
│ QQuery 3  │   30.64 ms │                30.53 ms │     no change │
│ QQuery 4  │  234.58 ms │               236.04 ms │     no change │
│ QQuery 5  │  273.52 ms │               272.17 ms │     no change │
│ QQuery 6  │    1.23 ms │                 1.22 ms │     no change │
│ QQuery 7  │   12.88 ms │                13.17 ms │     no change │
│ QQuery 8  │  325.78 ms │               329.53 ms │     no change │
│ QQuery 9  │  464.75 ms │               464.85 ms │     no change │
│ QQuery 10 │  198.86 ms │                65.91 ms │ +3.02x faster │
│ QQuery 11 │  211.87 ms │                76.34 ms │ +2.78x faster │
│ QQuery 12 │  261.97 ms │               264.15 ms │     no change │
│ QQuery 13 │  358.98 ms │               366.47 ms │     no change │
│ QQuery 14 │  277.03 ms │               280.16 ms │     no change │
│ QQuery 15 │  280.56 ms │               283.33 ms │     no change │
│ QQuery 16 │  613.28 ms │               614.67 ms │     no change │
│ QQuery 17 │  622.84 ms │               630.81 ms │     no change │
│ QQuery 18 │ 1213.77 ms │              1206.88 ms │     no change │
│ QQuery 19 │   27.25 ms │                27.56 ms │     no change │
│ QQuery 20 │  513.01 ms │               520.67 ms │     no change │
│ QQuery 21 │  504.71 ms │               508.19 ms │     no change │
│ QQuery 22 │  971.75 ms │               976.41 ms │     no change │
│ QQuery 23 │ 6146.32 ms │              2992.38 ms │ +2.05x faster │
│ QQuery 24 │   41.13 ms │                40.43 ms │     no change │
│ QQuery 25 │  105.64 ms │               106.88 ms │     no change │
│ QQuery 26 │   41.05 ms │                41.16 ms │     no change │
│ QQuery 27 │  504.97 ms │               512.20 ms │     no change │
│ QQuery 28 │ 2859.83 ms │              2864.69 ms │     no change │
│ QQuery 29 │   38.68 ms │                38.97 ms │     no change │
│ QQuery 30 │  306.74 ms │               305.95 ms │     no change │
│ QQuery 31 │  279.03 ms │               283.65 ms │     no change │
│ QQuery 32 │  936.07 ms │               931.89 ms │     no change │
│ QQuery 33 │ 1443.88 ms │              1449.54 ms │     no change │
│ QQuery 34 │ 1446.80 ms │              1466.82 ms │     no change │
│ QQuery 35 │  295.68 ms │               294.98 ms │     no change │
│ QQuery 36 │   67.03 ms │                68.67 ms │     no change │
│ QQuery 37 │   34.97 ms │                35.43 ms │     no change │
│ QQuery 38 │   40.61 ms │                40.13 ms │     no change │
│ QQuery 39 │  142.52 ms │               141.56 ms │     no change │
│ QQuery 40 │   14.48 ms │                14.20 ms │     no change │
│ QQuery 41 │   13.78 ms │                13.57 ms │     no change │
│ QQuery 42 │   13.10 ms │                13.11 ms │     no change │
└───────────┴────────────┴─────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 22219.19ms │
│ Total Time (codex_bench-arrow-12008)   │ 18873.52ms │
│ Average Time (HEAD)                    │   516.73ms │
│ Average Time (codex_bench-arrow-12008) │   438.92ms │
│ Queries Faster                         │          3 │
│ Queries Slower                         │          0 │
│ Queries with No Change                 │         40 │
│ Queries with Failure                   │          0 │
└────────────────────────────────────────┴────────────┘

Distribution per query (min / mean ±stddev / max):

Comparing HEAD and codex_bench-arrow-12008
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃                                  HEAD ┃               codex_bench-arrow-12008 ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0  │          1.16 / 4.18 ±5.90 / 15.97 ms │          1.19 / 4.21 ±5.90 / 16.01 ms │     no change │
│ QQuery 1  │        11.61 / 11.76 ±0.12 / 11.92 ms │        11.86 / 12.07 ±0.22 / 12.47 ms │     no change │
│ QQuery 2  │        34.82 / 35.29 ±0.46 / 36.15 ms │        35.22 / 35.33 ±0.17 / 35.66 ms │     no change │
│ QQuery 3  │        30.64 / 31.69 ±0.86 / 32.99 ms │        30.53 / 31.10 ±0.48 / 31.85 ms │     no change │
│ QQuery 4  │     234.58 / 239.37 ±3.55 / 244.82 ms │     236.04 / 238.97 ±2.58 / 242.52 ms │     no change │
│ QQuery 5  │     273.52 / 275.24 ±1.62 / 277.18 ms │     272.17 / 280.04 ±7.62 / 293.26 ms │     no change │
│ QQuery 6  │           1.23 / 1.47 ±0.40 / 2.27 ms │           1.22 / 1.39 ±0.25 / 1.89 ms │ +1.05x faster │
│ QQuery 7  │        12.88 / 12.97 ±0.07 / 13.07 ms │        13.17 / 13.28 ±0.15 / 13.57 ms │     no change │
│ QQuery 8  │     325.78 / 330.23 ±2.89 / 333.88 ms │     329.53 / 335.26 ±4.27 / 340.27 ms │     no change │
│ QQuery 9  │     464.75 / 467.98 ±3.38 / 474.34 ms │     464.85 / 478.62 ±7.86 / 485.61 ms │     no change │
│ QQuery 10 │     198.86 / 201.75 ±3.04 / 207.41 ms │        65.91 / 66.81 ±0.64 / 67.66 ms │ +3.02x faster │
│ QQuery 11 │     211.87 / 213.11 ±0.86 / 214.12 ms │        76.34 / 79.29 ±4.95 / 89.18 ms │ +2.69x faster │
│ QQuery 12 │    261.97 / 273.92 ±13.72 / 299.20 ms │     264.15 / 268.28 ±2.50 / 271.48 ms │     no change │
│ QQuery 13 │    358.98 / 377.47 ±14.78 / 400.77 ms │     366.47 / 380.38 ±8.30 / 392.37 ms │     no change │
│ QQuery 14 │     277.03 / 284.62 ±5.95 / 293.93 ms │     280.16 / 286.80 ±5.19 / 293.17 ms │     no change │
│ QQuery 15 │     280.56 / 286.47 ±3.55 / 290.57 ms │    283.33 / 294.37 ±16.80 / 326.96 ms │     no change │
│ QQuery 16 │     613.28 / 626.55 ±7.12 / 634.43 ms │     614.67 / 624.22 ±9.85 / 642.32 ms │     no change │
│ QQuery 17 │     622.84 / 628.05 ±4.52 / 636.01 ms │     630.81 / 636.98 ±6.40 / 648.73 ms │     no change │
│ QQuery 18 │ 1213.77 / 1242.39 ±20.26 / 1271.34 ms │ 1206.88 / 1228.59 ±17.01 / 1258.52 ms │     no change │
│ QQuery 19 │        27.25 / 29.95 ±4.68 / 39.31 ms │      27.56 / 47.57 ±32.55 / 111.70 ms │  1.59x slower │
│ QQuery 20 │    513.01 / 522.87 ±10.77 / 543.32 ms │     520.67 / 526.59 ±5.42 / 534.12 ms │     no change │
│ QQuery 21 │     504.71 / 513.69 ±5.53 / 519.89 ms │     508.19 / 517.61 ±6.72 / 528.17 ms │     no change │
│ QQuery 22 │     971.75 / 977.85 ±7.93 / 993.30 ms │     976.41 / 983.36 ±5.77 / 990.25 ms │     no change │
│ QQuery 23 │ 6146.32 / 6181.84 ±25.49 / 6224.78 ms │ 2992.38 / 3034.69 ±40.64 / 3103.73 ms │ +2.04x faster │
│ QQuery 24 │        41.13 / 42.50 ±1.21 / 44.64 ms │        40.43 / 41.60 ±1.16 / 43.08 ms │     no change │
│ QQuery 25 │     105.64 / 106.85 ±0.99 / 107.82 ms │     106.88 / 109.17 ±2.10 / 112.70 ms │     no change │
│ QQuery 26 │        41.05 / 42.60 ±1.87 / 46.09 ms │        41.16 / 41.85 ±0.69 / 43.12 ms │     no change │
│ QQuery 27 │    504.97 / 520.14 ±17.22 / 553.68 ms │    512.20 / 521.02 ±10.11 / 540.72 ms │     no change │
│ QQuery 28 │ 2859.83 / 2900.90 ±32.37 / 2956.81 ms │ 2864.69 / 2889.85 ±18.25 / 2915.56 ms │     no change │
│ QQuery 29 │       38.68 / 50.97 ±11.42 / 69.37 ms │       38.97 / 53.37 ±21.31 / 94.30 ms │     no change │
│ QQuery 30 │     306.74 / 314.53 ±6.92 / 326.24 ms │     305.95 / 313.19 ±4.32 / 318.96 ms │     no change │
│ QQuery 31 │     279.03 / 292.54 ±9.54 / 308.53 ms │     283.65 / 290.26 ±5.44 / 300.05 ms │     no change │
│ QQuery 32 │   936.07 / 967.83 ±24.58 / 1000.92 ms │   931.89 / 962.12 ±27.32 / 1004.25 ms │     no change │
│ QQuery 33 │ 1443.88 / 1479.46 ±21.76 / 1504.30 ms │ 1449.54 / 1466.95 ±12.22 / 1485.57 ms │     no change │
│ QQuery 34 │ 1446.80 / 1481.77 ±19.83 / 1508.60 ms │ 1466.82 / 1486.39 ±13.76 / 1505.79 ms │     no change │
│ QQuery 35 │    295.68 / 333.16 ±57.99 / 447.54 ms │    294.98 / 321.01 ±29.88 / 376.59 ms │     no change │
│ QQuery 36 │        67.03 / 73.05 ±7.33 / 86.28 ms │        68.67 / 72.79 ±4.12 / 80.15 ms │     no change │
│ QQuery 37 │        34.97 / 37.21 ±3.62 / 44.42 ms │        35.43 / 39.21 ±6.22 / 51.60 ms │  1.05x slower │
│ QQuery 38 │        40.61 / 42.09 ±1.34 / 43.93 ms │        40.13 / 43.64 ±3.03 / 49.13 ms │     no change │
│ QQuery 39 │     142.52 / 149.01 ±4.68 / 155.37 ms │    141.56 / 154.81 ±13.00 / 172.74 ms │     no change │
│ QQuery 40 │        14.48 / 15.97 ±2.87 / 21.71 ms │        14.20 / 14.52 ±0.24 / 14.95 ms │ +1.10x faster │
│ QQuery 41 │        13.78 / 15.48 ±3.03 / 21.54 ms │        13.57 / 13.78 ±0.21 / 14.05 ms │ +1.12x faster │
│ QQuery 42 │        13.10 / 13.44 ±0.25 / 13.84 ms │        13.11 / 15.69 ±4.57 / 24.83 ms │  1.17x slower │
└───────────┴───────────────────────────────────────┴───────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 22650.20ms │
│ Total Time (codex_bench-arrow-12008)   │ 19257.01ms │
│ Average Time (HEAD)                    │   526.75ms │
│ Average Time (codex_bench-arrow-12008) │   447.84ms │
│ Queries Faster                         │          6 │
│ Queries Slower                         │          3 │
│ Queries with No Change                 │         34 │
│ Queries with Failure                   │          0 │
└────────────────────────────────────────┴────────────┘

Memory Pool Peaks

Peak MemoryPool reservation per query — what DataFusion's accounting believes it reserved. Recorded only for benchmarks that write a results JSON (the dfbench suites, and the suites bench.sh runs through the Criterion SQL harness since #25644), and only under a memory limit: DATAFUSION_RUNTIME_MEMORY_LIMIT, or one the suite sets itself.

Base: 4b73f22a53c3cdec4c44da9eb5ae736ec4ac4bfb | Changed: 9e040a92df90fe7ad6348413ee2c37d1c457a068

clickbench_partitioned

Query Base Changed Change
Query 0 0 B 0 B 0.0%
Query 1 104 B 104 B +0.0%
Query 2 936 B 936 B +0.0%
Query 3 312 B 312 B +0.0%
Query 4 818.5 MiB 834.5 MiB +2.0%
Query 5 1.2 GiB 1.2 GiB -0.9%
Query 6 0 B 0 B 0.0%
Query 7 60.0 MiB 60.0 MiB -0.0%
Query 8 945.1 MiB 967.0 MiB +2.3%
Query 9 552.0 MiB 594.0 MiB +7.6%
Query 10 72.4 MiB 70.8 MiB -2.2%
Query 11 76.4 MiB 73.2 MiB -4.1%
Query 12 1.3 GiB 1.3 GiB +0.4%
Query 13 1.6 GiB 1.6 GiB -1.8%
Query 14 1.3 GiB 1.3 GiB -0.1%
Query 15 1.2 GiB 1.2 GiB -1.1%
Query 16 3.4 GiB 3.5 GiB +2.2%
Query 17 3.4 GiB 3.4 GiB +1.5%
Query 18 8.9 GiB 8.7 GiB -2.4%
Query 19 0 B 0 B 0.0%
Query 20 104 B 104 B +0.0%
Query 21 4.0 MiB 4.3 MiB +7.8%
Query 22 3.9 MiB 3.5 MiB -11.6%
Query 23 9.7 MiB 10.7 MiB +9.7%
Query 24 59.8 MiB 58.9 MiB -1.5%
Query 25 175.6 MiB 176.9 MiB +0.7%
Query 26 60.1 MiB 59.3 MiB -1.3%
Query 27 2.4 MiB 2.2 MiB -5.5%
Query 28 1.8 GiB 1.8 GiB -1.5%
Query 29 624 B 624 B +0.0%
Query 30 654.0 MiB 722.1 MiB +10.4%
Query 31 1.5 GiB 1.4 GiB -4.6%
Query 32 7.1 GiB 7.1 GiB +0.0%
Query 33 8.1 GiB 7.8 GiB -3.1%
Query 34 8.0 GiB 7.7 GiB -3.3%
Query 35 664.8 MiB 664.7 MiB -0.0%
Query 36 118.8 MiB 113.4 MiB -4.5%
Query 37 7.3 MiB 6.6 MiB -8.6%
Query 38 5.5 MiB 5.7 MiB +2.5%
Query 39 297.9 MiB 297.9 MiB +0.0%
Query 40 2.5 MiB 2.6 MiB +2.3%
Query 41 3.1 MiB 3.1 MiB +0.0%
Query 42 2.0 MiB 1.7 MiB -15.9%

Pool accounting vs. process RSS

Max pool peak is the largest reservation any single query in the run reached; peak RSS covers the whole invocation, including data loading and allocator retention, and the two high-water marks need not coincide in time. The gap is therefore an upper bound on what the pool did not account for, not a measurement of it.

Benchmark Side Max pool peak Peak RSS Gap RSS / pool
clickbench_partitioned base (4b73f22a53c3cdec4c44da9eb5ae736ec4ac4bfb) 8.9 GiB 12.6 GiB 3.8 GiB 1.4×
clickbench_partitioned changed (9e040a92df90fe7ad6348413ee2c37d1c457a068) 8.7 GiB 11.3 GiB 2.6 GiB 1.3×
Resource Usage

clickbench_partitioned — base (merge-base)

Metric Value
Wall time 115.0s
Peak memory 12.6 GiB
Avg memory 4.1 GiB
CPU user 1181.6s
CPU sys 70.1s
Peak spill 0 B

clickbench_partitioned — branch

Metric Value
Wall time 100.0s
Peak memory 11.3 GiB
Avg memory 4.4 GiB
CPU user 989.1s
CPU sys 67.7s
Peak spill 0 B

File an issue against this benchmark runner

@Rich-T-kid

Rich-T-kid commented Oct 31, 2026

Copy link
Copy Markdown
Contributor Author

run benchmark clickbench_partitioned

env:
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
baseline:
  ref: "9e040a92df90fe7ad6348413ee2c37d1c457a068"
  env:
    DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "false"
changed:
  ref: "9e040a92df90fe7ad6348413ee2c37d1c457a068"
  env:
    DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"

@adriangbot

adriangbot commented Oct 31, 2026

Copy link
Copy Markdown

🤖 Benchmark running (GKE) | trigger
Instance: c4a-highmem-16 (12 vCPU / 65 GiB) | Linux bench-c6548988155-4214-n52l6 6.12.105+ #1 SMP Sat Sep 19 07:55:25 UTC 2026 aarch64 GNU/Linux

CPU Details (lscpu) 
Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected

Comparing 9e040a9 (9e040a9) to 9e040a9 diff

Run configuration 
run benchmark clickbench_partitioned
env:
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
baseline:
  ref: "9e040a92df90fe7ad6348413ee2c37d1c457a068"
  env:
    DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "false"
changed:
  ref: "9e040a92df90fe7ad6348413ee2c37d1c457a068"
  env:
    DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"

Results will be posted here when complete

File an issue against this benchmark runner

@adriangbot

adriangbot commented Oct 31, 2026

Copy link
Copy Markdown

🤖 Benchmark completed (GKE) | trigger

Instance: c4a-highmem-16 (12 vCPU / 65 GiB)

Comparing 9e040a9 (9e040a9) to 9e040a9 diff

Run configuration 
run benchmark clickbench_partitioned
env:
  DATAFUSION_RUNTIME_MEMORY_LIMIT: "32G"
baseline:
  ref: "9e040a92df90fe7ad6348413ee2c37d1c457a068"
  env:
    DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "false"
changed:
  ref: "9e040a92df90fe7ad6348413ee2c37d1c457a068"
  env:
    DATAFUSION_EXECUTION_PARQUET_ENABLE_RLE_TO_DICTIONARY: "true"
CPU Details (lscpu) 
Architecture:                            aarch64
CPU op-mode(s):                          64-bit
Byte Order:                              Little Endian
CPU(s):                                  16
On-line CPU(s) list:                     0-15
Vendor ID:                               ARM
Model name:                              Neoverse-V2
Model:                                   1
Thread(s) per core:                      1
Core(s) per cluster:                     16
Socket(s):                               -
Cluster(s):                              1
Stepping:                                r0p1
BogoMIPS:                                2000.00
Flags:                                   fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm sb paca pacg dcpodp sve2 sveaes svepmull svebitperm svesha3 svesm4 flagm2 frint svei8mm svebf16 i8mm bf16 dgh rng bti
L1d cache:                               1 MiB (16 instances)
L1i cache:                               1 MiB (16 instances)
L2 cache:                                32 MiB (16 instances)
L3 cache:                                80 MiB (1 instance)
NUMA node(s):                            1
NUMA node0 CPU(s):                       0-15
Vulnerability Gather data sampling:      Not affected
Vulnerability Indirect target selection: Not affected
Vulnerability Itlb multihit:             Not affected
Vulnerability L1tf:                      Not affected
Vulnerability Mds:                       Not affected
Vulnerability Meltdown:                  Not affected
Vulnerability Mmio stale data:           Not affected
Vulnerability Reg file data sampling:    Not affected
Vulnerability Retbleed:                  Not affected
Vulnerability Spec rstack overflow:      Not affected
Vulnerability Spec store bypass:         Mitigation; Speculative Store Bypass disabled via prctl
Vulnerability Spectre v1:                Mitigation; __user pointer sanitization
Vulnerability Spectre v2:                Mitigation; CSV2, BHB
Vulnerability Srbds:                     Not affected
Vulnerability Tsa:                       Not affected
Vulnerability Tsx async abort:           Not affected
Vulnerability Vmscape:                   Not affected
Details 

Comparing HEAD and codex_bench-arrow-12008
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query     ┃       HEAD ┃ codex_bench-arrow-12008 ┃       Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ QQuery 0  │    1.20 ms │                 1.18 ms │    no change │
│ QQuery 1  │   11.58 ms │                11.82 ms │    no change │
│ QQuery 2  │   34.90 ms │                35.36 ms │    no change │
│ QQuery 3  │   31.20 ms │                31.97 ms │    no change │
│ QQuery 4  │  231.44 ms │               235.40 ms │    no change │
│ QQuery 5  │  271.46 ms │               277.15 ms │    no change │
│ QQuery 6  │    1.25 ms │                 1.23 ms │    no change │
│ QQuery 7  │   12.80 ms │                13.25 ms │    no change │
│ QQuery 8  │  327.43 ms │               326.49 ms │    no change │
│ QQuery 9  │  462.98 ms │               466.92 ms │    no change │
│ QQuery 10 │   63.76 ms │                66.27 ms │    no change │
│ QQuery 11 │   75.97 ms │                76.14 ms │    no change │
│ QQuery 12 │  262.02 ms │               261.09 ms │    no change │
│ QQuery 13 │  366.81 ms │               361.62 ms │    no change │
│ QQuery 14 │  276.40 ms │               278.78 ms │    no change │
│ QQuery 15 │  281.54 ms │               279.90 ms │    no change │
│ QQuery 16 │  608.47 ms │               611.96 ms │    no change │
│ QQuery 17 │  623.39 ms │               620.76 ms │    no change │
│ QQuery 18 │ 1197.92 ms │              1209.96 ms │    no change │
│ QQuery 19 │   27.49 ms │                27.68 ms │    no change │
│ QQuery 20 │  517.97 ms │               522.74 ms │    no change │
│ QQuery 21 │  506.91 ms │               508.64 ms │    no change │
│ QQuery 22 │  991.50 ms │               980.78 ms │    no change │
│ QQuery 23 │ 3071.07 ms │              3026.46 ms │    no change │
│ QQuery 24 │   40.71 ms │                40.92 ms │    no change │
│ QQuery 25 │  105.19 ms │               105.47 ms │    no change │
│ QQuery 26 │   41.00 ms │                41.59 ms │    no change │
│ QQuery 27 │  500.72 ms │               510.29 ms │    no change │
│ QQuery 28 │ 2853.07 ms │              2862.77 ms │    no change │
│ QQuery 29 │   38.94 ms │                39.76 ms │    no change │
│ QQuery 30 │  301.73 ms │               307.01 ms │    no change │
│ QQuery 31 │  280.96 ms │               284.89 ms │    no change │
│ QQuery 32 │  942.08 ms │               936.38 ms │    no change │
│ QQuery 33 │ 1439.20 ms │              1438.82 ms │    no change │
│ QQuery 34 │ 1451.65 ms │              1451.38 ms │    no change │
│ QQuery 35 │  288.37 ms │               295.06 ms │    no change │
│ QQuery 36 │   64.84 ms │                68.72 ms │ 1.06x slower │
│ QQuery 37 │   35.27 ms │                34.84 ms │    no change │
│ QQuery 38 │   40.16 ms │                39.33 ms │    no change │
│ QQuery 39 │  141.05 ms │               135.74 ms │    no change │
│ QQuery 40 │   13.93 ms │                13.90 ms │    no change │
│ QQuery 41 │   13.32 ms │                13.65 ms │    no change │
│ QQuery 42 │   13.06 ms │                12.89 ms │    no change │
└───────────┴────────────┴─────────────────────────┴──────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 18862.73ms │
│ Total Time (codex_bench-arrow-12008)   │ 18866.96ms │
│ Average Time (HEAD)                    │   438.67ms │
│ Average Time (codex_bench-arrow-12008) │   438.77ms │
│ Queries Faster                         │          0 │
│ Queries Slower                         │          1 │
│ Queries with No Change                 │         42 │
│ Queries with Failure                   │          0 │
└────────────────────────────────────────┴────────────┘

Distribution per query (min / mean ±stddev / max):

Comparing HEAD and codex_bench-arrow-12008
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query     ┃                                  HEAD ┃               codex_bench-arrow-12008 ┃        Change ┃
┡━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0  │          1.20 / 4.16 ±5.88 / 15.92 ms │          1.18 / 4.11 ±5.78 / 15.67 ms │     no change │
│ QQuery 1  │        11.58 / 11.83 ±0.13 / 11.97 ms │        11.82 / 12.01 ±0.23 / 12.37 ms │     no change │
│ QQuery 2  │        34.90 / 35.25 ±0.26 / 35.56 ms │        35.36 / 35.55 ±0.15 / 35.74 ms │     no change │
│ QQuery 3  │        31.20 / 32.73 ±1.05 / 34.29 ms │        31.97 / 32.58 ±0.38 / 32.98 ms │     no change │
│ QQuery 4  │     231.44 / 237.84 ±4.10 / 242.55 ms │     235.40 / 238.50 ±2.05 / 241.61 ms │     no change │
│ QQuery 5  │     271.46 / 275.24 ±4.97 / 284.21 ms │     277.15 / 279.87 ±2.67 / 283.84 ms │     no change │
│ QQuery 6  │           1.25 / 1.41 ±0.22 / 1.85 ms │           1.23 / 1.45 ±0.30 / 2.03 ms │     no change │
│ QQuery 7  │        12.80 / 12.96 ±0.12 / 13.17 ms │        13.25 / 13.31 ±0.03 / 13.35 ms │     no change │
│ QQuery 8  │     327.43 / 331.62 ±3.09 / 335.39 ms │     326.49 / 333.09 ±4.86 / 340.33 ms │     no change │
│ QQuery 9  │     462.98 / 470.84 ±5.52 / 477.96 ms │     466.92 / 475.77 ±8.70 / 492.01 ms │     no change │
│ QQuery 10 │        63.76 / 65.67 ±1.23 / 67.62 ms │      66.27 / 73.74 ±13.41 / 100.52 ms │  1.12x slower │
│ QQuery 11 │        75.97 / 76.48 ±0.43 / 77.09 ms │        76.14 / 76.77 ±0.59 / 77.57 ms │     no change │
│ QQuery 12 │     262.02 / 266.21 ±3.48 / 270.40 ms │     261.09 / 267.80 ±5.89 / 275.24 ms │     no change │
│ QQuery 13 │     366.81 / 375.31 ±8.25 / 386.96 ms │    361.62 / 375.94 ±13.45 / 400.05 ms │     no change │
│ QQuery 14 │     276.40 / 278.77 ±2.26 / 283.10 ms │     278.78 / 287.77 ±5.56 / 295.19 ms │     no change │
│ QQuery 15 │     281.54 / 287.86 ±7.91 / 302.68 ms │     279.90 / 286.36 ±5.36 / 295.10 ms │     no change │
│ QQuery 16 │    608.47 / 627.00 ±11.61 / 644.38 ms │    611.96 / 623.99 ±10.34 / 639.84 ms │     no change │
│ QQuery 17 │     623.39 / 629.40 ±6.59 / 639.26 ms │     620.76 / 626.46 ±5.97 / 636.71 ms │     no change │
│ QQuery 18 │ 1197.92 / 1228.93 ±18.95 / 1255.73 ms │  1209.96 / 1224.44 ±9.58 / 1238.96 ms │     no change │
│ QQuery 19 │        27.49 / 31.46 ±5.03 / 40.17 ms │       27.68 / 42.11 ±19.14 / 75.55 ms │  1.34x slower │
│ QQuery 20 │     517.97 / 525.30 ±8.20 / 541.29 ms │     522.74 / 526.21 ±4.13 / 534.23 ms │     no change │
│ QQuery 21 │     506.91 / 516.95 ±5.63 / 523.31 ms │     508.64 / 521.82 ±7.65 / 529.14 ms │     no change │
│ QQuery 22 │    991.50 / 999.31 ±9.04 / 1014.11 ms │    980.78 / 990.16 ±9.11 / 1004.15 ms │     no change │
│ QQuery 23 │ 3071.07 / 3092.50 ±13.67 / 3108.46 ms │ 3026.46 / 3065.27 ±20.72 / 3082.56 ms │     no change │
│ QQuery 24 │        40.71 / 43.20 ±1.82 / 45.23 ms │        40.92 / 44.89 ±4.60 / 53.50 ms │     no change │
│ QQuery 25 │     105.19 / 106.59 ±1.26 / 108.24 ms │     105.47 / 106.95 ±1.16 / 108.43 ms │     no change │
│ QQuery 26 │        41.00 / 42.66 ±2.09 / 46.73 ms │        41.59 / 42.49 ±1.07 / 44.55 ms │     no change │
│ QQuery 27 │     500.72 / 514.22 ±9.86 / 531.24 ms │     510.29 / 526.69 ±8.32 / 533.10 ms │     no change │
│ QQuery 28 │ 2853.07 / 2882.62 ±19.67 / 2906.57 ms │ 2862.77 / 2901.52 ±22.56 / 2921.41 ms │     no change │
│ QQuery 29 │        38.94 / 43.39 ±5.56 / 53.11 ms │        39.76 / 42.91 ±5.02 / 52.92 ms │     no change │
│ QQuery 30 │    301.73 / 313.99 ±11.04 / 332.39 ms │     307.01 / 310.98 ±4.01 / 318.07 ms │     no change │
│ QQuery 31 │     280.96 / 285.64 ±4.39 / 291.76 ms │     284.89 / 291.04 ±6.74 / 303.49 ms │     no change │
│ QQuery 32 │    942.08 / 965.47 ±18.72 / 984.54 ms │   936.38 / 981.72 ±38.33 / 1040.28 ms │     no change │
│ QQuery 33 │ 1439.20 / 1485.93 ±37.14 / 1552.44 ms │ 1438.82 / 1468.25 ±22.87 / 1497.39 ms │     no change │
│ QQuery 34 │ 1451.65 / 1517.99 ±55.57 / 1592.73 ms │ 1451.38 / 1487.54 ±21.11 / 1514.29 ms │     no change │
│ QQuery 35 │    288.37 / 312.82 ±26.84 / 364.36 ms │    295.06 / 312.95 ±12.24 / 327.92 ms │     no change │
│ QQuery 36 │        64.84 / 66.83 ±1.33 / 68.61 ms │        68.72 / 72.52 ±5.11 / 82.49 ms │  1.09x slower │
│ QQuery 37 │        35.27 / 38.65 ±4.12 / 44.37 ms │        34.84 / 36.79 ±2.79 / 42.21 ms │     no change │
│ QQuery 38 │        40.16 / 45.67 ±3.65 / 50.59 ms │        39.33 / 42.73 ±3.38 / 47.40 ms │ +1.07x faster │
│ QQuery 39 │     141.05 / 153.47 ±7.97 / 165.95 ms │    135.74 / 149.06 ±13.87 / 167.88 ms │     no change │
│ QQuery 40 │        13.93 / 15.28 ±2.57 / 20.41 ms │        13.90 / 15.97 ±2.83 / 21.54 ms │     no change │
│ QQuery 41 │        13.32 / 17.77 ±5.90 / 29.07 ms │        13.65 / 14.71 ±1.04 / 16.16 ms │ +1.21x faster │
│ QQuery 42 │        13.06 / 15.88 ±5.03 / 25.93 ms │        12.89 / 14.02 ±2.05 / 18.11 ms │ +1.13x faster │
└───────────┴───────────────────────────────────────┴───────────────────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 19283.08ms │
│ Total Time (codex_bench-arrow-12008)   │ 19278.82ms │
│ Average Time (HEAD)                    │   448.44ms │
│ Average Time (codex_bench-arrow-12008) │   448.34ms │
│ Queries Faster                         │          3 │
│ Queries Slower                         │          3 │
│ Queries with No Change                 │         37 │
│ Queries with Failure                   │          0 │
└────────────────────────────────────────┴────────────┘

Memory Pool Peaks

Peak MemoryPool reservation per query — what DataFusion's accounting believes it reserved. Recorded only for benchmarks that write a results JSON (the dfbench suites, and the suites bench.sh runs through the Criterion SQL harness since #25644), and only under a memory limit: DATAFUSION_RUNTIME_MEMORY_LIMIT, or one the suite sets itself.

Base: 9e040a92df90fe7ad6348413ee2c37d1c457a068 | Changed: 9e040a92df90fe7ad6348413ee2c37d1c457a068

clickbench_partitioned

Query Base Changed Change
Query 0 0 B 0 B 0.0%
Query 1 104 B 104 B +0.0%
Query 2 936 B 936 B +0.0%
Query 3 312 B 312 B +0.0%
Query 4 827.9 MiB 829.1 MiB +0.1%
Query 5 1.2 GiB 1.2 GiB -2.7%
Query 6 0 B 0 B 0.0%
Query 7 40.4 MiB 40.3 MiB -0.2%
Query 8 971.1 MiB 971.3 MiB +0.0%
Query 9 594.2 MiB 594.0 MiB -0.0%
Query 10 114.0 MiB 70.7 MiB -38.0%
Query 11 118.2 MiB 73.5 MiB -37.8%
Query 12 1.4 GiB 1.4 GiB -2.2%
Query 13 1.6 GiB 1.6 GiB +0.8%
Query 14 1.3 GiB 1.3 GiB -0.3%
Query 15 1.2 GiB 1.2 GiB +3.9%
Query 16 3.4 GiB 3.4 GiB +0.1%
Query 17 3.4 GiB 3.4 GiB -0.4%
Query 18 8.5 GiB 8.2 GiB -3.6%
Query 19 0 B 0 B 0.0%
Query 20 104 B 104 B +0.0%
Query 21 3.7 MiB 4.0 MiB +8.3%
Query 22 3.5 MiB 3.0 MiB -15.5%
Query 23 29.6 MiB 9.2 MiB -68.8%
Query 24 58.8 MiB 60.1 MiB +2.2%
Query 25 167.5 MiB 176.1 MiB +5.1%
Query 26 59.9 MiB 64.1 MiB +7.0%
Query 27 2.2 MiB 2.4 MiB +5.2%
Query 28 1.8 GiB 1.8 GiB +0.7%
Query 29 624 B 624 B +0.0%
Query 30 677.6 MiB 724.0 MiB +6.9%
Query 31 1.5 GiB 1.5 GiB +0.0%
Query 32 7.1 GiB 7.1 GiB +0.2%
Query 33 8.0 GiB 8.0 GiB -0.2%
Query 34 7.9 GiB 7.7 GiB -2.0%
Query 35 660.8 MiB 664.3 MiB +0.5%
Query 36 107.9 MiB 116.6 MiB +8.0%
Query 37 6.6 MiB 7.3 MiB +9.4%
Query 38 5.7 MiB 5.2 MiB -8.7%
Query 39 298.2 MiB 297.9 MiB -0.1%
Query 40 2.1 MiB 2.3 MiB +11.4%
Query 41 3.1 MiB 3.1 MiB +0.1%
Query 42 1.6 MiB 1.8 MiB +13.1%

Pool accounting vs. process RSS

Max pool peak is the largest reservation any single query in the run reached; peak RSS covers the whole invocation, including data loading and allocator retention, and the two high-water marks need not coincide in time. The gap is therefore an upper bound on what the pool did not account for, not a measurement of it.

Benchmark Side Max pool peak Peak RSS Gap RSS / pool
clickbench_partitioned base (9e040a92df90fe7ad6348413ee2c37d1c457a068) 8.5 GiB 11.6 GiB 3.1 GiB 1.4×
clickbench_partitioned changed (9e040a92df90fe7ad6348413ee2c37d1c457a068) 8.2 GiB 11.3 GiB 3.2 GiB 1.4×
Resource Usage

clickbench_partitioned — base (merge-base)

Metric Value
Wall time 100.0s
Peak memory 11.6 GiB
Avg memory 4.6 GiB
CPU user 987.4s
CPU sys 68.9s
Peak spill 0 B

clickbench_partitioned — branch

Metric Value
Wall time 100.0s
Peak memory 11.3 GiB
Avg memory 4.4 GiB
CPU user 983.7s
CPU sys 69.7s
Peak spill 0 B

File an issue against this benchmark runner

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

common Related to common crate datasource Changes to the datasource crate documentation Improvements or additions to documentation physical-plan Changes to the physical-plan crate sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Rich-T-kid @adriangbot