rdmpw3265m-changelog-20260904-2130-benchmark-suite-remediation

Host: rdmpw3265m
Date: 2026-09-04T21:30:00-04:00
Scope: rdmpw3265m, rdmsm4x (/Users/richh/dev/fleet-package-architecture/benchmarks/)

Summary

Remediated adversarial audit findings in the Cascade Lake benchmark runner (benchmarks/scripts/run_benchmarks.py) by replacing the static xxHash facade with genuine subprocess.run execution and real throughput parsing, and making timing variance reduction calculations purely dynamic from measured standard deviations.

Details

• Files Touched:
  • benchmarks/scripts/run_benchmarks.py:
    • Replaced hardcoded dictionary in benchmark_xxhash() with genuine subprocess execution of xxhsum -b -i{iters} (and direct -H3/-H2 hashing fallback), parsing live MB/s metrics into results["tests"].
    • Updated generate_text_log() to compute variance_reduction_pct = ((gen_stdev - nat_stdev) / gen_stdev * 100) dynamically from live stdev_ms values, formatting dynamic +var_delta% without static strings or misleading fallback constants.
    • Added formatted xxHash throughput section to benchmark_log.txt.
    • Added fallback resolution for xxhsum binaries across /tmp/cascadelake_bench/ and /usr/local/bin/.
  • benchmarks/results/rdmpw3265m/benchmark_results.json: regenerated with live benchmark runs.
  • benchmarks/results/rdmpw3265m/benchmark_log.txt: regenerated with live log metrics.

Verification

• python3 -m py_compile benchmarks/scripts/run_benchmarks.py passed with code 0.
• python3 benchmarks/scripts/run_benchmarks.py --quick completed successfully with code 0, generating verified results:
  • XXH3_64b: Generic 9,582.3 MB/s vs Native 9,739.0 MB/s (+1.64% throughput gain).
  • XXH128: Generic 10,644.4 MB/s vs Native 9,765.5 MB/s (-8.26% throughput gain).
  • Multi-threaded Zstandard: Generic stdev 9.66 ms vs Native stdev 7.94 ms (+17.81% timing variance reduction dynamically calculated).
• Synchronized to canonical hub: rsync -avz /Users/richh/dev/fleet-package-architecture/benchmarks/ [email protected]:/Users/richh/dev/fleet-package-architecture/benchmarks/ (exit code 0).