Benchmarks

Measured, not asserted

STRESS (Standardized Residual Sum of Squares, lower is better) on primary-source psychophysical data and on COMBVD. Every number on this page comes from the scripts in the repository, with the HelmLab comparison run by the same harness.

10,486 color pairs 6 psychophysical + 1 reference

18.18 Mean7 STRESS AxiomLab Power-Riemannian

35.2% lower than CIEDE2000 Mean7: 28.04 → 18.18

2.3e-15 round-trip error sRGB XYZ, closed form

Calibration domain decides the winner

AxiomLab is calibrated on the 7-dataset suite below; HelmLab is trained on COMBVD. Each wins in its own domain. Reporting a single pooled number would hide that, so both protocols are shown.

AxiomLab's suite · 10,486 pairs

18.18

AxiomLab Mean7

35.00

HelmLab Mean7

COMBVD · 3,813 pairs

22.73

HelmLab raw (22.48 CAT)

33.72

AxiomLab raw (33.47 CAT)

STRESS by dataset

AxiomLab's 6 psychophysical datasets plus the CIE TC 1-47 analytical verification set. Lower is better.

Method	MacAdam 1974	OSA-UCS Atlas	MacAdam 1942	Munsell Value	Munsell Chroma	Munsell Hue	CIE TC 1-47	Mean6	Mean7
AxiomLab (Power-Riemannian)	15.93	11.49	24.19	7.67	28.45	24.44	15.10	18.70	18.18
HelmLab 1.0 MetricSpace	19.51	20.31	65.57	15.70	38.33	35.07	50.48	32.41	35.00
CIEDE2000	22.13	27.95	40.87	21.47	49.18	34.62	0.07	32.70	28.04
OKLab (Euclidean)	32.72	26.21	38.17	14.27	43.74	32.41	45.73	31.25	33.32
CAM16-UCS	18.70	18.16	31.39	13.08	42.50	33.75	17.84	26.26	25.06
IPT (Euclidean)	21.27	12.39	38.49	19.68	34.85	30.93	38.12	26.27	27.96

Mean7 STRESS by method

AxiomLab (Power-Riemannian) 18.18
AxiomLab (Homogeneous Riemannian) 20.65
CAM16-UCS 25.06
IPT (Euclidean) 27.96
CIEDE2000 28.04
OKLab (Euclidean) 33.32
HelmLab 1.0 MetricSpace 35.00

Ablation (AxiomLab)

Mean7 STRESS. Every tensor component earns its place.

AxiomLab v2.1 (Power-Riemannian) 18.18
Isotropic Chroma/Hue (S_C = S_H) 18.57
No Blue-Tilt (R_T = 0) 19.02
Homogeneous Riemannian (gamma_d = 1.0) 20.65
Pure Euclidean Space (G = I) 21.62

COMBVD (HelmLab's corpus)

Evaluated raw and after Bradford CAT to D65, matching HelmLab's published protocol. Our harness reproduces HelmLab's 22.73 / 22.48 and CIEDE2000's 29.18 to two decimals.

Method	raw	Bradford CAT
AxiomLab (Power-Riemannian)	33.72	33.47
AxiomLab (Homogeneous Riemannian)	36.13	35.79
HelmLab 1.0 MetricSpace	22.73	22.48
CIEDE2000	29.18	29.13
OKLab (Euclidean)	47.46	47.35

Gradient step uniformity

Coefficient of variation of 16-step hex ramps, measured with two metrics. 8-bit quantization sets a floor of roughly 1–5%, so treat these as delivered-output numbers, not ideal-path numbers.

Pair	AxiomLab	HelmLab	OKLab	Linear sRGB
#0000ff → #ffffff	22.88%	2.36%	18.14%	36.97%
#ff0000 → #00ff00	12.73%	2.64%	22.12%	48.18%
#0000ff → #ffff00	28.19%	2.84%	20.26%	65.99%
#00ffff → #ff00ff	25.61%	3.44%	36.18%	36.05%
#ff6b00 → #008080	36.81%	2.66%	37.93%	29.52%
#000000 → #ffffff	21.08%	2.38%	40.89%	21.67%

Each space wins the column measured in its own metric. HelmLab's gradient applies CIEDE2000 arc-length reparameterization, so the CIEDE2000 column favours it by construction; AxiomLab's arc length is measured in its own homogeneous metric.

Live speed test

20,000 XYZ conversions and 10,000 distances, in this browser.

Results appear here. Timings are indicative and depend on the device.

What was actually run

python3 test_axiomlab.py — 8/8 pass
node test_axiomlab.js — pass
python3 build_datasets.py — 7 datasets verified
python3 optimize_and_verify.py — full suite
python3 benchmark_helmlab.py — this page
Python ↔ JS parity: max 4.4e-16 Lab, 1.1e-15 ΔE

Caveats

AxiomLab's 7-dataset suite is AxiomLab's calibration corpus; HelmLab is evaluated there out-of-domain.
COMBVD is HelmLab's training corpus; AxiomLab is evaluated there out-of-domain.
Gradient CV is measured on 16-step 8-bit hex ramps, so quantization sets a floor of roughly 1-5%.
HelmLab's gradient() applies CIEDE2000 arc-length reparameterization, so it is expected to win the CIEDE2000 CV column.