Validation Record
What every statistic in Stratum is checked against — and where Stratum deliberately differs from other packages.
Every analysis is verified against values computed outside Stratum: by R, SciPy, NumPy, scikit-learn, or the certified NIST Statistical Reference Datasets. Those reference values are committed to the source tree and asserted on every build, so a result cannot quietly stop matching them.
We do not claim that Stratum “matches R”. That claim is not well
defined. R's own functions disagree with each other in places — quantile()
and boxplot() use different quartiles — and the same rule names mean
different things in different packages. What is published here instead is the specific
claim, analysis by analysis, naming the tool and the function, together with every
place we chose differently and why.
48 analyses. 19 checked against an outside tool · 12 partly checked · 3 pinned by a published formula · 14 no statistic of its own. This page is generated from the same ledger the test suite emits, so it cannot describe evidence that is not there. Last generated 2026-08-31. Reference tool versions: ca (R package) 0.71.1, mpmath 1.4.1, numpy 2.4.6, pingouin 0.6.1, R 4.6.1, scikit-learn 1.9.0, scipy 1.16.3, statsmodels 0.14.6.
How to read this
- Checked against an outside tool (19)
- Committed reference values from a named function of a named package at a recorded version, asserted on every build.
- Partly checked (12)
- Some published outputs are verified and others are not — and the entry says which. A fitted random forest cannot be matched tree-for-tree against another library, so its diagnostics are checked and the model itself rests on invariants.
- Pinned by a published formula (3)
- No package computes the quantity, so the published formula is transcribed independently twice and the two must agree.
- No statistic of its own (14)
- The chart draws values it was handed. A scatter plot has nothing to verify beyond the numbers given to it.
“Partly checked” is not a disclaimer added after the fact — it names work that is still outstanding, deliberately, because a verification claim later found to be wrong costs more than one never made.
Deliberate divergences
If you reconcile a Stratum result against another package and the numbers differ slightly, these are the likely reasons. Each is a documented choice.
- Quartiles are the Type-7 quantile — R's
quantile()default and NumPy's — not Tukey's hinges, which R's ownboxplot()andfivenum()draw. One definition serves the box, the percentile columns, thequantile()formula function and the MAD, so a quartile read off a box plot is the same number the report prints. Because the fences derive from the quartiles, the whiskers and the outlier set move with them. - Silverman's bandwidth uses the exact constant 1.3489795 where R's
bw.nrd0()uses the rounded 1.34 — identical while the standard deviation is the smaller term, about 0.67% apart when the IQR is. - Scott's bandwidth uses the classical 3.49, as NumPy does; R's
nclass.scott()uses 3.5. - Sturges' bin count matches R's
nclass.Sturges(); NumPy's'sturges'differs whenever log₂ n is not a whole number. Bin counts are additionally held to 5–200 (Sturges) and 1–500 (width rules) for legibility — limits no package has. - Q-Q plotting positions are (i − ½)/n; R's
ppoints()switches to a different constant at ten points or fewer. - r² for power and exponential trends is measured on the original
scale against the back-transformed curve you can see. R's
summary()reports the log-scale r² for the same model — usually higher, and describing a curve that is never drawn. - Control-chart σ̂ uses the published table value d₂ = 1.128, as SPC practice expects, not the exact 2/√π — a difference of about 0.03%.
- Correlation deletes pairwise; every other multi-variable analysis deletes listwise.
Analysis by analysis
| Analysis | Status | Checked against |
|---|---|---|
| Summary | Verified |
|
| Histogram | Verified |
|
| Heatmap/HexBin | Verified |
|
| Mosaic | Verified |
|
| Box-Whisker | Verified |
|
| Violin | Verified |
|
| Density | Verified |
|
| ECDF | Verified |
|
| Correlation | Verified |
|
| One-Way ANOVA | Verified |
|
| Two-Way ANOVA | Verified |
|
| MANOVA | Verified |
|
| Two-Sample | Verified |
|
| Differences | Verified |
|
| Contingency Table | Verified |
|
| Principal Components | Verified |
|
| Linear Regression | Verified |
|
| Process Capability | Verified |
|
| Regression Plot | Verified |
|
| Q-Q | Partly verified |
Not yet covered: The CONFIDENCE BAND (QQConfidenceBand) has no fixture — only the points and the theoretical axes are covered. |
| Trend | Partly verified |
Not yet covered: The non-parametric smoothers — LOWESS, kernel smoother and moving average — have no fixture. Their implementations differ in detail between packages, so pinning one spelling of LOWESS would pin R's choices rather than a shared definition; closing this means deciding which definition Stratum claims first. |
| Contour Plot | Partly verified |
Not yet covered: The blur is an APPROXIMATION to a 2-D KDE, so the surface is checked for shape against the exact estimator rather than value-for-value — the bound is a recorded measurement, not an equality. The value and aggregate modes (with a Z column) and their 2% masking threshold have no fixture. Note this chart does NOT share KernelDensityEngine: it uses the 2-D n^(-1/6) exponent where Density uses n^(-1/5), so it inherits nothing from that fixture. |
| Decision Tree | Partly verified |
Not yet covered: The DIAGNOSTICS (ROC, AUC, calibration binning, partial dependence) are pinned; the fitted TREE is not. A single tree can be matched to sklearn exactly given the same split rule and deterministic tie-breaking, so this is a gap to close, not an impossibility. |
| Random Forest | Partly verified |
Not yet covered: Diagnostics only. An ensemble cannot be matched tree-for-tree against sklearn — the RNG streams and split tie-breaking differ by construction — so the model itself needs invariants (OOB convergence, importance monotonicity) rather than a value comparison. |
| Boosted Trees | Partly verified |
Not yet covered: Diagnostics only — same reasoning as Random Forest. |
| Correspondence Analysis | Partly verified |
Not yet covered: The DISPLAY CAP is not covered. Above StratumLimits.maxReportGroups the engine pools the tail into 'Other' and decomposes the POOLED grid, which moves both the inertia and the degrees of freedom; every fixture table is well under the cap. Row/column contributions and quality (cos²) have no fixture yet either. |
| Clustering | Partly verified |
Not yet covered: The k-means PARTITION is compared to scikit-learn only where the optimum is unambiguous. On ambiguous data, and under the Manhattan cost (which sklearn's KMeans does not implement), the evidence is invariants rather than an outside implementation — because no outside implementation of that estimator exists to compare against, not because the check was skipped. Elbow and silhouette have no fixture yet. |
| I-MR Chart | Partly verified |
Not yet covered: The unbiasing CONSTANTS are pinned to first principles; the Nelson / Western Electric run rules are not, and they are a documented convention rather than a computed quantity. |
| X-bar Chart | Partly verified |
Not yet covered: Constants pinned; run rules are convention. Same as I-MR. |
| P Chart | Partly verified |
Not yet covered: Attribute-chart limits are not yet cross-checked against an outside tool. |
| C Chart | Partly verified |
Not yet covered: Attribute-chart limits are not yet cross-checked against an outside tool. |
| Pareto | By formula | ranking and a running total are exact arithmetic, so there is no outside tool to disagree with. What is pinned is falsifiable and was: the tie-break is the CATEGORY LABEL (so equal frequencies order deterministically rather than by dictionary order), and topN truncation does NOT renormalise — the cumulative line runs against the FULL total and stops short of 100%. Renormalising it trips 42 assertions. |
| CUSUM Chart | By formula | Montgomery's CUSUM recursion — C+ = max(0, C+ + z - k), C- = max(0, C- - z - k), FIR headstart h/2 — transcribed INDEPENDENTLY in R and numpy and asserted equal before either was recorded. No qcc/spc is installed, so this is the published formula rather than a package's implementation; installing either upgrades this to external. The sigma it standardises by is MRbar/d2, which shared_estimators pins to first principles. NOT covered: the ARL. |
| EWMA Chart | By formula | Montgomery's EWMA recursion with the TIME-VARYING limit width L*sigma*sqrt(lambda/(2-lambda)*(1-(1-lambda)^(2i))), transcribed independently in R and numpy. The test additionally asserts the band widens monotonically, so substituting the asymptotic width fails even where the two happen to be close. Same qcc/spc caveat as CUSUM. |
| Data | No statistic | The data sheet. Displays stored values. |
| Dot/Strip | No statistic | Plots raw values along one axis. |
| Pie | No statistic | Shares of a supplied total. |
| Treemap | No statistic | Areas proportional to a supplied value. |
| Bar | No statistic | Plots the aggregate the control bar selects; the aggregates themselves are Summary's, and covered there. |
| Dot Plot | No statistic | Plots raw values along one axis. |
| Dumbbell | No statistic | Draws two supplied values per category. |
| Slopegraph | No statistic | Draws two supplied values per category. |
| Scatter | No statistic | Plots raw x/y pairs. Any fitted overlay is a separate mode with its own entry. |
| Bubble | No statistic | Plots raw x/y pairs with a size channel. |
| PCA Scree Plot | No statistic | Plots the PCA eigenvalues, which are covered under Principal Components. |
| PCA Biplot | No statistic | Plots the PCA loadings and scores, which are covered under Principal Components. |
| Correspondence Map | No statistic | Plots the correspondence-analysis coordinates, which are covered under Correspondence Analysis. |
| Parallel Plot | No statistic | Draws one polyline per row over scaled axes. |
Finding a disagreement
If a Stratum result differs from another package in a way this page does not explain, please tell us — send the data and both results to support.