Decision Trees on Real Data

Regression and PCA both start from an assumption about the shape of the relationship. A decision tree makes no such assumption — it just repeatedly splits the data into more homogeneous groups and lets the structure emerge. That makes it a great second opinion: when a tree, working completely differently, lands on the same finding as your other methods, you know the finding is real and not an artifact of one technique.

What a decision tree actually does

A decision tree builds a flowchart. At each step it asks: of all the variables, which single split separates the outcomes best? It picks that split, divides the data into two branches, and repeats inside each branch. The result is a tree of yes/no questions ending in leaves that predict a class (classification) or a number (regression). No equation, no transform, no assumption of linearity — just recursive partitioning.

Two properties make trees valuable:

• They're readable. You can follow the path from root to leaf and explain exactly why a prediction was made — the split conditions are the reasons.
• They reveal what matters. The variables the tree splits on first, near the root, are the ones carrying the most signal. That's a built-in feature-importance ranking.

The diamonds experiment

In the full analysis, regression and PCA both concluded that a diamond's cut quality is about its proportions (depth and table), independent of its size. Let's test that from a completely different direction: grow a classification tree that tries to predict a diamond's cut grade from its raw physical measurements alone — carat, x, y, z, depth, table.

What does the tree reach for first? Not carat. Not the dimensions. It splits on table and depth — the exact two proportion variables that PCA had flagged as the "cut geometry" axis (PC2). An algorithm that knows nothing about gemology, given only raw numbers and a greedy splitting rule, independently rediscovers that cut quality is a story about proportions.

That convergence is the point. An unsupervised rotation (PCA) and a greedy supervised decision tree — two methods with nothing in common mechanically — agreeing on which variables define cut is far stronger evidence than either result alone. Read the tree's out-of-sample accuracy against the majority-class baseline in Stratum's output to see how much signal those two proportions carry.

When to reach for a tree

Decision trees earn their place when:

• You want an interpretable model you can hand to a non-statistician — the flowchart is the explanation.
• Relationships are non-linear or full of interactions that a straight regression would miss.
• You want a fast read on which variables matter before committing to a heavier model.
• The predictors are a mix of numeric and categorical — trees handle both natively, no dummy coding.

Their weakness is variance: a single deep tree can overfit and is sensitive to small data changes. That's exactly what random forests and boosted trees fix — averaging or sequentially correcting many trees for far better accuracy at the cost of the single tree's readability. (Each gets its own tutorial in this series.)

The bigger lesson

Good analysis rarely rests on one method. In the diamond investigation, charts raised the paradox, conditioning and regression resolved it, PCA explained its structure, and a decision tree confirmed it from the opposite direction. Four lenses, one convergent answer. When your tools disagree, you've found something worth investigating; when they agree despite being mechanically unrelated, you can trust the result.

In Stratum, growing a decision tree is a menu selection — it builds the tree, shows the splits, reports accuracy, and ranks variable importance, and clustering, random forests, and boosted trees sit right beside it. No code.

Read the full Diamond Paradox analysis → · ▶ Watch the walkthrough · Try Stratum free

Download Stratum

Related: Reading a PCA You Didn't Write → · Multiple Regression That Isn't Fooled by Confounders →