tree() — Decision trees (CART)

Splits the data into regions with simple yes/no questions.

tree(depth, X, y)

Key idea

• Ask yes/no questions to split the data into boxes (leaves); predict the average of the points in each box. (e.g. depth 3 -> up to 8 leaves)
• CART grows greedily, always taking the split that reduces error the most right now.
• Shallow trees underfit (too coarse); deep trees overfit (memorise noise), so depth is capped or the tree is pruned.
• Easy to read as rules, but a single tree is high-variance — forests and boosting fix that.

Definition

$$\hat{f}(x)=\sum _m{c}_{m}1(x\in R_m)$$

$R_m$

a region (leaf) carved by the splits

$c_m$

the prediction in that region (its mean)

$\text{depth}$

how many questions deep the tree goes

In plain words

Play twenty questions with the data: each yes/no answer narrows things down until you can make a guess.

Where you'd use it

• Transparent rule-based decisions in medicine or finance ("if blood pressure > 140 and age > 50 …").
• A quick, interpretable baseline before reaching for heavier models.
• The base learner inside random forests and gradient boosting.

Frequently asked questions

Why do single trees overfit?

They keep splitting until the leaves are pure. Prune, limit depth, or use a forest.