forest() — Random forests

Averages many random trees to cut variance.

forest(trees, depth, X, y)

Key idea

• Grow many trees, each on a bootstrap resample (data drawn with replacement), then average their predictions. (e.g. B trees -> average)
• Each split sees only a random handful of the inputs, so the trees disagree in independent ways.
• Averaging these disagreeing trees cancels the random errors — much lower variance than one tree.
• Needs little tuning and gives a free out-of-bag error estimate and input-importance scores.

Definition

$$\hat{f}(x)=\frac{1}{B}\sum _{b=1}^B{T}_b(x)$$

$B$

how many trees in the forest

$T_b$

tree b, grown on a bootstrap resample

$\hat{f}$

the averaged prediction

In plain words

One opinionated tree is shaky. Ask a thousand slightly different trees and average them, and the wobble disappears.

Where you'd use it

• A strong, low-effort default for almost any tabular prediction problem.
• Ranking which features matter via built-in importance scores.
• Robust predictions when you have mixed numeric and categorical data.

Frequently asked questions

Why is a random forest better than one tree?

Averaging many varied trees cuts variance and lifts accuracy — at the cost of interpretability.