bootstrap() — The bootstrap

Estimate uncertainty by resampling your own data.

bootstrap(statistic, data, reps)

Key idea

• Resample your data WITH replacement (the same point can be picked twice), same size, many times. (e.g. B = 2000 resamples)
• Recompute your number on each resample to map out its full range.
• The spread of those values is the standard error (how much it varies); percentiles give a confidence interval. (e.g. 2.5% & 97.5% -> 95% CI)
• Works for almost any number and needs no bell-curve assumption.

Definition

$$\widehat{\text{SE}}=\mathrm{sd}({\hat{\theta }}^{\ast 1},{\hat{\theta }}^{\ast 2},\dots ,{\hat{\theta }}^{\ast B})$$

${\hat{\theta }}^{\ast b}$

the statistic from resample b

$B$

how many resamples

$\widehat{\text{SE}}$

the bootstrap standard error

In plain words

Can't collect new data? Pretend your sample is the whole world and keep re-drawing from it to see how much your answer wobbles.

Where you'd use it

• Putting error bars on a statistic with no neat formula (a median, a ratio, a correlation).
• Confidence intervals when the data is clearly not normal.
• Estimating the uncertainty of a machine-learning model’s score.

Frequently asked questions

What is bootstrapping?

Resampling your data with replacement many times to estimate uncertainty (e.g. confidence intervals) without new data.