How would you bootstrap a regression model?

How would you bootstrap a regression model?

• A. Resample residuals only and re-fit.
• B. Resample observations with replacement, fit the model to each bootstrap sample, and analyze the distribution of coefficients.
• C. Shuffle the response variable across observations.
• D. Resample only the response variable and refit.

Answer: (B)

When you bootstrap a regression model, you want to mimic drawing new samples from the same population that produced your data. The standard approach is to resample the observed pairs (predictors and response) with replacement to form a bootstrap sample, fit the regression to that sample, and repeat this many times. The collection of coefficient estimates from all bootstrap fits forms an empirical distribution that approximates the sampling distribution of the estimators. This lets you estimate standard errors, confidence intervals, and bias directly from the data. This method works well because each bootstrap sample preserves the relationship between X and Y that was present in the observed data, so the variability you see across bootstrap fits reflects how the coefficients would vary if you had drawn a different sample of units from the same population. Resampling residuals only and refitting is a residual bootstrap, which relies on assumptions about the error structure (like homoscedasticity) and can be less robust if those assumptions don’t hold. Shuffling the response variable across observations breaks the association between X and Y and is more appropriate for hypothesis testing under permutation than for estimating the sampling distribution of regression coefficients. Resampling only the response variable also ignores the predictor structure and distorts the relationship you’re trying to estimate.

When you bootstrap a regression model, you want to mimic drawing new samples from the same population that produced your data. The standard approach is to resample the observed pairs (predictors and response) with replacement to form a bootstrap sample, fit the regression to that sample, and repeat this many times. The collection of coefficient estimates from all bootstrap fits forms an empirical distribution that approximates the sampling distribution of the estimators. This lets you estimate standard errors, confidence intervals, and bias directly from the data.

This method works well because each bootstrap sample preserves the relationship between X and Y that was present in the observed data, so the variability you see across bootstrap fits reflects how the coefficients would vary if you had drawn a different sample of units from the same population.

Resampling residuals only and refitting is a residual bootstrap, which relies on assumptions about the error structure (like homoscedasticity) and can be less robust if those assumptions don’t hold. Shuffling the response variable across observations breaks the association between X and Y and is more appropriate for hypothesis testing under permutation than for estimating the sampling distribution of regression coefficients. Resampling only the response variable also ignores the predictor structure and distorts the relationship you’re trying to estimate.