Linear Regression

Model: y_i = β₀ + β₁x_i1 + … + β_kx_ik + ε_i

• y_i: dependent variable
• x_ij: regressors
• β_j: coefficients (partial effects)
• ε_i: unobserved factors affecting y

OLS minimizes the sum of squared residuals:

Σ_i=1ⁿ (y_i − ŷ_i)²

β_j is the expected change in y from a one-unit increase in x_j, holding other regressors constant.

• Linearity in parameters
• Random sampling
• No perfect multicollinearity
• Exogeneity / zero conditional mean: E[ε | X] = 0 (equivalently Cov(X_j, ε)=0)
• Homoskedasticity (for classical SEs)
• No autocorrelation
• Normal errors (or large-sample CLT for inference)

Exogeneity is the key assumption for unbiased and consistent OLS.

• Exogenous regressor: Cov(X, ε)=0
• Endogenous regressor: Cov(X, ε)≠0

Endogeneity breaks causal interpretation.

OVB occurs when (1) a relevant variable affects y, and (2) it is correlated with an included regressor. The omitted factor enters ε, inducing Cov(X, ε)≠0 → biased and inconsistent OLS estimates.

• Leverage: how extreme an observation is in regressor space
• Influence: how much an observation affects coefficient estimates

• R²: variance explained relative to the mean-only model; increases with more regressors
• Adjusted R²: penalizes unnecessary regressors

Negative R² means the model fits worse than predicting the sample mean (often misspecification, no intercept, or out-of-sample evaluation).

• Confidence interval: uncertainty around the conditional mean E[y | X]
• Prediction interval: uncertainty around a new observation; always wider

No. In OLS with an intercept, residuals are orthogonal to regressors by construction (X' ê = 0). So corr(X, residuals) ≈ 0 even if the model is endogenous.

OLS guarantees Cov(X, ê)=0 in-sample. Exogeneity requires Cov(X, ε)=0, which involves the true unobserved error ε. Residual-correlation checks are circular and uninformative for bias.

• H₀: regressors are exogenous (OLS consistent)
• H₁: regressors are endogenous (OLS inconsistent)

Rejecting H₀ indicates endogeneity. Requires valid instruments.

• Exogeneity violations (Cov(X, ε)≠0): biased coefficients + invalid inference
• Heteroskedasticity/autocorrelation: coefficients often unbiased, standard errors wrong → inference invalid
• Multicollinearity: unbiased coefficients, inflated SEs → low power
• Residual checks diagnose error structure, not bias; Hausman/IV frameworks address endogeneity