Linear
Regression

Dr. Ajay Kumar Koli, PhD | SARA Institute of Data Science, India

Regression


Regression is a statistical method to model and analyze relationships between variables.

Types of Regression

  • Linear Regression: Simple (1 predictor) and Multiple (more than 1 predictor).

  • Logistic Regression, Polynomial Regression, etc.

Objectives of Regression

  • Predict outcomes based on predictor variables.

  • Understand the strength and direction of relationships.

  • Make informed decisions using models.

Linear Regression Model

  • Simple Linear Regression Equation:

\[y = \beta_0 + \beta_1x + \epsilon\]

  • \(y\) = Dependent variable (outcome).

  • \(x\) = Independent variable (predictor).

  • \(\beta_0\) = Intercept.

  • \(\beta_0\) = Slope.

  • \(\epsilon\) = Error term.

Assumptions of Linear Regression

  • Linearity: Relationship between \(x\) and \(y\) is linear.

  • Independence: Observations are independent.

  • Homoscedasticity: Errors have constant variance.

  • Normality: Residuals are normally distributed.

Practice GLM

General Explanation of Regression Metrics:

  • Intercept: The baseline prediction when all predictors are zero.

  • Coefficients: The change in the dependent variable for a one-unit increase in the predictor.

  • P-value: Determines statistical significance (p < 0.05 is typically significant).

  • R-squared: Explains the proportion of variance captured by the model (higher values indicate a better fit).