P-Value in Linear Regression

In Linear Regression, if the p-value of the f-stat is < 0.05 we reject the H0 and accept the Ha. Which means we know that at least one variable has a coefficient greater than zero. For individual variables, we consider the variable to be statistically significant if the p-value<0.05. Why this is so?

1 comment

  1. If the p-value for f statistic is less than 0.05, we reject the null hypothesis which means that at least one beta coefficient is not zero. We conclude that the over all model is significant.

    By checking the f-statistic we concluded that the overall model is statistically significant, however we need to identify if any of the predictors included in the model are not related to the response variable.

    We check the t-statistic to confirm which of the predictor variables are NOT related to the response variable and which of these variables are statistically significant predictors of the response variables.

    Alternatively, we may argue that if we are checking the t-statistic of individual predictor variables, and even if one of the predictor variable is a significant, then the overall model should be considered as significant or valid, then why do we check the F-statistic for overall validity of the model.

    That’s because  5% of the predictor variables will be significant by sheer chance(@ 95% confidence level). This will be specially true for models with a model with multiple predictor variables. F-statistic does not suffer from this as it adjusts for the number of predictor variable. Hence we confirm the overall validity of the model using F-statistic.

Leave a comment