The LOS reads as follows:
e. calculate and interpret the F-statistic, and discuss how it is used in regression analysis
Discuss how it is used in regression analysis
The use of the F statistic has already been discussed in Reading 11, 3.6. The difference between Reading 11 and Reading 12 relates to the number of independent variables.
We use the t-test to determine whether the coefficients are equal to 0. If none of the independent variables in a regression model helps explain the dependent variable, the slope coefficients should all equal 0.
In a multiple regression, however, we cannot test the null hypothesis that all slope coefficients equal 0 based on t-tests that each individual slope coefficient equals 0, because the individual tests do not account for the effects of interactions among the independent variables. We therefore use the F-statistic.
Calculate the F-statistic
For one independent variable the formula for the F statistic is as follows:
But remember, Reading 12 deals with multiple independent variables. The formula is therefore:
where k = number of slope coefficients
In a previous post, we used data from BHP and the excel function to create a ANOVA table. The output was as follows:
Using the F-statistic, we are able to reconstruct how F (0.881045325) was calculated.
RSS = 101.240
SSE = 114.909
k = 2
n = 5
Therefore;
(101.240/2) / (114.909/(5-(2+1))
50.620 / 57.454
= 0.88104
Interpret the F-statistic
Remember that we are using the F-statistic to determine whether the slope coefficients equal 0. To do that we need a significance level and need to compare the calculated F-statistic to the tables.
Lets assume that the level of significance is 0.05
WE ALWAYS USE A ONE SIDED TEST AS THE MSR NECESSARILY INCREASES RELATIVE TO MSE AS THE EXPLANATORY POWER OF THE REGRESSION INCREASES
So, on the F-statistic table, with 2df in the Numerator & 2df in the Denominator, we note a value of 19.
This indicates that we can accept the null hypothesis and that all the slope coefficients equal 0.
The independent variables therefore DO NOT help explain the dependent variable.
The table also provides a “Significance F,” p-value of 0.53. This p-value means that the smallest level of significance at which the null hypothesis can be rejected is 0.53.
The small value for this F-statistic implies a high probability of incorrectly rejecting the null hypothesis (a mistake known as a Type I error).
RSS = 101.240
SSE = 114.909
k = 2
n = 5
Therefore;
(101.240/2) / (114.909/(5-(2+1))
50.620 / 57.454
= 0.88104
Interpret the F-statistic
Remember that we are using the F-statistic to determine whether the slope coefficients equal 0. To do that we need a significance level and need to compare the calculated F-statistic to the tables.
Lets assume that the level of significance is 0.05
WE ALWAYS USE A ONE SIDED TEST AS THE MSR NECESSARILY INCREASES RELATIVE TO MSE AS THE EXPLANATORY POWER OF THE REGRESSION INCREASES
So, on the F-statistic table, with 2df in the Numerator & 2df in the Denominator, we note a value of 19.
This indicates that we can accept the null hypothesis and that all the slope coefficients equal 0.
The independent variables therefore DO NOT help explain the dependent variable.
The table also provides a “Significance F,” p-value of 0.53. This p-value means that the smallest level of significance at which the null hypothesis can be rejected is 0.53.
The small value for this F-statistic implies a high probability of incorrectly rejecting the null hypothesis (a mistake known as a Type I error).
