Mixed Models: Testing Significance of Effects solutions

This article contains solutions to exercises for an article in the mixed models series. For a list of topics covered by this series, see the Introduction.

There is often more than one approach to the exercises. Do not be concerned if your approach is different than the solution provided.

Exercise solutions

Use the models from sleepstudy data set for the following exercises. The models used in these solutions were constructed in the the Solutions for the Model's article.

We will load some packages which will be used in the solutions.
```
library(pbkrtest)
library(pbnm)
library(RLRsim)
```

Test the significance of the fixed effects in your model from exercise 5 in the Models article.

The profiled confidence interval provides a good check of significance.

confint(ssc)

Computing profile confidence intervals ...

                 2.5 %    97.5 %
.sig01       26.007120  52.93598
.sigma       27.813847  34.59105
(Intercept) 231.992326 270.81788
Days          8.886551  12.04802

The conficence interval for the Day coefficient is far from zero.
This is evidence that the Days is significant.

The Kenward-Roger F test will be used to confirm the finding from confint() and also to provide a p-value.

sscDday <- lmer(Reaction ~ (1 | Subject), data=sleepstudy)
(KRsscDay <- KRmodcomp(ssc,sscDday))

F-test with Kenward-Roger approximation; computing time: 0.11 sec.
large : Reaction ~ Days + (1 | Subject)
small : Reaction ~ (1 | Subject)
       stat   ndf   ddf F.scaling   p.value    
Ftest 169.4   1.0 161.0         1 < 2.2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

The p-value of approximentely 0 provides additional evidence that Days is significant.

Test the significance of the fixed effects in your model from exercise 6 in the Models article.

The profiled confidence interval provides a good check of significance.

confint(ssd)

Computing profile confidence intervals ...

                 2.5 %    97.5 %
.sig01       1.6943096  4.851559
(Intercept) -5.6358535 -1.772446
Days         0.5142808  1.029079

The conficence interval for the Days coefficient is does not include zero. This is evidence that the Day is significant.

A parametric bootstrap test will be used to confirm the finding from confint() and also to provide a p-value.

ssdDday <- glmer(aboveAve ~ (1 | Subject), data=sleepstudy, family=binomial)
pbssdDday <- pbnm(ssd,ssdDday,nsim=5000,tasks=10,cores=2,seed=72740419)
summary(pbssdDday)

Parametric bootstrap testing: Days = 0 
from: glmer(formula = aboveAve ~ Days + (1 | Subject), data = sleepstudy,  family = binomial) 
5000 samples were taken Sun Mar 27 05:02:39 2016 
19 samples had warnings, 16 in alternate model 13 in null model 
19 unused samples.  0 <= P(abs(Days) > |0.7425723|) <= 0.0038

The p-value of no greater than 0.004 is small and provides additional evidence that Days is significant.

Test the significance of the random parameter in your model from exercise 5 in the Models article.

The profiled confidence interval provides a good check of significance.
```
confint(ssc)
```
```
Computing profile confidence intervals ...
```
```
                 2.5 %    97.5 %
.sig01       26.007120  52.93598
.sigma       27.813847  34.59105
(Intercept) 231.992326 270.81788
Days          8.886551  12.04802
```
The conficence interval for the Subject variance parameter, sig01 in the above display, is far from zero.
This is evidence that the variance is not zero.

The exactRLRT() function will be used to confirm the finding from confint() and also to provide a p-value.
```
(ERsscDay <- exactRLRT(ssc))
```
```
    simulated finite sample distribution of RLRT.

    (p-value based on 10000 simulated values)

data:  
RLRT = 107.2, p-value < 2.2e-16
```
The p-value of approximentely 0 provides additional evidence that the variance of Subject is not zero.

Test the significance of the random parameter in your model from exercise 6 in the Models article.

The profiled confidence interval provides a good check of significance.

confint(ssd)

Computing profile confidence intervals ...

                 2.5 %    97.5 %
.sig01       1.6943096  4.851559
(Intercept) -5.6358535 -1.772446
Days         0.5142808  1.029079

The conficence interval for the Subject variance parameter, sig01 in the above display, does not include zero.
This is evidence that the variance is not zero.

A parametric bootstrap test will be used to confirm the finding from confint() and also to provide a p-value.

ssdDsub <- glm(aboveAve ~ Days, data=sleepstudy, family=binomial)
pbssdDsub <- pbnm(ssd,ssdDsub,nsim=5000,tasks=10,cores=2,seed=93057913)
summary(pbssdDsub)

Parametric bootstrap testing: (Intercept) | Subject = 0 
from: glmer(formula = aboveAve ~ Days + (1 | Subject), data = sleepstudy,  family = binomial) 
5000 samples were taken Sun Mar 27 05:06:03 2016 
1 samples had warnings, 1 in alternate model 0 in null model 
1 unused samples.  0 <= P(abs((Intercept) | Subject) > |7.95053|) <= 2e-04

The p-value of no greater than 0 provides additional evidence that the variance of Subject is not zero.

Return to the Testing Significance of Effects article

Last Revised: 3/9/2016