You may use your notes and the class text, but you may not consult other resources or other people. Send me an email if you have questions or difficulties of any sort.
Prepare your solutions using R Markdown and turn in a printed copy. Also email me your Rmd file.
A study was done to see how cognitive test scores of three- and four-year-old children were related to the IQ scores of their mothers and other characteristics of their mothers.
data(KidsIQ, package = "CalvinBayes")
head(KidsIQ, 3)| kid_score | mom_hs | mom_iq | mom_work | mom_age |
|---|---|---|---|---|
| 65 | Yes | 121.1 | 4 | 27 |
| 98 | Yes | 89.4 | 4 | 25 |
| 85 | Yes | 115.4 | 4 | 27 |
Fit four models with the kid’s score as reponse.
In each case use “normal noise” and reasonable priors.
What does “normal noise” mean? Explain this in the context of model 3.
Explain your choice of priors.
Give a 95% HDI for what each model predicts for the average score of a child whose mother graduated from high school and has an IQ of 120.
Now suppose that Sally’s mother graduated from high school and has an IQ of 120. Give a 95% HDI for what each model predicts for Sally’s score.
Explain why the answers to the previous two questions are different.
What is an interaction effect in the context of model 4? What would a positive interaction effect mean? What would a negative interaction effect mean? Does model 4 think there is an interaction effect? If so, is it positive or negative?
Suppose you are interested in whether completing high school has an effect on the child’s score. Which model would you use? Why? Using that model, how would you answer the question? (If the answer is yes, be sure to say something about the magnitude of the effect.)
Suppose you are interested in how a mother’s IQ has an effect on the child’s score.
Which model would you use? Why? Using that model, how would you answer the question? (If the answer is yes, be sure to say something about the magnitude of the effect.)
(Yes, this was state-of-the-art TV when I was a kid.)
Test 3 included a problem about a study of the effect of watching the PBS TV show The Electric Company on kids’ reading preformance. That problem ignored an important part of the study: It was a paired design. At each school, the two worst performing class were selected, and it was randomly determined which class would watch The Electric Company and which one would not. id is a numeric code for each school, and you can see that there are two classes, one in each group, at each school. Once again, we will use only the grade 2 data.
Grade2 <- ElecComp %>%
filter(grade == 2) %>%
mutate(id = factor(id)) # convert from numeric to categorical
Grade2 %>% arrange(id) %>% head(6)| city | id | grade | pre | post | group | method |
|---|---|---|---|---|---|---|
| F | 12 | 2 | 55.1 | 81.6 | treatment | R |
| F | 12 | 2 | 50.3 | 69.1 | control | NA |
| F | 13 | 2 | 73.1 | 101.2 | treatment | R |
| F | 13 | 2 | 63.3 | 77.0 | control | NA |
| F | 14 | 2 | 51.4 | 66.4 | treatment | R |
| F | 14 | 2 | 50.6 | 72.9 | control | NA |
Here are two models that take this into account, along with one of the models we fit before that did not take this into account.
ec0 <- brm(post ~ pre + group, data = Grade2)
ec1 <- brm(post ~ pre + group + id, data = Grade2)
ec2 <- brm(post ~ pre + group + (1 | id), data = Grade2)Why was it important for ec1 that we converted the numeric ids into categorical values using factor()? What would happen if you did not?
What is the difference between ec1 and ec2? Why is ec2 more appropriate for the design of this study?
ec2 has a lot more parameters than ec0. Which model does WAIC indicate is a better choice? (Don’t use LOOIC here. The PSIS approximation does not work well for ec2, so we would need to do some sort of k-fold cross validation (basically “leave several out” instead of “leave one out”) to get a trustworthy approximation, but we haven’t talked about that.)
Give an estimate of the treatment effect using the ec0 and ec2 and explain how/why they do/do not differ.
Explain how to interpret the “Group-Level Effects” section of the output for ec2.
Note: in a complete treatment of this data, we might include other things as well. Examples: all four grades of students, the city in which the schools were, and interactions. These were not included here to keep things a tad simpler.