Some of you still need to work on algebra with sums, products, and logs. Mistakes that creap in there can make the rest of the problem meaningless.
If you compute a number in R, show it to me. If you save it and don’t show it, I can’t tell what it is. (And I’m not going to retype all of your code to find out.) If you don’t show it, I can’t grade it. In particular, for hypothesis tests, you should always show the test statistic and the p-value. In general, it is a good idea to show of the intermediate results in a longer calculation. That makes it much easier to diagnose things when there are problems. (When you are working in a live session, you can always query any values you want, but I can’t do that from your paper.)
For the most part, your code is looking much better this semester. But I’ll keep commenting on things that you should improve.
It would seem that some of you never look at the things you print. Please look them over. If you have text flowing off the page, I can’t read it. Use the return key to put line breaks in good places to keep things from running off the page.
Don’t forget to do sanity checking on your R code. Do some things that will let you know whether things might be broken. If you write a function, plug in a few numbers to make sure it is working. If you generate data randomly, look at the distribution. (If all the values are the same, you probably didn’t do that random thing you thought you did.)
Name things well. I saw several functions named pval_minus_critial() that had no direct connection to p-values. When I see that sort of thing, I get concerned that you don’t fully understand what you are doing. [Note: We can get likelihood confidence interval either by looking at the value of \(W = -2\) likelihood ratio, or by looking at p-values. The solutions show both ways.]
Most of you missed the point of the statndard error comparison. \(I^{-1/2}(\hat \theta)\) is an estimate of the standard error. But in this example, we can work out the standard error exactly (because we know a lot about binomial distributions). If you compare the actual standard error to the estimated standard error, you will see that the estimate simply plugs in \(\hat \pi\) where \(\pi\) should be. (This is sometime referred to as a “plug-in estimate”.)
The is an extra word in the statment of this problem. “Ratio” should not be there. Just log odds, not log odds ratio. (We can use log odds ratio as a way to compare two odds, but we only have one odds in this situation.)
Most of you did not work with \(\theta =\) log odds \(= \log(\pi/(1-\pi))\).
Also, there was some confusion along the way regarding parameters, which resulted in \(\theta\)’s and \(\pi\)’s getting swapped for each other. Be sure to take a look at the solution.
When computing method of moments estimates, don’t forget that var() does not compute the 2nd data moment. (The second data moment has \(n\) in the denominator instead of \(n - 1\)).
I wasn’t always sure everyone was understanding how the likelihood ratio works when we have more than one parameter. Be sure you understand what \(L(\hat\Omega_0)\) and \(L(\hat\Omega)\) are and how to calculate them (either algebraically or numerically, depending on the situation).
The simulations did not go well for some of you. You might start by stating in words what you need to simulate.
In this case we need to
simulate a sample of size 30 drawn from a \(Unif(0,1)\) distribution (that’s what the null hypothesis tells us)
compute \(W = -2 \log( \frac{L(\hat\theta_0)}{L(\hat\theta)}\) for this sample
We will do this some large number of times, collecting up all those \(W\)’s. Then we can compare the \(W\) computed form the actual data with the distribution of \(W\)’s computed from many simulated samples.
If you write a function to do one simulation and call it one_sim(...), then do(10000) * one_sim(...) will run the simulation 10,000 times (which still isn’t really enough to estimate the p-value very well when it is so small).
There are some other things we can do via simulation as well.
We can also compare the distribution of the simulated \(W\)’s to \(Chisq(1)\) so see if they look similar.
We can check whether simulated p-values appear to be uniformly distributed. (This is equivalent to the chi-squared check.)
We can also check whether the log-likelihood function is roughly quadratic. (We will be less concerned the better the quadratic approximation is – but remember that invariance means that things might be better than they appear.)
Final note: When you calculate a p-value using simulations, that too is only an estimate and there will be sampling variability. If you treat the simulations like a binomial situation (a bunch of simulations, each one had a test statistic either larger or smaller than the observered one, etc.), then you can use binom.test() to get a confidence interval for the p-value (which would be the unknown probability of observing a large value of \(W\) in this situation). Some software that computes p-values via simulations will report these sorts of confidence intervals.