Error and Uncertainty




Example: sample mean

\[ \overline X \approx {\sf Norm}(\mu , \frac{\sigma}{\sqrt{n}})\; , \] so the distribution of errors is

\[ \overline X - \mu \approx {\sf Norm}(0, \frac{\sigma}{\sqrt{n}})\; , \] and the standard uncertainty for this estimator is

\[ \mbox{standard uncertainty} = \frac{\sigma}{\sqrt{n}} \approx \frac{s}{\sqrt{n}}\; . \]

We will denote the standard uncertainty of a quantity \(X\) as \(u_X\).

Example: Difference in means

Similarly, for the difference in two population means (estiamted by taking the difference in two sample means), we have

\[ \mbox{standard uncertainty} = \sqrt{ \frac{\sigma_1^2}{{n_1}} + \frac{\sigma_2^2}{{n_2}} } \approx \sqrt{ \frac{s_1^2}{{n_1}} + \frac{s_2^2}{{n_2}} } \]

Goal: Estimate standard uncertainty for more situations.

  • The methods above worked because we have rules for propagation of means and variances for linear transformations and linear combinations.

  • We will learn an approximate method that works when things are approximately linear.

  • These methods will estimate standard uncertainty (ie, standard deviation of the estimator = SE), but they do not tell us about the shape of the distribution of errors. In many situtations the distribution of errors will be at least approximately normal, but in other situations it may be quite different.

Transforamtions of one random variable

An estimator is just a random variable, so we will derive our methods in terms of random variables and then apply them to estimators.

We already know that

This means that when \(Y = f(X)\) and \(f\) is a linear function (\(f(x) = ax + b\)), then we know how to compute the mean and variance for \(Y\) if we know the mean and variance for \(X\).

From calculus, you may remember that we can approximate arbitrary functions using the tangent line at a point.

\[ f(x) \approx f(a) + f'(a) (x - a) \]

This approximation will be better if

  • \(x\) is close to \(a\)
  • \(f(x)\) is not too curvy (ie, \(|f''|\) is small)

“Better” and “small” are relative – be sure to keep units and the range of reasonable values in mind.

Let’s apply this to our random variables. Suppose we know the mean and variance of \(X\). We want to estimate the mean and variance of \(Y = f(X)\).

  • let \(a = \operatorname{E}(X) = \mu\)

    • Most of the values of \(X\) will be near \(\operatorname{E}(X)\), so that is a good place to estimate from.
    • \(\operatorname{E}(X - \mu) = \operatorname{E}(X) - \mu = 0\), so that’s also handy.


  • Expected Value

\[\begin{align*} \operatorname{E}(Y) &= \operatorname{E}(f(X)) \\ &\approx \operatorname{E}(f(\mu) + f'(\mu) (x - \mu)) \\ &= f(\mu) + f'(\mu) \operatorname{E}(x - \mu) \\ &= f(\mu) \end{align*}\]

  • so \(\operatorname{E}(f(X)) \approx f(\operatorname{E}(X))\).


  • Variance

\[\begin{align*} \operatorname{Var}(Y) & = \operatorname{Var}(f(\mu) + f'(\mu) (X - \mu)) \\ &= \operatorname{Var}(f'(\mu) (X - \mu)) \\ &= f'(\mu)^2 \operatorname{Var}(X) \end{align*}\]

  • Standard deviation \[ \operatorname{SD}(Y) = |f'(\mu)| \operatorname{SD}(X) \]

Application to uncertainty estimation

We can’t quite use the approximation formula above because we don’t know \(\mu\). (It’s an unknown paramter that we are estimating.) But we can use our estimate in place of \(\mu\). Since the standard uncertainty is just the standard deviation of an estimator,

If

\[Y = f(X) \;, \]

then

\[ u_Y \approx | f'(\hat x) | u_X \]

where \(\hat x\) is our estimate.

We are simply replacing the mean \(\mu\) with our estimate \(\hat x\). (Typically, \(\operatorname{E}(X) = \mu\), so this estimate will be quite good if our sample size is large.)

Example 1. Let \(A\) be a square with side length \(L\). Suppose we estimate \(L\) to be 3.7 with uncertainty \(u_L = 0.2\) cm. (This is often written as \(3.7 \pm 0.2\). When you see \(\pm\) notation, you must also check to see how it is being used.)

What is our uncertainty in the computed area?

\(A = L^2\), so our function is \(f(x) = x^2\). \(f'(x) = 2x\). So we get

\[\begin{align*} u_A & \approx |f'(\hat L)| u_L \\ &= f'(3.7) \cdot 0.2 \\ &= 7.4 \cdot 0.2 = 1.48 \end{align*}\]

We could express this as \(13.69 \pm 1.48\). (But stay tuned for a note about how many digits we should be reporting.)

Example 1 (revisited). There is another way to estimate uncertainty of a transformed measurement. Let’s just simulate. In this case, we simulate values of \(L\) that are normally distributed near 3.7 with a standard deviation of 0.2. Then we can compute a bunch of simulated areas and see what the standard deviation is.

L <- rnorm(10000, mean = 3.7, sd = 0.2)
A <- L^2
sd(~ A)
## [1] 1.49
gf_histogram(~ A)

gf_qq(~A) %>% gf_qqline()

The estimate in this case is very close to the simulated value.

Compared to a normal distrubtion, this has somewhat “light tails” (the smallest values are not quite as small nor then largest quite as large as we would expect).

Seems to work pretty well in this case.

Sig Figs and Uncertainty

The guidelines in this sections are based on commonly used practices in physics and engineering.

What to record, What to report

When you record the results of a measurement for which there is an estimate of uncertainty, the uncertainty should be recorded along with the measurement itself. Similarly, reports of quantities estimated from data should also include estimated uncertainties.

As a general guideline, a properly reported scientific estimated quantity includes the following five elements:

  1. A number (the estimate)
  2. Units (e.g., m or kg or seconds)
  3. A statement about how it was measured or calculated
  4. A statement about most likely sources of (the largest components of) error
  5. An estimate of the uncertainty

Example If you measured the length of a pendulum using a meter stick, you might report the measurement this way:

  • Length \(= 0.834 \pm 0.002\) m
  • Measured with a meter stick from pivot point to the center of the steel weight.
  • Uncertainty reflects the limited accuracy of measurement with a meter stick.

In plots, the number is given by the scales of the plot, the units are typically included in the axes labels, uncertainties may be represented by ``error bars", and a statement describing the method of measurement or calculation should appear in the plot legend.

How many decimal places?

Numerical values and their uncertainties should be recorded to the proper number of decimal places. Most software either reports too many significant digits or rounds numbers too much. For correct professional presentation of your data, follow these guidelines:

  • Rule 1: The experimental uncertainty should be rounded to one significant figure unless the leading digit is a 1, in which case, it is generally better to use two digits. [Note: some people prefer that two digits be used when the leading digit is a 2 as well.]

  • Rule 2: A measurement should be displayed to the same number of decimal places as the uncertainty on that measurement.

Note carefully the difference between significant figure and decimal place.

The following examples will help:

Example The timer reports a value of 0.3451 seconds. The uncertainty on the measurement is 0.0038 seconds.

  • By Rule 1, the uncertainty should be reported to one significant figure, so we round it to 0.004 seconds.
  • By Rule 2, the measurement must also be rounded to the third decimal place.

Thus, the measurement should be reported as \(0.345\pm0.004\) seconds.

Example The measured value is \(7.92538 \cdot 10^4\), and its uncertainty is \(2.3872 \cdot 10^2\).

  • By Rule 1, the uncertainty should be rounded to one significant figure, so \(2 \cdot 10^2\).
  • By Rule 2, we report the measurement to the same decimal place as the uncertainty, so \(7.93 \cdot 10^4\).

Putting it together, the measurement should be reported as \((7.93 \pm0.02) 10^4\).

If we use the alternative method for Rule 2, we would report as \((7.925\pm 0.024) 10^4\).

Example The estimated value is \(89.231\), and its uncertainty is \(0.1472\).

  • By Rule 1, the uncertainty should be rounded to two significant figures, so \(0.15\).
  • By Rule 2, we report the estimate to the same decimal place as the uncertainty, so \(89.23 \pm 0.15\).

Reporting numbers in a table

Multiple similar measurements should be reported in a table. The column headings should clearly and concisely indicate the quantity in each column; the column heading must include the units. Uncertainties should be listed in a separate column, located just to the right of the measurement column. (Sometimes, uncertainties are listed in parentheses after the estimate instead; just make sure the header and legend of the table makes it clear what values are being reported, and where.)

Example A lab group calculated these numbers for kinetic energy and its uncertainty:

Kinetic Energy uncertainty
0.8682 0.059
1.0661 0.071
1.0536 0.070
1.3881 0.058
0.8782 0.108

This should be reported with appropriate rounding (and units) as

Kinetic Energy (J) uncertainty
0.87 0.06
1.07 0.07
1.05 0.07
1.39 0.06
0.88 0.11

Some Practice

1. Suppse you are estimating the area of a square. To do this, you measure the side length 30 times. Your 30 measurements have a mean of 10.5 cm and a standard deviation of 0.25 cm. Use this to

  1. Compute the standard uncertainty of the measured side length.
  2. Estimate the standard uncertainy of the computed area.

2. 10 measurements of the edge of a cube produce a sample mean \(\overline{x} = 3.11\) in and a sample standard deviation \(s = .13\) in. How should the volume of the cube be reported?

3. A temperature is reported as \(357 \pm 2\) degrees Fahrentheit. How should it be reported in Celsius? Do this two ways.

  1. Use the approximation method we have just learned.
  2. Work it out exactly using our rules for expected value and variance.
  3. How do the two results compare? Why?

4. An angle has size \(\theta\), measured in radians. The exact value of \(\theta\) isn’t known, but a sample of size \(n = 20\) produces a sample mean of 1.21. and a sd of 0.09. How should the value of \(\sin(\theta)\) be reported?

5. An angle \(\theta\) is reported as \(1.52 \pm .01\) (in radians),
How should \(\tan(\theta)\) be reported?

6. An angle is reported as \(0.32 \pm .02\) radians. How should we report the cosine of the angle?

7. The radius of a circle is reported as \(r = 3.45 \pm .06\) in.
How should we report the area of the circle?

8. The radius of a sphere is reported as \(r = 4.23 \pm .10\) in. How should we report the volume?

Combinations of mutliple random variables

We can do a similar thing with functions of multiple variables.

For a function of two independent random variables, the linear approximation has the form

\[ f(x,y) \approx f(a,b) + \left(\frac{\partial f}{\partial x}\right) (x - a) + \left(\frac{\partial f}{\partial y}\right) (y - b) \] where the partial derivatives are evaluated at \((a, b)\).1

Applying this to random variables \(X\) and \(Y\) with means \(\mu_X\) and \(\mu_Y\), we get

\[ f(X,Y) \approx f(\mu_X,\mu_Y) + \left(\frac{\partial f}{\partial X}\right) (X - \mu_X) + \left(\frac{\partial f}{\partial Y}\right) (Y - \mu_Y) \]

where the partial derivatives are evaluated at \((\mu_X, \mu_Y)\). Now we just need to calculate the variance:

\[\begin{align*} \operatorname{Var}(f(X,Y)) & \approx \operatorname{Var}\left( f(\mu_X,\mu_Y) + \left(\frac{\partial f}{\partial X}\right) (X - \mu_X) + \left(\frac{\partial f}{\partial Y}\right) (Y - \mu_Y) \right) \\ &= \left(\frac{\partial f}{\partial X}\right)^2 \operatorname{Var}(X) + \left(\frac{\partial f}{\partial Y}\right)^2 \operatorname{Var}(Y) \end{align*}\]

As before, we don’t know \(\mu_X\) and \(\mu_Y\). So we will plug our estimates into the partial derivatives. Taking a square root gives a formula for estimating the standard uncertainty.


This method of estimating uncertainties is usually called the delta method and is summarized in the box below.

The Delta Method for independent estimates

Let \(X\) and \(Y\) be independent estimators with uncertainties \(u_{X}\) and \(u_{Y}\), and let \(W = f(X,Y)\). Then the uncertainty in the estimator \(W\) can be estimated as \[ u_{W} \approx \sqrt{ \left(\frac{\partial f}{\partial X}\right)^2 u_X^2 + \left(\frac{\partial f}{\partial Y}\right)^2 u_Y^2 } \]

where the partial derivatives are evaluated using estimated values of \(X\) and \(Y\).


The Delta Method can be extended to functions of more (or fewer) than two variables by adding (or removing) terms. Slightly more complicated formulas exist to handle situations where the estimators are not independent (but we will not cover those in this course).


Because this method is based on using a linear approximation to \(f\), it works better when the linear approximation is better. In particular, when \(\frac{\partial^2 f}{\partial X^2}\) or \(\frac{\partial^2 f}{\partial Y^2}\) are large near the estimated values of \(X\) and \(Y\), the approximations might not be very good.

Dimes Suppose you want to estimate the number of dimes in a large sack of dimes. Here is one method you could use:

  1. Measure the weight of all the dimes in the bag by placing them (without the bag) on an appropriately sized scale. (Call this \(\hat B\), our estimate for \(B\), the actual weight of the dimes in the bag.)

  2. Measure the weight of 30 individual dimes and use those measurements to estimate the mean weight of dimes. (Call this \(\hat D\).)

  3. Combine these two estimates to compute an estimated number of dimes in the bag. (\(\hat N = \hat B / \hat D\).)

Suppose that the dimes in our our bag together weigh 10.2 kg and the mean weight of our 30 measured dimes is 2.258. Then we would estimate the number of dimes to be

\[ 10200 / 2.258 = 4516.805 \;. \]

But how good is this estimate? Do we expect to be within a small handful of dimes? Might we be off by 100 or 500? Standard uncertainty provides a way to quantify this. We will proceed in three steps

  1. Determine the uncertainty in our estimate for \(D\).

This is the part we already know how to do. We just need the standard error for a mean: \(\frac{s}{\sqrt{n}}\).

df_stats(~ mass, data = Dimes, mean, sd, n = length)
##   response mean     sd  n
## 1     mass 2.26 0.0221 30

So \(u_D = 0.0221 / \sqrt{30} = 0.004\).


  1. Determine the uncertainty in our estimate for \(B\).

For the mass of all the dimes, we need a different approach, since this is not based on the average of several measurments of different bags. We only have on measurement. The scale reads 10.2 kg. That’s probably not the exact mass. In fact, if the decimal readout only shows tenths of a kg, then any value between 10.15 and 10.25 would read as 10.2. So a good model for the distribution of errors would be \({\sf Unif}(-0.05, 0.05)\). Looking in our table, we see that the variance for this distribution is \(\frac{(b-a)^2}{12}\), so the uncertainty is

\[ u_B = \frac{0.1}{\sqrt{12}} = 0.029 kg = 28.9 g \]


  1. Combine these using the delta method to get the uncertainty for \(N\).

So far we have

\[ u_D = 0.004 \qquad u_B = 0.029 kg = 29g \] The delta method gives an approximate uncertainty for \(N\). First, we compute our two partial derivatives for \(f(D, B) = B/D\).

\[\begin{align*} \frac{\partial f}{\partial D} &= - B/D^2 \\ \frac{\partial f}{\partial B} &= 1/D \end{align*}\]

Then we combine everything to get \(u_N\).

\[\begin{align*} u_N &= \sqrt{ \frac{\hat B^2 }{ \hat D^4} u_D^2 + \frac{1}{\hat D^2} u_B^2} \\ &= \sqrt{ \frac{10200^2 }{ 2.26^4} 0.004^2 + \frac{1}{2.26^2} 29^2 } \\ &= 15.115 \\ \end{align*}\]

So we report the number of dimes as \(4517 \pm 15\).


Dimes and simulation We could also estimate the uncertainty in our estimate for the number of dimes using simulations.

B <- runif(10000, 10150, 10250)
SampleMeans <- do(10000) * mean( ~ mass, data = resample(Dimes) ) 
head(SampleMeans, 3)
##   mean
## 1 2.26
## 2 2.25
## 3 2.26
D <- SampleMeans$mean 
N <- B / D
gf_histogram( ~ N) 

gf_qq( ~ N)

sd(N)
## [1] 15

Recall: resample() samples with replacement. This is a way to use our data set to simulate many possible data sets. Some observations from the original sample may appear more than once, others not at all. If we use sample(), we will get the same data set every time (just in a different order).


Where do the original uncertainties come from?

In order to use the delta method, we need to have uncertainties for the quantities involved in our function. Where do they come from? There at least to possibilities.

  1. From our data

    If our estimate comes from sample data, we should be able to use the uncertainty of our estimator. That’s how we determined \(u_D\) in the dimes example. Since \(D\) was a sample mean, we used the standard error for the mean (\(\frac{s}{\sqrt{n}}\)). This is measuring uncertainty due to sampling variability (variability from sample to sample because of the particular items selected for the sample).

  2. From a model

    Our estimated uncertainy for \(B\) came from a model for how the scale works. The accuracy of this estimate depends on how well our model matches the behavior of the scale.

    Three commonly used models for this type of measurement are the uniform, triangle, and normal distributions. Each has an uncertainty that is based on an interval \([a,b]\) in which the true value almost surely lies. (It is actually only the width of this interval that matters.) Here is a table of the uncertainties for these three families, described in terms of \(b - a\).

distribution uncertainty
uniform \(\displaystyle \frac{b-a}{2 \sqrt{3}}\)
triangle \(\displaystyle \frac{b-a}{2 \sqrt{6}}\)
normal \(\displaystyle \frac{b-a}{2 \cdot 3}\)

More Practice

9. Choosing good names in R can help you stay organized. For the dimes problem we might set things up like this:

B <-       # measured mass of dimes in bag
D <-       # estimated average mass of a dime
u_B <-     # uncertainty for B
u_D <-     # uncertainty for D
dB <-      # partial derivative with respect to B
dD <-      # partial derivative with respect to D
N <-       # expression to compute N from B and D
u_N <- sqrt( dB^2 * u_B^2 + dD^2 * u_D^2 )

Fill in the missing parts, run the code, and check that you get the same result as we had above.

10. Return to the dimes example. Suppose you found out that the scale used only uses even digits for the tenths of kilogram. This means the scale is not as accurate as we thought. Given this new information, how should we report the uncertainty for the mass of the dimes in the bag? For the number of dimes?

\(u_B = 57.735; \quad u_N = 26.807\) (How many digits should you keep for each uncertainty?)


11. Suppose \(x ̂= 3.41 \pm 0.04\) is the estimate of the length of a rectangle and \(y ̂= 2.34 \pm 0.02\) is the estimate of its width. How do we report the area?

\(7.98 \pm 0.12\)


12. The length of a rectangle is reported as \(\hat x = 3.05 \pm 0.03\) and the width of the rectangle is reported as \(\hat y = 5.45 \pm 0.11\). How should perimeter of the rectangle be reported?

\(17.00 \pm 0.23\) (Or \(17.0 \pm 0.2\))


13. To estimate the average speed of an object, a physics student measures the time and distance traveled as \(3.21 \pm 0.05\) meters in \(5.25 \pm 0.03\) seconds.

How should the average velocity be reported?

\(0.611 \pm 0.027\) m/s (Or \(0.61 \pm 0.03\) m/s)


14. When two resistors with resistances \(R_1\) and \(R_2\) are connected in parallel, the combined resistance satisfies \[ R = \frac{R_1 R_2}{R_1 + R_2} \] Suppose the resistances of the two resistors are reported as \(20 \pm 0.7\) ohms and \(50 \pm 1.2\) ohms. How should you report the combined resistance?

\(14.3 \pm 0.4\)


15. On an analog, we can usually tell which two marks surround our measured value. In fact, we can usually tell which one is closer. So if our scale as numbers at each integer (\(0, 1, 2, 3, \dots\)), we might be able to tell that the measurement is, for example, “between 13 and 14, but closer to 13”. What uncertainty should we use for a measurement from this device (assuming a uniform model)?

Since we are sure the value is between 13 and 13.5, we get \(\displaystyle \frac{0.5}{2 \sqrt{3}}\) = 0.144


Relative Uncertainty

Is an uncertainty of 2.5 large or small? Well, that depends on the scale of numbers involved. If our estimate is in the thousands or millions, then 2.5 seems small. If our esimate is 4.3, then 2.5 seems large.

One way to take this into account is to report relative uncertainty:

\[ \mbox{ralative uncertainty} = \frac{\mbox{uncertainty}}{\mbox{estimate}} = \frac{u_X}{\hat x} = \frac{u_X}{X} \]

Again, we have shown the formula both in a statistician’s notation (with a hat and lower case to indicate the estimate) and in notation that scientists often use (which is less precise regarding estimates vs estimators).

Let’s compute the relative uncertainty for a product \(P = XY\).

Example Let \(P = XY\) where \(X\) and \(Y\) have uncertainties \(u_X\) and \(u_Y\). Then \(\frac{\partial}{\partial X} P = Y\) and \(\frac{\partial}{\partial Y} P = X\), so

\[\begin{align*} \frac{u_P}{P} & = \frac{ \sqrt{Y^2 u_X^2 + X^2 u_Y^2}}{P} \\ & = \sqrt{ \frac{ Y^2 u_X^2 + X^2 u_Y^2}{P^2} } \\ & = \sqrt{ \frac{ Y^2 u_X^2 + X^2 u_Y^2}{X^2Y^2} } \\ & = \sqrt{ \frac{ u_X^2}{X^2} + \frac{u_Y^2}{Y^2} } \\ & = \sqrt{ \left(\frac{ u_X}{X}\right)^2 + \left(\frac{u_Y}{Y}\right)^2 } \end{align*}\] which gives a Pythagorean identity for the relative uncertainties.

We can use the rule for relative uncertainty to compute the standard uncertainty for a product

Example Use relative uncertainty to compute the standard uncertainty in the area of a rectangle computed by multiplying length and width if length and width are reported as \(x = 3.41 \pm 0.04\) is the estimate of the length of a rectangle and \(y = 2.34 \pm 0.02\) is the estimate of its width.

The relative uncertainties in the length and width are \[ \frac{0.04}{3.41} = 0.012 \ \operatorname{and}\ \frac{0.02}{2.34} = 0.009 \;. \]

So the relative uncertainty in the area estimation is \[ \sqrt{ (0.012)^2 + (0.009)^2 } = 0.015 \;. \]

Now we solve \[ \frac{u_A}{7.979} = 0.015 \] to get \[ u_A = (7.979) (0.015) = 0.116 \;. \]

More Practice

16. Compute the relative uncertainties for the length, width, and perimeter in problem 12.

17. Compute the relative uncertainties for the three quantities in the dimes example.

18. A measurement is reported as 152 with a relative uncertainty of 3%. What is the standard uncertainty for this measurement?

19. If \(X\) has a relative uncertainty of 3% and \(Y\) has a relative uncertainty of 4%, what is the relative uncertainty of the product \(XY\)?

20. Compute the relative uncertainties for time, distance, and average velocity in problem 13.

21. Let \(Q = X / Y\). Work out a formula for the relative uncertainty of \(Q\) in terms of the relative uncertainties of \(X\) and \(Y\).

22. Check that your formula in 20 is correct for problem 19.

23. Check that your formula in 20 is correct for the dimes example.

  1. We could denote this as \(\left. \frac{\partial f}{\partial x}\right|_{x=a, y=b}\), but it makes the notation pretty messy.↩︎