Main

Expectation And Variance

When performing an experiment, it is useful to know what the expected outcome will be as well as how much variation one can expect among the outcomes. The notions of expected outcome and variation are made formal in this module by the terms expectation,variance, and standard deviation.

This module will also show some of the connections of these statistical metrics with the applications of the previous modules.

Expectation

Consider a random variable with probability density function (PDF) defined on some domain . The expectation of , denoted by , is defined by

where is the probability element. The expectation of is sometimes called the mean of , the expected value, or the first moment. In some books it is denoted . It is best to think of the expectation as the number one gets by repeating the experiment many times and taking the average of the outputs.

The notion of expectation is more general than the mean because one can also take the expectation of a function of . The expectation of is defined by

Example

Find the expectation of , where is uniformly distributed on the interval .

Recall that the PDF associated with is given by for . Thus, the mean is given by

Example

Recall that the random variable is said to have the exponential distribution if the PDF associated with is for , where is some constant. Find the expectation of the exponential distribution (in terms of ).

From the definition of expectation, one finds

Using integration by parts, with

we find that

Variance

Consider a random variable with PDF . The variance of , denoted , is defined by

In the notation of the lecture,

Note: it requires some calculation to show the second equality above holds. Either of the above expressions may be taken as the definition of variance, and the second one might be slightly simpler for the sake of computation.

Expanding out the expression and using the linearity of the integral, we find

because and , by the definition of expectation and the definition of the probability density function, respectively.

Example

Compute the variance of the exponential density function .

The variance requires us to compute

Using integration by parts, with

we find

This second integral can be done with integration by parts again, or we can use the fact that this is almost the integral for the expectation. Namely, we know

and so by dividing through by , we have

Putting this together, we have

Finally, then, the variance is

Standard deviation

Consider a random variable with PDF . Then the standard deviation of , denoted , is defined by

Example

Find the standard deviation of , where is uniformly distributed over .

Again, recall that the PDF for the uniform distribution is for . Thus,

From the previous example, . Thus,

Interpretations

If one interprets the PDF as the density of a rod at location , then:

  1. The mean, , gives the center of mass of the rod.
  2. The variance, , gives the moment of inertia about the line .
  3. The standard deviation, , gives the radius of gyration about the line .

The normal distribution

A random variable is said to have the normal distribution, or to be normally distributed, with mean and standard deviation if its PDF is of the form

Due to its ubiquity throughout the sciences, the normal distribution is one of the most well-known probability distributions. However, because its PDF does not have an elementary anti-derivative, it is not easy to calculate exact probabilities associated with the normal distribution. Instead, there are is a rule of thumb which can be used.

The 68-95-99.7 rule

Given a random variable which is normally distributed with mean and standard deviation , the following hold:

  1. .
  2. .
  3. .

In other words, 68% of samples will fall within 1 standard deviation of the mean. 95% of samples will fall within 2 standard deviations of the mean. And 99.7% of samples will fall within 3 standard deviations. These rules, along with the symmetry of the normal PDF, can be used to approximate many probabilities relating to the normal distribution:

Example

The height of men in a certain population is normally distributed with mean inches and standard deviation inches. If a man is chosen at random from the population, what is the probability that he is taller than 72 inches?

Let be the height of a randomly chosen man. Then by the above rule. By symmetry . Also, by symmetry, . Thus,

It follows that

This is best visualized by labeling the various regions under the normal curve with their areas:

So the probability that a randomly chosen man from the population is taller than 72 inches is .16.


EXERCISES

  • Compute the expected value of normally distributed random variable with probability density function on .