• About
• Statistics
• Number Theory
• Java
• Data Structures
• Cornerstones
• Calculus

Variance & Standard Deviation of a Discrete Random Variable

For a given random variable $X$, with associated sample space $S$, expected value $\mu$, and probability mass function $P(x)$, we define the standard deviation of $X$, denoted $SD(X)$ or $\sigma$, with the following: $$SD(X) = \sqrt{\sum_{x \in S} (x-\mu)^2 \cdot P(x)}$$

The sum underneath the square root above will prove useful enough in the future to deserve its own name. As such, we define the variance of $X$, denoted $Var(X)$ or $\sigma^2$, by $$Var(X) = \sum_{x \in S} (x-\mu)^2 \cdot P(x)$$ Beyond being the square of the standard deviation, note that the variance can also be interpreted as the expected value of $(X - \mu)^2$. That is to say, the variance is the average squared distance between the outcomes $x$ and $\mu$, the "center" of the distribution for $X$:

Now, if one knows the probability mass function for $X$ as a table, and the sample space associated with $X$ is $S$, the expression above can be calculated as

Recall that the standard deviation is the square root of the variance, so the above gives us a more convenient way to calculate the standard deviation as well: $$SD(X) = \sqrt{\sum [x^2P(x)] - \mu^2}$$

An Example:

Suppose one wishes to find the standard deviation of a random variable $X$ with probability mass function given by the following table:

To do this, one finds the expected value, variance, and finally the standard deviation for $X$, each in turn: $$\begin{array}{rcl} E(X) &=& (-4)(0.50)+(2)(0.30)+(5)(0.15)+(10)(0.05)\\ &=& -0.15\\ &&\\ Var(X) &=& \left[(-4)^2(0.50)+(2)^2(0.30)+(5)^2(0.15)+(10)^2(0.05)\right] - (-0.15)^2\\ &\doteq& 17.9275\\ &&\\ SD(X) &\doteq& \sqrt{17.9275}\\ &\doteq& 4.234088 \end{array}$$ One should be careful to not forget to subtract $\mu^2 = (E(X))^2$ at the end of the calculation of $Var(X)$ -- this is a common mistake among students first learning this calculation.

Two Properties of the Variance:

The Variance of a Sum or Difference

Just as there was a simple way to find the expected value of the sum or difference of two discrete random variables (i.e., $E(X \pm Y) = E(X) \pm E(Y)$). one can find a similar (but slightly different) way to find the variance of a sum or difference of two discrete random variables.

Specifically,

Note the differences between this and the related property regarding the expected value. First, we require that $X$ and $Y$ are independent. Second, the expression on the right is always a sum of two variances, even when finding the variance of a difference of two random variables.

To see why this property holds, again suppose both $X$ and $Y$ are discrete random variables with outcome spaces $S_x = \{x_1, x_2, \ldots\}$, and $S_y = \{y_1, y_2, \ldots\}$, respectively, and then consider the following: $$\begin{array}{rcl} Var(X \pm Y) &=& E[(X \pm Y)^2] - (\mu_{X \pm Y})^2\\\\ &=& E[(X \pm Y)^2] - (\mu_X \pm \mu_Y)^2\\\\ &=& E[X^2 \pm 2XY + Y^2] - (\mu_{X}^2 \pm 2\mu_{X}\mu_{Y} + \mu_{Y}^2)\\\\ &=& E(X^2) \pm 2E(XY) + E(Y^2) - \mu_{X}^2 \mp 2\mu_{X}\mu_{Y} - \mu_{Y}^2\\\\ &=& [E(X^2) - \mu_{X}^2] \pm 2[E(XY) - \mu_{X}\mu_{Y}] + [E(Y^2) - \mu_{Y}^2]\\\\ &=& Var(X) \pm 2[E(XY) - \mu_{X}\mu_{Y}] + Var(Y) \end{array}$$ However, note that $$\begin{array}{rcl} E(XY) &=& \displaystyle{\sum_{x \in S_x, \, y \in S_y} x y \cdot P(X = x \textrm{ and } Y = y)}\\\\ &=& \displaystyle{\sum_{x \in S_x, \, y \in S_y} x y \cdot P(x)P(y) \quad \quad \textrm{(as $X$ and $Y$ are independent)}}\\\\ &=& \displaystyle{\sum_{x \in S_x, \, y \in S_y} xP(x) \cdot yP(y)}\\\\ &=& \displaystyle{\left( \sum_{x \in S_x} xP(x) \right) \left( \sum_{y \in S_y} yP(y) \right)}\\\\ &=& \displaystyle{\mu_{X}\mu_{Y}} \end{array}$$ Thus, the middle term in the expression for $Var(X \pm Y)$ above (i.e., $2[E(XY) - \mu_X \mu_Y]$) is zero, and $$Var(X \pm Y) = Var(X) + Var(Y)$$

The Variance of a Constant Multiple of a Random Variable

Using the properties of expected value, we can also show the following:

To see this, consider the following: