---
jupytext:
formats: md:myst
text_representation:
extension: .md
format_name: myst
kernelspec:
display_name: Python 3
language: python
name: python3
---
# Random variables
Recommended reference: {cite:t}`wasserman`, Sections 1.1–1.3,
2.1–2.2 and 3.1–3.3.
## Introduction
A **probability distribution** assigns *probabilities* (real numbers
in $[0,1]$) to elements or subsets of a *sample space*. The elements
of a sample space are called *outcomes*, subsets are called *events*.
The space of outcomes is usually of one of two kinds:
- some finite or countable set (modelling the number of particles
hitting a detector, for example), or
- the real line, a higher-dimensional space or some subset of one of
these (modelling the position of a particle, for example).
These correspond to the two types of probability distributions that
are usually distinguished: *discrete* and *continuous* probability
distributions.
## Random variables
A **random variable** is any real-valued function on the space of
outcomes of a probability distribution. Random variables can often be
interpreted as observable (scalar) quantities such as length,
position, energy, the number of occurrences of some event or the spin
of an elementary particle.
Any random variable has a *distribution function*, which is how we
usually describe random variables. We will look at distribution
functions separately for discrete and continuous random variables.
## Discrete random variables
A discrete random variable $X$ can take finitely many or countably
many distinct values $x_0,x_1,x_2,\ldots$ in $\RR$. It is
characterised by its *probability mass function* (PMF).
:::{prf:definition} Probability mass function
The *probability mass function* of the discrete random variable $X$
is defined by
$$
f_X(x) = P(X = x).
$$
:::
One can visualise a probability mass function by placing a vertical
bar of height $f_X(x)$ at each value $x$, as in the [figure
below](pmf).
```{code-cell} ipython3
:tags: [hide-input, remove-output]
from matplotlib import pyplot
from myst_nb import glue
fig, ax = pyplot.subplots()
for x, px in ((-0.2, 0.3), (0.4, 0.2), (0.7, 0.5)):
ax.add_line(pyplot.Line2D((x, x), (0, px), linewidth=2))
ax.set_xbound(-0.3, 0.8)
ax.set_ybound(0, 0.6)
ax.set_xlabel('$x$')
ax.set_ylabel('$f(x)$')
glue("pmf", fig)
```
(pmf)=
:::{glue:figure} pmf
Probability mass function of a discrete random variable.
:::
:::{prf:property} Properties of a probability mass function
Since the $f_X(x)$ are probabilities, they satisfy
$$f_X(x) \in [0,1]\quad\text{for all }x\in\RR.
$$
Since the $x_i$ are all the possible values and their total
probability equals 1, we also have
$$
\sum_{i=0}^\infty f_X(x_i) = 1.
$$
:::
## Continuous random variables
For a continuous random variable $X$, the set of possible values is
usually a (finite or infinite) interval, and the probability of any
single value occurring is usually zero. We therefore consider the
probability of the value lying in some interval. This can be
described by a *probability density function* (PDF).
:::{prf:definition} Probability density function
The *probability density function* of the continuous random variable
$X$ is a function $f_X\colon\RR\to\RR$ such that
$$
P(X\in[a,b]) = \int_a^b f_X(x) dx.
$$
:::
To visualise a continuous random variable, one often plots the
probability density function, as in the [figure below](pdf).
:::{prf:property} Properties of a probability density function
- $f_X(x)\ge0$ for all $x$;
- $\int_{-\infty}^\infty f(x)dx=1$.
:::
```{code-cell} ipython3
:tags: [hide-input, remove-output]
from matplotlib import pyplot
from myst_nb import glue
import numpy as np
x = np.linspace(0, 16, 101)
fx = 1/120 * x**5 * np.exp(-x)
fig, ax = pyplot.subplots()
ax.plot(x, fx)
ax.set_xbound(0, 16)
ax.set_ybound(0, 0.2)
ax.set_xlabel('$x$')
ax.set_ylabel('$f(x)$')
glue("pdf", fig)
```
(pdf)=
:::{glue:figure} pdf
Probability density function of a continuous random variable.
:::
## Expectation and variance
The *expectation* or (*mean*) of a random variable is the average
value of many samples. The *variance* and the related *standard
deviation* measure by how much samples tend to deviate from the
average.
:::{prf:definition} Expectation
The *expectation* or *mean* of a discrete random variable $X$ with
probability mass function $f_X$ is
$$
E(X) = \sum_x f_X(x) x = \sum_x P(X = x)x.
$$
The *expectation* or *mean* of a continuous random variable $X$
with probability density function $f_X$ is
$$
E(X) = \int_{-\infty}^\infty f_X(x) xdx.
$$
:::
The expectation of $X$ is often denoted by $\mu$ or $\mu(X)$.
:::{prf:definition} Variance and standard deviation
The *variance* of a (discrete or continuous) random variable $X$
with mean $\mu$ is
$$
\Var(X) = E((X-\mu)^2).
$$
The *standard deviation* of $X$ is
$$
\sigma(X) = \sqrt{\Var(X)}.
$$
:::
It is not hard to show (see {ref}`variance`) that
$$
\Var(X) = E(X^2) - E(X)^2.
$$(eq-variance)
***
## Exercises
:::{exercise}
Show that the function
$$
f(x) = \begin{cases}
\displaystyle\frac{x^2}{30}& \text{if }x=1,2,3,4\\
0& \text{otherwise}
\end{cases}
$$
is a probability mass function.
:::
:::{exercise}
Which of the following functions are probability density functions?
1. $f(x)=\begin{cases} 0& \text{if }x<0\\
x\exp(-x)& \text{if }x\ge 0\end{cases}$
2. $f(x)=\begin{cases} 1/4& \text{if }-2\le x\le 2\\
0& \text{otherwise}\end{cases}$
3. $f(x)=\begin{cases} \frac{3}{4}(x^2-1)& \text{if }-2\le x\le 2\\
0& \text{otherwise} \end{cases}$
:::
:::{exercise}
:label: variance
Deduce {eq}`eq-variance` from the definition of the variance.
:::
:::{exercise}
Consider a continuous random variable $X$ with probability density
function
$$
f(x)=\begin{cases} 0& \text{if }x<0,\\
\exp(-x)& \text{if }x\ge 0.\end{cases}
$$
Compute the expectation and the variance of $X$.
:::