Probability (Expectation)

Abdullah Al Mahmud

Expectation vs AVerage (AM)

Accident Frequency
(f)
Relative
Frequency
(rf)
Bus 70 0.56
Train 35 0.28
Ship 20 0.16
Total 125 1
  • Do rfs look like probabilities?
  • What is the probability that an accident is ocurred in a Bus service?

Weighted Mean Revisited

Daily
Income
(\(x\))
Frequency
(\(f\))
Relative
Frequency
(\(rf\))
\(x \times f\) \(x \times rf\)
400 70 0.56 28000 224
500 35 0.28 17500 140
800 20 0.16 16000 128
Total 125 1 61500 492

If a man is randomly selected, what is the probability that his daily income is 400 BDT?

  • \(AM = \frac{61500}{125}=492\)
  • In expectation, \(rf\) is the weight

Expectation from Mean

\(\displaystyle \bar X=\frac{\Sigma x_if_i}{\Sigma f_i}\)

  • \(= \frac{x_1f_1+x_2f_2+\cdots+x_3f_3}{N}\) (letting \(\Sigma f_i=N\))
  • \(= x_1\frac{f_1}{N}+x_2\frac{f_2}{N}+\cdots +x_2\frac{f_n}{N}\)
  • \(= x_1p_1+x_2p_2+\cdots+x_np_n\)
  • \(= \sum x_i\cdot p(x_i)\)

Expectation

Probabilities are weights

\[\begin{eqnarray} E(X) &=& x_1 \times P(x_1) + x_2 \times P(x_2)+ \cdots +x_n \times P(x_n) \nonumber \\ &=& \displaystyle \sum_{i=1}^n x_i \times P(x_i) \nonumber \\ \end{eqnarray}\]

For continuous X
  • \(E(X^2) = \sum x^2 \times p(x)\)

Expectation Example

\(X\) 0 1 2
\(P(x)\) \(\frac 1 4\) \(\frac 1 2\) \(\frac 1 4\)

Find

  1. \(E(X)\)
  2. \(E(X^2)\)

Properties of Expectation

Aslo refer to the properties of AM (\(E(X)=\bar X\))

  • Expectation of a constant, \(E(c)=c\)
  • \(E(aX)=aE(X)\)
  • \(E(X-a) = E(X) - E(a)\)
  • \(E(aX+b) = aE(X)+b\)
  • \(E(X+Y) = E(X)+E(Y)\)
  • \(E(XY) = E(X) E(Y)\) (if independent, relate with probability)
  • \(E(X^2)\ge \{E(X)\}^2\)
  • \(E\left(\frac 1 X \right) \ge \frac 1 {E(X)}\)

Variance

Recall this (click)

We knew, \(\sigma^2 = \bar {X^2} - (\bar X)^2=\) Mean of square - square of mean

  • \(V(X)=E(X^2)-\{E(X)\}^2\)
  • Original Formula: \(V(X) = E\{x-E(X)\}^2\) (Match)
  • Expand and prove these are equal

Variance Properties

  • \(V(c)=0\) ponder, why?
  • \(V(X+a)=V(X)\)
  • \(V(aX+c) = a^2V(X)\) (depends only on scale, recall)
  • \(V(X+Y) = V(X)+V(Y)\)

Covariance

\(Cov(X,Y) = \frac{\Sigma (x-\bar x)(y-\bar x)}{n}\)

Write in terms of \(E(X)\)

  • \(Cov(X,Y) = E [\{x-E(X)\}\{y-E(Y)\}]\) (Expand)
  • \(E(XY)-E(X)E(Y)\)

Prove E(X) Properties

Go back to see the properties

E(a)

\(E(a) = \sum a \cdot p(x_i)\)

  • \(=a \cdot \sum p(x_i)\)
  • \(=a \times 1 = a\)
  • Others can be proven similarly

Double Summation Revisited

Exam (X) \(\to\)
Result (Y) \(\downarrow\)
PSC JSC SSC HSC Total
Passed 30 26 23 25 104
Failed 12 13 10 14 49
Absent 5 2 3 4 14
Total 47 41 36 43 167
  • If sum over \(x_i\), we get only 1 column at a time
  • If sum over \(y_i\), we get only 1 row at a time
  • SO to get grand total, we must use \(\sum \sum (x_i+y_j)\)

E(X+Y) & E(XY)

\(E(X+Y)\)

  • \(= \displaystyle \sum_{i=1}^m \sum_{j=1}^n (x_i+y_j)P(x_i,y_j)\)
  • \(=\displaystyle \sum_{i=1}^m \sum_{j=1}^n \{x_iP(x_i,y_j)+y_jP(x_i,y_j)\}\)
  • \(=\displaystyle \sum_{i=1}^m \sum_{j=1}^n x_iP(x_i,y_j)+\sum_{i=1}^m \sum_{j=1}^n y_jP(x_i,y_j)\)
  • \(=\displaystyle \sum_{i=1}^m x_i\sum_{j=1}^n P(x_i,y_j)+\sum_{j=1}^n x_i\sum_{i=1}^m P(x_i,y_j)\)
  • \(=\displaystyle \sum_{i=1}^m x_i P(x_i) + \sum_{j=1}^n y_j P(y_j)\)
  • \(=E(X)+E(Y)\)

E(X2) ≥ {E(X)}2

\(V(X)\ge 0\)

  • \(\therefore E(X^2) - \{E(X)\}^2 \ge 0\)

AM & HM

\(E(\frac 1 X) \ge \frac 1 {E(X)}\)

  • \(AM \ge HM\)
  • \(HM =\) opposite of mean of opposite \(\to \frac{1}{\frac{\sum \frac 1 {x_i}}{n}} = \frac 1 {E(\frac 1 x)}\)
  • \(E(X) \ge \frac{1}{E(\frac 1 x)}\)
  • If \(0.5 \gt \frac 1 3 \to 3 \gt \frac 1 {0.5}\)

Variance of constant (a)

\(V(X) = E\{X-E(X)\}^2\)

  • \(V(a) = E\{a-E(a)\}^2\)
  • \(E(a-a)^2\)
  • \(E(0)=0\)

V(aX+b)

\(V(X) = E\{X-E(X)\}^2\)

\(V(aX+b)=\)

  • \(E\{(aX+b)-E(aX+b)\}^2\)
  • \(E\{aX+b-aE(X)-b\}^2\)
  • \(E\{aX-aE(X)\}^2\)
  • \(a^2E\{X-E(X)\}^2\)
  • \(a^2V(X)\)
  • Does variance depend on origin and scale?
  • \(V(X+Y)=?\)

Cov(X,Y) Properties

For independent X, Y

  • \(\to E(XY) = E(X)E(Y)\)
  • \(Cov(X,Y) = E(XY) - E(X)E(Y) = 0\)
  • Correlation, \(r = \frac{Cov(X,Y)}{\sigma_x \sigma_y}=0\)

Example 01

x -3 -2 -1 0 1 2
P(x) k 2k 3k 2k 4k 0.4
  1. Find k
  2. Find \(E(X)\) and \(V(X)\)

Example 02

\(P(x) = \frac 1 n; x = 1,2,3, \cdots,n\) Find \(E(X)\) and \(V(X)\)

Example 03

\(P(x)=\frac {3-|4-x|} k; x = 2,3,4,5,6\)

Find

  1. k
  2. V(2X-1)