Bayes' theorem

ramyani

Bayes' theorem

Bayes' theorem is a result in probability theory that relates conditional probabilities. If A and B denote two events, P(A|B) denotes the conditional probability of A occurring, given that B occurs. The two conditional probabilities P(A|B) and P(B|A) are in general different. Bayes theorem gives a relation between P(A|B) and P(B|A).

$P(A|B) = rac{P(B | A), P(A)}{P(B)}.$

The theorem may be paraphrased as

$mbox{posterior} = rac{mbox{likelihood} imes mbox{prior}} {mbox{normalizing constant}}$

Example #1: Conditional probabilities

Suppose there are two bowls full of cookies. Bowl #1 has 10 chocolate chip cookies and 30 plain cookies, while bowl #2 has 20 of each. Fred picks a bowl at random, and then picks a cookie at random. We may assume there is no reason to believe Fred treats one bowl differently from another, likewise for the cookies. The cookie turns out to be a plain one. How probable is it that Fred picked it out of bowl #1?

Intuitively, this should be greater than half since bowl #1 contains the same number of cookies as bowl #2, yet it has more plain.

We can clarify the situation by rephrasing the question to "what?s the probability that Fred picked bowl #1, given that he has a plain cookie?? The event A is that Fred picked bowl #1, and the event B is that Fred picked a plain cookie. To compute P(A|B), we first need to know:

P(A), or the probability that Fred picked bowl #1 regardless of any other information. Since Fred is treating both bowls equally, it is 0.5.
P(B), or the probability of getting a plain cookie regardless of any other information. Since there are 80 total cookies, and 50 of them are plain, the probability of selecting a plain cookie is 50/80 = 0.625.
P(B|A), or the probability of getting a plain cookie given Fred picked bowl #1. Since there are 40 cookies in bowl #1 and 30 of them are plain, the probability is 30/40 = 0.75.

Given all this information, we can compute the probability of Fred having selected bowl #1 given that he got a plain cookie by substitution:

$P(A|B) = rac{P(B | A) P(A)}{P(B)} = rac{0.75 imes 0.5}{0.625} = 0.6$

As we expected, it is more than half.

Example #4: The Monty Hall problem

We are presented with three doors - red, green, and blue - one of which has a prize. We choose the red door, which is not opened until the presenter performs an action. The presenter who knows what door the prize is behind, and who must open a door, but is not permitted to open the door we have picked or the door with the prize, opens the green door and reveals that there is no prize behind it and subsequently asks if we wish to change our mind about our initial selection of red. What is the probability that the prize is behind the blue and red doors?

Let us call the situation that the prize is behind a given door A_r, A_g, and A_b.

To start with,

$P(A_r) = P(A_g) = P(A_b) = \frac 1 3$ ,

and to make things simpler we shall assume that we have already picked the red door.

Let us call B "the presenter opens the green door". Without any prior knowledge, we would assign this a probability of 50%

In the situation where the prize is behind the red door, the host is free to pick between the green or the blue door at random. Thus, $P (B | A r) = 1 / 2$
In the situation where the prize is behind the green door, the host must pick the blue door. Thus, $P (B | A g) = 0$
In the situation where the prize is behind the blue door, the host must pick the green door. Thus, $P (B | A b) = 1$

Thus,

$egin{matrix} P(A_r|B) & = rac{P(B | A_r) P(A_r)}{P(B)} & = rac{ rac 1 2 rac 1 3}{ rac 1 2} & = rac 1 3 \ P(A_g|B) & = rac{P(B | A_g) P(A_g)}{P(B)} & = rac{0 rac 1 3}{ rac 1 2} & = 0 \ P(A_b|B) & = rac{P(B | A_b) P(A_b)}{P(B)} & = rac{1 rac 1 3}{ rac 1 2} & = rac 2 3 end{matrix}$

*** An investigation by a statistics professor (Stigler 1983) suggests that Bayes' theorem was discovered by Nicholas Saunderson some time before Bayes.

( source : http://en.wikipedia.org/wiki/Baye%27s_Theorem )

Community Contributions - Articles by goIITians

Bayes' theorem

Example #1: Conditional probabilities