Bayes’ Theorem – Definition, Formula and Applications

What is Bayes’ Theorem?

Definition: Bayes’ theorem is defined as a theorem that helps us calculate the probabilities of the event as per the prior knowledge of the conditions that might be associated with the event.

Bayes’ theorem is a very famous rule followed in probability and statistics to determine the conditional probability of an event. It gives the method followed to determine the probability of occurrence of an event given that a particular event has already occurred.

For example, finding the probability of drawing a red-colored ball a second time from a bag containing three red and three black balls, given that a red ball was drawn and replaced the first time. Also, your chances of getting a parking space are related to the time of day as well as the place where you park, etc.

It gives us a rather simple formula to calculate conditional probability. For example, if there are two events A and B, and the total probabilities of both of these happening are P(A) and P(B) respectively.

Then, the probability of event A occurring, given that event B has also occurred is denoted by P(A|B).

All in all, Bayes’ theorem helps you in having the actual probability of an event based upon the given information about tests.

Here, events are quite different from the tests like when you go for a kidney disease test, it would be different from the event of having a kidney disease. In addition, different tests can also be flawed like if an individual has given a positive test, it does not confirm that he or she actually has the disease.

There can be rare events of tests having high false-positive rates. In such situations, Bayes’ theorem would take the test results and check the real probability that the test has identified the event accurately or not. Let us delve deeper into the world of this theorem and understand what it is and how it works-

What is the formula of Bayes’ Theorem?

The formula used for finding this conditional probability is given by Bayes’ theorem and is as mentioned.

P(A)= The probability of A occurring

P(B)= The probability of B occurring

P(A? B)=The probability of A given B

P(B? A)= The probability of B given A

P(A? B))= The probability of both A and B occurring

Bayes’ Theorem can also be expressed in a variety of forms for specific purposes. One of its quite popular versions is associated with Rudolf Carnap’s relevance quotient or probability ratio (Carnap 1962, 466).

This is the factor PR(H, E) = PE(H)/P(H)

In this special form of the theorem, the unconditional probability of H is supposed to be multiplied to get its probability conditional on E. It suggests that Bayes’ Theorem is similar to a simple symmetry principle for probability ratios

Where to use the Theorem of Bayes?

As stated above, this theorem is used to find the conditional probability of an event given another event. So, take as an example the two events:

• A= rainy clouds are there in the sky
• B= It rains that day

Then, we can define

• P(A)= probability that there are rainy clouds in the sky=0.2
• P(B)= probability that it rains on a day=0.6
• P(A|B)= probability of rainy clouds in the sky given that it rained that day=0.9
• P(B|A)= probability of raining on a day when there were clouds in the sky

The above two are conditional probabilities. To calculate any one of these, you need to know the other and the individual probabilities of both. Then you can apply the theorem to get the conditional probability that you require.

For example, to calculate P(B|A), you need P(A|B), P(A), and P(B). You can then apply the Bayes’ theorem as follows:

P(B|A) = P(A|B) * P(B) / P(A)

= 0.9*0.6/0.2

=0.27

This shows us that the probability of raining on a day given that there were clouds in the sky is 0.27.

Applications

The theorem of Thomas Bayes can be applied to various problems. It can be used to determine the accuracy of a test given the other set of probabilities required.

It requires prior probability distribution to find out the posterior probability. Prior probability is the probability before any new data is collected. On the other hand, Posterior is the revised probability of that event knowing the new information.

In simpler words, the posterior probability is P(A|B), i.e., the probability of A given that B has already happened before the experiment.

Its formula is used to see how the probability of an event’s occurrence is affected due to some given constraints and events that have already happened.

Naming the terms used in Theorem of Thomas Bayes

The various terms used in this theorem have been appropriately named. Here are those.

• P(A|B)= Posterior probability: This is the conditional probability that we need to find.
• P(A)= Prior probability: This is the probability that we have before the experiment.
• P(B|A)= Likelihood
• P(B)= Evidence

Thus, we can restate the Bayes’ theorem as

Posterior = Likelihood * Prior / Evidence

The denominator

In the formula of Bayes’ Theorem, you can see that the denominator is the marginal probability of an event. This is also called the total probability of the event, which means that it is the probability of that event occurring under every circumstance.

The total probability of the event is often unknown; what is known is the conditional probabilities of that event happening given various constraints.

• The total probability can then be found out using the formula given below.

P(A)= P(A|E1)+P(A|E2)+P(A|E3)….

Where

E1, E2, E3 are events that might have happened in the past.

• Or it can be found out using

P(A) = P(A|B) * P(B) + P(A|not B) * P(not B)

If all the required conditional probabilities are known

For example,

Two Balls are drawn from a bag containing three black, three red, and three yellow balls. Then the probability of picking up a red ball the second time will be P(R), and it will be found out by following these steps:

• P(R|R’)= probability of second ball being red given the first ball was black or yellow
• P(R|R)= probability of the second ball being red given the first ball is red.

Then the total probability of the second ball being red given by

P(R)= P(R|R’)+P(R|R)

Numerically,

P(R|R’)=?

P(R|R)=¼

So, the total probability of drawing a red ball becomes:

P(R)= ?+¼= ?

Real-life Examples of Medical Tests using Theorem of Bayes

Here we give you an example of how Bayes’ theorem can be applied to real-world scenarios. The given example is a widespread area where the theorem is used.

Generally, the medical tests for the diagnosis of various diseases are not 100% accurate. It may happen that a person might be suffering from a disease, and still, the test shows the reports negatively. Or it might so happen that the person is not suffering from the disease, yet the test shows the results to be positive. – Means having confusion around false positive, false negative, or true positive when you have given a positive test.

Yes, it sounds horrifying, but it is a fact that nothing is 100% perfect, and neither are these tests!

Here, using the data related to a hypothetical testing device for HIV, we have the various probabilities required to find out the device’s accuracy.

We know that the testing device sometimes gives the wrong result, but we need to find if the number of such wrong results is very high to question the reliability of the test? So, here we will be finding out the probability that a person who tested positive is favorable or not.

The testing device gives the reports correctly for 90% of the positive cases, while the rest 10% positive cases go unnoticed by it. Further, it can judge the person negatively if he is negative 99% of the time. But, 1% of people test positive even though they are harmful. We also know that only 0.1% of the total population is suffering from the disease. Now, a person is selected at random from a large population and is tested positive. We need to find the probability of the person being HIV positive.

Let E be the event of the person turning out to be HIV positive. E’ is thus, the event of him being HIV negative.

Let A be the event of the test report coming out to be positive.

We thus require the probability of a person who has HIV, given that he has been tested positive. This means we need to find P(E|A).

Now, we have the following probabilities.

P(E)= 0.1%=0.001=probability of suffering from HIV

P(E’)=0.999=probability of not suffering from HIV

P(A|E)=99%=0.99= probability of person suffering from HIV given that his reports diagnosed him positively as well.

P(A|E’)= 1%=0.01= probability of person not suffering from HIV but reports tested him positively.

Therefore using Bayes’ theorem,

 P(E|A)={P(E)P(A|E)}/{ P(E)P(A|E)+ P(E )P(A|E )}

=0.083 (approx)

Thus, the probability that a person randomly chosen and tested positive for HIV is 0.083, which is only 8.3%. This clearly shows that the test is not acceptable and needs to be stopped from being used for the diagnosis of the disease.

Such calculation of probabilities related to diseases is usually carried out to determine the usability of the devices. Bayes’ theorem indeed plays a pivotal role here. But, this isn’t the only area where Bayes’ Theorem is used.

Final Thoughts!

Bayes theorem is a very common theorem used in machine learning to make a prediction based on the data that we previously have. It also helps classify the data into various categories, again through machine learning techniques.

Well, it isn’t only used for high-level scientific calculations. It is also used for some different studies. Ever wondered how those internet dating sites bring you the most suitable matches?

Well, they use the Bayes’ theorem to identify your chances of matching with the person, given that they already know the qualities you preferred in the past!

Bayes’ theorem is also used in environmental studies to predict various consequences of an activity. Predicting Water quality conditions, understanding the effects of an anti-pollution campaign, what factor contributes to the pollution, actually predicting pretty much everything involves the usage of Bayes’s theorem. It thus forms a crucial part of probability theory.

What are your thoughts about the importance of the theorem given by Thomas Bayes in calculating the probability of an event?