Axiomatic Approach to Probability

Hearing the word "probability" brings up ideas related to uncertainty or randomness. Although the concept of probability can be hard to describe formally, it helps us analyze how likely it is that a certain event will happen. This analysis helps us understand and describe many phenomena we see in real life. Even seemingly random processes can be explained and predicted using probability models. That's why probability forms the foundation for many artificial intelligence algorithms we use today. Before diving into the formal laws of probability, let's look at some basic terminology.

Events and Sample Space

In probability, we conduct experiments. These experiments are random, which means we cannot predict the outcomes of the same. All these outcomes constitute the sample space. Let us consider the experiment of tossing a coin two times. This experiment has four possible outcomes:

Sample space = { HH , TT , HT , T H }

• Random Experiment: A random experiment is an experiment in which outcomes are random and thus cannot be predicted with certainty. 
• Sample Space: Sample space is the set of all possible outcomes associated with a random experiment. It is denoted using the symbol S.  
• Event: An event is the set of outcomes from the total possible number of outcomes; thus, it is also called a subset of the sample space.

Let's measure the probability of getting two heads in the above experiment. Then the probability of this outcome is defined as, 

P = \dfrac{\text{Number of favourable Outcomes}}{\text{Total number of possible outcomes}}

For this case, a favorable outcome is HH, and the total number of possible outcomes is four. 

So, the event can be defined as getting two heads: Probability(Getting two heads) = \frac{1}{4}

Different Probability Approaches

The previous formula for calculating the probabilities assumes that all the outcomes are equally likely. For example, in the tossing of a fair coin. The outcomes head and tails are equally likely. So this cannot be generalized to every experiment. Initially, there were two schools of thought in probability: 

1. Classical Probability
2. Frequentist Probability

Classical Probability 

This approach assumes that all the outcomes are equally likely. If our event can happen in “n” ways out of a total of “N” ways. Then the probability can be given by, 

P(event) = \frac{n}{N}

Frequentist Probability  

This is a more general approach to calculating the probability. It does not make the assumption that all the outcomes are equally likely. When outcomes are not equally likely, we repeat the experiment many times, let's say M. Then, observe how many times that particular event occurred, let' say m. Then, calculate the empirical estimate of the probability. So, use the relation, 

P(event) =\lim_{M \to \infty} \frac{m}{M}

Both of these approaches fail to generalize well and stand up to mathematical rigor. 

The axiomatic approach to probability takes the approach of considering probability as a function associated with any event.

Axiomatic Approach

Perform a random experiment whose sample space is S, and P is the probability of the occurrence of any random event. This model assumes that P should be a real-valued function with a range between 0 and 1. The domain of this function is defined to be the power set of the sample space. If all these conditions are satisfied, then the function should satisfy the following axioms: 

Axiom 1: For any given event X, the probability of that event must be greater than or equal to 0. Thus, 

0 ≤ P(X)

Axiom 2: We know that the sample space S of the experiment is the set of all the outcomes. This means that the probability of any one outcome happening is 100 percent i.e P(S) = 1. Intuitively this means that whenever this experiment is performed, the probability of getting some outcome is 100 percent.

P(S) = 1

Axiom 3: For the experiments where we have two outcomes A and B. If A and B are mutually exclusive, 

P(A ∪ B) = P(A) + P(B)  and P(A ∩ B) = 0

Here, ∪ stands for union, ∩ stands for the intersection of two sets. This can be understood as if saying, “If A and B are mutually exclusive outcomes, the probability that either one of these events will happen is the probability of A happening plus the probability of B happening”. 

These axioms are also called Kolmogorov's three axioms. The third axiom can also be extended to several outcomes, given that all are mutually exclusive. 

Let's say the experiment has A₁, A₂, A₃, and ... A_n. All these events are mutually exclusive. In this case, the three axioms become: 

Axiom 1: 0 ≤ P(A_i) ≤ 1 for all i = 1,2,3,... n. 

Axiom 2: P(A₁) + P(A₂) + P(A₃) +.... = 1

Axiom 3: P(A₁ ∪ A₂∪ A₃ ....) = P(A₁) + P(A₂) + P(A₃) ....

Let's look at some sample problems based on these concepts. 

Sample Problems on Axiomatic Approach to Probability

Question 1: Find out the sample space “S” for a random experiment involving the tossing of three coins. 

Solution. 

We know that tossing a coin gives us either Heads or Tails. Tossing three coins will give us either triplets of either heads or tails. So, the possible outcomes can be, 

HHH, HHT, HTH, HTT, .... 

All these outcomes will constitute the sample space. 
S = {HHH, HHT, HTH, HTT, THH, THT, TTH, TTT}

Question 2: Find out the probability of getting a number 3 when a die is tossed. 

Answer: 

We know that possible outcomes when a die is tossed are, 

{1, 2, 3, 4, 5 and 6} 

We want to calculate the probability for getting a number 3. 

Number of favorable outcomes = 1 
Total Number of outcomes = 6. 

So, the probability of getting a number 3, P(3) = \frac{\text{Number of favourable Outcomes}}{\text{Total number of possible outcomes}}

P(3) =\frac{1}{6}

Question 3: Let's say a class is choosing their class captain through a random draw. The class has 30% Indian students, 50% American students, and 20% Chinese students. Calculate the probability that the chosen captain will be an Indian. 

Answer: 

Let's define an event A: Chosen captain is Indian. We know that there are only 30% Indian students in class. 

Measure of favorable outcome = 0.3 

Total Number of outcomes = 1 

So, P(A) = \frac{\text{Number of favourable Outcomes}}{\text{Total number of possible outcomes}}
P(A) =\frac{0.3}{1}
P(A) = 0.3 

So, there is a 30% probability that an Indian student will be chosen as class captain. 

Question 4: Find out the probability of getting an even number when a die is tossed. 

Answer: 

We know that possible outcomes when a die is tossed are, 

{1, 2, 3, 4, 5 and 6} 

We want to calculate the probability for getting an even number. Even number are {2,4,6}

Number of favorable outcomes = 3 

Total Number of outcomes = 6. 

So, the probability of getting an even number, P(Even) = \frac{\text{Number of favourable Outcomes}}{\text{Total number of possible outcomes}}

P(Even) =\frac{3}{6}
⇒ P(Even) = \frac{1}{2}

Question 5: Let's say we have an urn with 5 red balls and 3 black balls. We want to draw balls from this bag. Find out the probability of picking a red ball. 

Answer: 

Let's define the experiment as “Drawing a ball from the bag”. Now it is required to calculate the probability for getting a red ball. 

Number of favorable outcomes = 5 
Total Number of outcomes = 8. 

So, P(red) = \frac{5}{8}

Question 6: For the above experiment, verify that the probability of getting a red ball and the probability of getting a black ball follow the axioms of probability mentioned above. 

Answer: 

Let's define two events, 

R = Red ball is picked 
B = Black Ball is picked 

Calculate the probability for getting a red ball in the previous example,  

P(R) = \frac{5}{8}

Similarly, 

P(B) = \frac{3}{8}

Now notice that both P(R) and P(B) lie between 0 and 1. So they satisfy axiom 1. Let's verify it for second axiom. 

P(R) + P(B) 
⇒P(R) + P(B) =\frac{5}{8} + \frac{3}{8}
⇒ P(R) + P(B) = 1

Thus second axiom is also satisfied. 

We know that both of these events are mutually exclusive. 

So, P(R ∪ B) = P(Getting either a Red Ball or Black Ball) 
⇒ P(R ∪ B) = P(R) + P(B) 
⇒ P(R ∪ B) =\frac{5}{8}    + \frac{3}{8}
⇒ P(R ∪ B) = 1

Thus, all three of these axioms are satisfied. Thus, above experiment follows the axioms of the probability. 

Practice Problems on Axiomatic Approach to Probability

Question 1: For two events A and B, find the probability that exactly one of the two events occurs.

Question 2: A box contains 3 red and 4 blue socks. Find the probability of choosing two socks of the same colour.

Question 3: Alice has five toys, which are identical, and one of them is underweight. Her sister, Sesa, chooses one of these toys at random. Find the probability for Sesa to choose an underweight toy?

Question 4: Aliya selects three cards at random from a pack of 52 cards. Find the probability of drawing:

• 3 spade cards.
• One spade and two knave cards
• one spade, one knave, and one heart card.

Question 5: For three events A, B, and C, show that

• P (at least two of A, B, C occur) = P(A∩B) + P(B∩C) + P(C∩A) − 2P(A∩B∩C)
• P (exactly two of A, B, C occur) = P(A∩B) + P(B∩C) + P(C∩A) − 3P(A∩B∩C)
• P (exactly one of A, B, C occurs) = P(A) + P(B) + P(C) − 2P(A∩B) − 2P(B∩C) − 2P(C∩A) + 3P(A∩B∩C)

Question 6: In a bag of 10 balls, 4 are blue and 6 are green. What is the probability of drawing a green ball?

Question 7: A deck of cards contains 52 cards, 26 of which are red and 26 are black. What is the probability of drawing a red card?

Question 8: In a group of 50 students, 30 students like basketball, 20 like football, and 10 like both basketball and football. What is the probability that a randomly selected student likes either basketball or football (or both)?

➣ Related Articles:

• Probability in Maths
• Probability Theory
• Properties of Probability

Conclusion

The axiomatic approach to probability, introduced by Andrey Kolmogorov, provides a robust and formal framework for understanding and analyzing random events. By defining probability through a set of fundamental axioms—non-negativity, normalization, and additivity—this approach ensures consistency and clarity in the study of probability. These axioms allow for the development of various probability properties and theorems, making the axiomatic approach a cornerstone of modern probability theory. It has wide applications in fields ranging from mathematics and statistics to physics, engineering, and artificial intelligence, helping to model and predict outcomes in uncertain situations.