8 Probability
You typically know probability as chance. But what does that chance represent?
Probability is a number between 0 and 1 that describes the proportion of times the outcome would occur in a very long series of repetitions. We observe this definition of probability through the law of large numbers says that if we observe more and more repetitions of any chance process, the proportion of times that a specific outcome occurs approaches a single value.
Because of random processes, we know that that randomness is predictable in the long run. Our intuition tries to tell us random phenomena should also be predictable in the short run. However, probability does not allow us to make short-run predictions. Even with how we can tell things in the long run, probability does not tell us certainties about future outcomes. That is, past outcomes do not influence the likelihood of individual outcomes occurring in the future.
If we know how a certain process is supposed to act out through our understanding of probability, we can simulate it. A simulation is an imitation of chance behavior, based on a model that accurately reflects the situation. This means that we will not be doing what is being described, but rather choose a way to represent the situation using a box of tickets, a coin flip, a random number generator, etc.
When we describe a simulation, we want to describe the following:
Describe how you would do one simulation of the situation, using a physical medium and/or random number generator. e.g. flip a coin, choose \(n\) unique random numbers from \(a\) to \(b\), choose slips of paper out a box.
Do your described procedure many many times.
Estimate the probability in question.
Keep in mind that simulations are simulations. This means that if you’re describing how to simulate something, you absolutely cannot say to do what you’re asked to describe to simulate.
i.e. If you’re asked to describe a simulation of picking a Skittle out of a bag randomly, you cannot say that to simulate it, pick a Skittle out of a bag randomly.
8.1 Probability Rules
The sample space \(\Omega\) of a chance process is the set of all possible outcomes.
A probability model is a description of some chance process that consists of two parts: a sample space \(\Omega\) and a probability for each outcome.
An event is any collection of outcomes from some chance process. That is, an event is a subset of the sample space. Events are usually designated by capital letters, like \(A\), \(B\), \(C\), and so on.
For any event A, \(0 \leq P(A) \leq 1\)
If \(\Omega\) is the sample space in the probability model, \(P(\Omega)=1\)
In the case of equally likely outcomes, \(P(A)=\frac{\text{number of outcomes corresponding to event A}}{\text{total number of outcomes in sample space}}\)
We also have the following rules to calculate other things:
\(P(A|B)\) : the probability of event \(A\) occurring, given that event \(B\) has already occurred. (conditional probability)
Two events \(A\) and \(B\) are independent if the occurrence of one has no effect on the likelihood of the other occurring (like successive coin flips).
For independent events, \[P(A)=P(A|B) \text{ and } P(B)=P(B|A)\]
Events \(A\) and \(B\) are mutually exclusive if it’s impossible for both to occur. In one flip of a coin, heads and tails are mutually exclusive events.
For mutually exclusive events,
\[P(A\cap B)=0\]
8.1.1 Complement Rule
The complement to event \(A\) is the event that \(A\) doesn’t happen. This event is sometimes written \(AC\). In one flip of a coin, heads and tails are complementary events (assuming the coin won’t land on its side).
For complementary events,
\[P(A)+P(A^c)=1 \text{ or } P(A^c)=1-P(A)\]
8.1.2 Addition Rule
The general addition rule for any events \(A\) and \(B\) is:
\[P(A \cup B)=P(A) + P(B)-P(A \cap B)\]
For mutually exclusive events, we know that \(P(A\cap B)=0\), so we can also have the following simplified rule:
\[P(A \cup B)=P(A) + P(B)\]
8.1.3 Multiplication Rule
The general multiplication rule for any events A and B is:
\[P(A\cap B)=P(A)\cdot P(B|A)\]
For independent events \(A\) and \(B\), we know that \(P(B|A) = P(B)\), so we can also have the simplified rule:
\[P(A\cap B)=P(A)\cdot P(B)\]