Relative Frequency Interpretation

The frequency interpretation is based on the following idea: Probability of an outcome is not a concept which could be assigned to an individual experiment, but rather it is assigned to a long sequence of repeated experiments. Denote ${\cal A}$ the Boolean algebra of the outcome events, and let $p:\,{\cal A}\rightarrow\left[0,1\right]$ be the probability to be interpreted in empirical terms. Performing an experiment, each of the possible outcome events does or does not occur, and this fact can be described by a suitable ``outcome function'' (classical two-valued truth function, if you want) $u:\,{\cal A}\rightarrow\left\{ 0,1\right\}$ satisfying the following conditions:

$$\begin{eqnarray*} u\left(\emptyset\right) & = & 0\\ u\left(\neg A\right) & = & ... ...)\\ u\left(A\wedge B\right) & = & u\left(A\right)u\left(B\right)\end{eqnarray*}$$

where the outcome function has value 0 if the corresponding event does not occur, and has value 1 if does.

Consider now the sequential repetitions of an experiment. Let

$$u_{1},u_{2},\ldots,u_{N},\ldots$$ (1)

be the sequence of the outcome functions we obtain. For each $N$ we define the relative frequency function as follows:

$$\nu_{N}:\, A\in{\cal A}\mapsto\nu_{N}\left(A\right)=\frac{1}{N}\sum_{i=1}^{N}u_{i}\left(A\right)\in\left[0,1\right]$$ (2)

One can easily prove that if the sequence

$$\nu_{1},\nu_{2},\ldots,\nu_{N},\ldots$$ (3)

is pointwise convergent, then the function $p=\lim_{N\rightarrow\infty}\nu_{N}$ is a probability function on ${\cal A}$, satisfying the Kolmogorov axioms.⁴

Here we encounter the first difficulty of the Frequency Interpretation: the limit of an infinite sequence is independent of the first $N$ elements, where $N$ can be an arbitrarily large number. In other words, the observed relative frequency in any finite sample is irrelevant to the probability. How, then, we are supposed to find out what these probabilities are? Or if we have a hypotheses about the value of a probability, how can we empirically confirm this hypotheses, if there is no logical relationship between what we observe in a finite sample and the value we would like to confirm?

The second difficulty is that

$$u_{1},u_{2},\ldots,u_{N},\ldots$$

is the sequence of the real outcomes of the consecutive experiments, therefore, there is no guarantee that the sequence

$$\nu_{1},\nu_{2},\ldots,\nu_{N},\ldots$$

is convergent. Note, that the randomness of the outcomes does not guarantee the convergence. To illustrate this, consider the following simple example: We flip two coins. If the result is <Heads> & <Heads> then the outcome of the experiment is <1>, otherwise the outcome is <0>. Repeat the experiment until the relative frequency of <1> is less than 0.4. Then we change the roles of <0> and <1>, and repeat the experiment until the relative frequency of <1> becomes larger than 0.6. Then we change again, and so on. The sequence of outcomes obtained through this method is completely random, but there is no limiting value of the relative frequencies. So, if we insist that probability is nothing but limiting relative frequency, we must conclude that probabilities $p\left(<0>\right)$ and $p\left(<1>\right)$do not exist.

This conclusion is, however, counter-intuitive, because we ``know'' that in each run of the experiment probabilities $p\left(<0>\right)$ and $p\left(<1>\right)$ do exist, for instance $p\left(<1>\right)=0.25$ for a while, then it changes for $0.75$, then changes for $0.25$ again, and so on.

The above example also throws light on the third problem of the Frequency Interpretation. Namely, that it does not account for the probability of the outcome of an individual experiment, which probability, on the other hand, is a meaningful concept in our intuition.