Will the sun rise tomorrow?
Tonight's idea
You have seen the sun rise \(n\) days running. How sure should you be about tomorrow? "Certain" overreaches, and the frequency \(n/n = 1\) is what you would say after one morning too. Laplace's answer: \((n+1)/(n+2)\).
Suppose the sun rises each day with a fixed, unknown probability \(p\). Treat every \(p\) from 0 to 1 as equally likely beforehand: a uniform prior. For a given \(p\), \(n\) rises in a row has probability \(p^n\), so by Bayes' theorem the credibility of each \(p\) after the data is proportional to \(p^n\). This is the posterior.
Tomorrow's rise has probability \(p\) for each candidate, averaged with the posterior weights:
\(\displaystyle P(\text{rise}) = \frac{\int_0^1 p \cdot p^n\, dp}{\int_0^1 p^n\, dp} = \frac{1/(n+2)}{1/(n+1)} = \frac{n+1}{n+2}.\)
That is the rule of succession.
The quiet assumption: the uniform prior says that before any data, \(p = 0.01\) and \(p = 0.99\) were equally credible. Hence \(\tfrac12\) at \(n = 0\) and \(\tfrac23\) after one success: the rule acts as if you had already seen one success and one failure. Laplace added that anyone who knows why the sun rises should be far more confident than any tally.
Watch one: three heads from a strange coin
A coin from a magic shop lands heads three times out of three. Frequency says the chance of a fourth head is 1. Laplace says
\[\frac{\int_0^1 p^4\, dp}{\int_0^1 p^3\, dp} = \frac{1/5}{1/4} = \frac45.\]
The data have pushed you from \(\tfrac12\) to \(\tfrac45\), but three tosses cannot rule out a fair coin, which would produce three heads one time in eight. The posterior \(p^3\) still gives \(p = 0.5\) a weight of \(0.125\) against \(1\) for \(p = 1\), and that is where the missing \(\tfrac15\) lives.
Your turn
- Write the rule's answer for \(n = 0, 1, 10\) and \(100\), as fractions, and the last also as a decimal.
- With \(s\) successes and \(f\) failures the posterior is proportional to \(p^s(1-p)^f\). For \(s = 3\), \(f = 1\), compute \(\int_0^1 p^4(1-p)\,dp\) and \(\int_0^1 p^3(1-p)\,dp\) and show the probability of another success is \(\tfrac23\). Guess the general formula in \(s\) and \(f\), and check it against the rule.
- Take recorded history as 5,000 years, about 1.83 million days of sunrises. What probability does the rule give that the sun fails tomorrow? Give one reason the true figure should be far smaller, and one reason the coin model is the wrong model for this question at all.
- After \(n\) successes, find the probability that the next two trials both succeed, as one integral ratio. For \(n = 3\) compare it with \(\left(\tfrac45\right)^2\). Why are they different, and how is your answer related to \(\tfrac{n+1}{n+2}\) and \(\tfrac{n+2}{n+3}\)?
Stretch: a different prior
Replace the uniform prior with one proportional to \(p(1-p)\), a hump centred at \(\tfrac12\) that says extreme values are unlikely. Repeat the calculation for \(n\) successes and no failures. What rule replaces \((n+1)/(n+2)\), and how many imaginary successes and failures does this prior amount to?
Answers
- \(\tfrac12, \tfrac23, \tfrac{11}{12}, \tfrac{101}{102} \approx 0.990\). A hundred clean trials still leave about a 1% chance of a failure next time.
- \(\int_0^1 (p^4 - p^5)\,dp = \tfrac15 - \tfrac16 = \tfrac1{30}\) and \(\int_0^1 (p^3 - p^4)\,dp = \tfrac14 - \tfrac15 = \tfrac1{20}\). Ratio \(\tfrac{20}{30} = \tfrac23\). General formula \(\dfrac{s+1}{s+f+2}\): with \(f = 0\) it is \(\dfrac{n+1}{n+2}\). The prior adds one imaginary success and one imaginary failure to the count.
- \(n \approx 1{,}826{,}000\), so the chance of failure is \(\dfrac{1}{n+2} \approx 5.5 \times 10^{-7}\), about one in 1.83 million. It should be far smaller because we know the mechanism, not just the record: the Earth's rotation does not reset each night. And the coin model assumes each morning is an independent draw with a fixed unknown \(p\); sunrises are not draws from anything.
- \(\dfrac{\int_0^1 p^{n+2}\,dp}{\int_0^1 p^n\,dp} = \dfrac{n+1}{n+3}\). For \(n = 3\): \(\tfrac46 = 0.667\) against \(\left(\tfrac45\right)^2 = 0.64\). The two trials are not independent once \(p\) is unknown: a fourth success raises your estimate of \(p\), so the fifth is more likely than before. Indeed \(\tfrac{n+1}{n+2} \cdot \tfrac{n+2}{n+3} = \tfrac{n+1}{n+3}\): the rule applied twice, updating in between.
Stretch. The posterior is proportional to \(p^{n+1}(1-p)\). Then \(\int_0^1 p^{n+2}(1-p)\,dp = \tfrac{1}{n+3} - \tfrac{1}{n+4} = \tfrac{1}{(n+3)(n+4)}\) and \(\int_0^1 p^{n+1}(1-p)\,dp = \tfrac{1}{(n+2)(n+3)}\). The ratio is \(\dfrac{n+2}{n+4}\). This prior behaves like two imaginary successes and two imaginary failures; the uniform prior was one of each. The data eventually swamp either, but a stronger prior takes longer to overrule.