The Important Theorem of Thomas Bayes

Thomas Bayes (1701 – 1761)

On April 17, 1761, English mathematician and Presbyterian minister Thomas Bayes passed away. He is best known as name giver of the Bayes’ theorem, of which he had developed a special case. It expresses (in the Bayesian interpretation) how a subjective degree of belief should rationally change to account for evidence, and finds application in in fields including science, engineering, economics (particularly microeconomics), game theory, medicine and law.

“[My favourite fellow of the Royal Society is the Reverend Thomas Bayes, an obscure 18th-century Kent clergyman and a brilliant mathematician who] devised a complex equation known as the Bayes theorem, which can be used to work out probability distributions. It had no practical application in his lifetime, but today, thanks to computers, is routinely used in the modelling of climate change, astrophysics and stock-market analysis. “
— Bill Bryson about Thomas Bayes, as quoted in [12]

Early Years

Thomas Bayes was born into a prominent family from Sheffield in 1701 and enrolled at the University of Edinburgh in 1719. He began studying logic and theology, assisting his father at the non-conformist chapel in London. He was ordained in 1727 and moved to Box Lane Chapel, Bovington, about 25 miles from London. In later years, Bayes became minister of the Mount Sion chapel.

Theology and Mathematics

We do know that in 1719 Bayes matriculated at the University of Edinburgh where he studied logic and theology. He had to choose a Scottish university if he was to obtain his education without going overseas since, at this time, Nonconformists were not allowed to matriculate at Oxford or Cambridge.[5] Even though Bayes is known to have published only two major works, they were quite influential. The first, published in 1731, was titled ‘Divine Benevolence, or an Attempt to Prove That the Principal End of the Divine Providence and Government is the Happiness of His Creatures‘ was rather dedicated to the field of theology. Five years later however, Bayes published anonymously ‘An Introduction to the Doctrine of Fluxions, and a Defence of the Mathematicians Against the Objections of the Author of the Analyst‘. This work was rather mathematical and in it, Bayes defended Isaac Newton‘s theories on calculus and its logical foundations.[6]

Fellow of the Royal Society

It is speculated that Bayes was elected as a Fellow of the Royal Society in 1742 on the strength of the Introduction to the Doctrine of Fluxions, as he is not known to have published any other mathematical works during his lifetime. In following years, Bayes’ interest in probability theories grew and his interesting ideas and findings were collected in his manuscripts and most of them only became known after his passing.

Posthumous Fame with Probability Theory

Unfortunately, Bayes’ most important and most influential work was published after his death. The ‘Essay towards solving a Problem in the Doctrine of Chances‘ was read to the Royal Society in 1763. The work contained a statement of a special case of probability now called Bayes’ theorem. The theorem can be seen as a “way of understanding how the probability that a theory is true is affected by a new piece of evidence”. Through the years, it has been helpful in a variety of scientific fields, and is often used to clarify the relationship between theory and evidence. Richard Price, who once published Bayes’ papers after his death believed that Bayes’ theorem could prove the existence of God. This technically combines Bayes’ research fields in mathematics and theology and Price’s claim opened up a wide field of discussion for future scientists and it still depicts a debate topic up to this day.

Bayes’ Theorem

In probability theory and statistics, Bayes’ theorem describes the probability of an event, based on prior knowledge of conditions that might be related to the event. For example, if cancer is related to age, then, using Bayes’ theorem, a person’s age can be used to more accurately assess the probability that they have cancer, compared to the assessment of the probability of cancer made without knowledge of the person’s age. One of the many applications of Bayes’ theorem is Bayesian inference, a particular approach to statistical inference. When applied, the probabilities involved in Bayes’ theorem may have different probability interpretations. With the Bayesian probability interpretation the theorem expresses how a subjective degree of belief should rationally change to account for availability of related evidence. Bayesian inference is fundamental to Bayesian statistics. After Bayes initial formulation, the theory was further developed by Pierre-Simon Laplace,[7] who first published the modern formulation in his 1812 “Théorie analytique des probabilités“. Sir Harold Jeffreys put Bayes’ algorithm and Laplace’s formulation on an axiomatic basis. Jeffreys wrote that Bayes’ theorem “is to the theory of probability what the Pythagorean theorem is to geometry“

A rigorous introduction to probability theory: Lecture 1 with Michal Fabinger, [11]

References and Further Reading:

• [1] Bayes Theorem at Trinity University
• [2] Bayes Theorem at the Stanford Encyclopedia of Philosophy
• [3] International Society for Bayesian Analysis
• [4] Thomas Bayes at zbMATH
• [5] John J. O’Connor, Edmund F. Robertson: Thomas Bayes. In: MacTutor History of Mathematics archive
• [6] Standing on the Shoulders of Giants – Sir Isaac Newton, SciHi Blog
• [7] Pierre Simon de Laplace and his true love for Astronomy and Mathematics, SciHi Blog
• [8] Thomas Bayes at Wikidata
• [9]  The Reverend Thomas Bayes, F.R.S. — 1701?–1761. Who Is this gentleman? When and where was he born?
• [10] D. R. Bellhouse, 2004, “The Reverend Thomas Bayes, FRS: A Biography to Celebrate the Tercentenary of His Birth,“ Statistical Science 19 (1): 3–43.
• [11] A rigorous introduction to probability theory: Lecture 1 with Michal Fabinger, MLT Artificial Intelligence @ youtube
• [12] Max Davidson, ‘Bill Bryson: Have faith, science can solve our problems‘, Daily Telegraph (26 Sep 2010)
• [13] Timeline of Probability Theorists, via DBpedia and Wikidata