The Pentium FDIV Bug

66 MHz Intel Pentium (sSpec=SX837) with the FDIV bug

On October 30, 1994, Thomas Nicely, a professor of mathematics at Lynchburg College, published his findings about a serious bug in the arithmetic unit of Intel’s latest Pentium processor, known as the Pentium FDIV Bug. Because of the bug, the processor can return incorrect decimal results, an issue troublesome for the precise calculations needed in fields like math and science.

How the FDIV Bug was discovered

The Pentium FDIV bug is the most famous (or infamous) of the Intel microprocessor bugs. Thomas Nicely, a professor of mathematics at Lynchburg College, had written code to enumerate primes, twin primes, prime triplets, and prime quadruplets. Nicely noticed some inconsistencies in the calculations on June 13, 1994, shortly after adding a Pentium system to his group of computers, but was unable to eliminate other factors until October 19, 1994. On October 24, 1994, he reported the issue to Intel. According to Nicely, his contact person at Intel later admitted that Intel had been aware of the problem already since May 1994, when the flaw was discovered by Tom Kraljevic, a Purdue co-op student working for Intel in Hillsboro, OR, during testing of the FPU for its new P6 core, first used in the Pentium Pro.

“Usually mathematicians have to shoot somebody to get this much publicity.”
– Thomas Nicely, Cincinnati Enquirer, December 18, 1994, Section A, page 19.[1]

…and verified

On October 30, 1994, after receiving no more information from Intel, Nicely sent an email describing the error he had discovered in the Pentium floating point unit to various contacts, requesting reports of testing for the flaw on 486-DX4s, Pentiums and Pentium clones. This flaw in the Pentium FPU was quickly verified by other people around the Internet, and became known as the Pentium FDIV bug (FDIV = floating-point division). One example was found where the division result returned by the Pentium was off by about 61 parts per million. The story first appeared in the press on November 7, 1994, in an article in Electronic Engineering Times, subsequently picked up by CNN in a segment aired on November 21, 1994, which brought it into widespread public prominence.

Intel’s Answer

Publicly, Intel acknowledged the floating-point flaw, but claimed that it was not serious and would not affect most users. Intel offered to replace processors to users who could prove that they were affected. However, although most independent estimates found the bug to be of little importance and would have negligible effect on most users, it caused a great public outcry.

Explaining the Bug

Actually, the problem occurs only on some models of the original Pentium processor. Any Pentium family processor with a clock speed of at least 120 MHz is new enough not to have this bug. Using the basic Microsoft Windows calculator, a Pentium user could check for the presence of the flaw by performing the following calculation:

(4195835 * 3145727) / 3145727

The expected result of dividing a number by itself is one, so the equation above should yield a result of 4,195,835 but the flawed Pentium Floating Point Unit (FPU) produced a value of 4,195,579; an error of 0.006%. Not all calculations performed by the FDIV instruction on a Pentium CPU were incorrect however. The occurrence and degree of inaccuracy of the floating point division calculations were highly dependent on the input data and specific divide instruction used, and in most cases, the flaw was not apparent at all. According to Intel Corp., the flaw would only be encountered once every 27,000 years under normal use, although other groups have produced significantly different failure rates.[4]

My Personal Relation with the FDIV Bug

Actually I have also a personal relation to the FDIV bug. I started my PhD in 1996 in the field of formal verification, a rather specialized section of theoretical computer science. Formal verification aims to proof that e.g. a given hardware design of a computer chip does match the original logical design, i.e. it proofs formal correctness. The Intel FDIV bug gave this branch of computer science an extraordinary boost, since it showed that to proof the correctness of a microprocessor design can protect you from a lot of troubles…including saving money. The traditional way of verification is performed via (partial) testing and application of statistics, simply because exhaustive testing of a let’s say 64 bit microprocessor would require 2**64 single test runs, which computes to about 18 quintillion, a number with 20 digits…which roughly comes to approximately 635 years of computation time if you are able perform 1 billion tests per second. Thus, formal proof methods became ‘en vogue’, i.e. there must exist an enormous number of scientific papers which cite the Pentium FDIV bug in their problem motivation section…

Demo of the Pentium FDIV Bug, [5]

References and Further Reading:

• [1] Thomas Nicely quotes at the Mathematical Association of America
• [2] Vince Emery: The Pentium Chip Story: An Internet Learning Experience
• [3] Thomas R. Nicely: Pentium FDIV flaw
• [4] Failing like never before – The Intel FDIV Bug
• [5] Demo of the Pentium FDIV Bug, Retrocet @ youtube
• [6] Edelman, Alan (January 1, 1997). “The Mathematics of the Pentium Division Bug”. SIAM Review. 39 (1): 54–67.
• [7] Alexander Wolfe (November 9, 1994). “Intel fixes a Pentium FPU glitch”. Electronic Engineering Times.