If thus all events through all eternity could be repeated, one would find that everything in the world happens from definite causes and according to definite rules, and that we would be forced to assume amongst the most apparently fortuitous things a certain necessity, or, so to say, FATE.
-- Jacob Bernoulli=====================================================
A web site:
The Briefing Room of the White House home page has the latest federal government economic and social statistics as reported by federal agencies. In each case, a graph is provided and links are provided to more detailed information about how the data was collected and, in some cases, to the data itself.
<<<========<<
>>>>>==============>
John Hoffman writes that the difficulty of allowing wild cards in poker, discussed in the last Chance News, has already been observed by John Scarne in his book "Scarne on Cards", Penquin Books, first published in 1949.
Scarne considers the case of a single wild card and computes the probabilities of getting the various hands assuming this 53-card deck. He then writes:
The author brings to the attention of the reader a very unusual mathematical situation which arises in Joker Wild regarding the relative value of three of a kind and two pair."
Scarne offers two solutions for players who want to play a "mathematically sound" game: (a) When a player has joker and a pair he cannot count the joker as a wild card or (b) if the pair consists of sevens or lower he cannot value his hand as higher than two pair. Scarne rejects including either of these in his rules for Joker Wild Poker since "you cannot change habits that easily."
Emart and Umbert, in their article in Chance Magazine (Summer 1996), propose a method of ranking the hands that can be implemented for any wild card game and that they feel is better than the traditional ranking when playing with wild cards.
They define the "inclusion frequency" for each type of hand as the number of five-card hands that can be declared as this type of hand. The hands are then ranking in order of decreasing inclusion frequency. With no wild cards, this ranking agrees with the usual ranking. With wild cards, you still run into inconsistencies such as a pair of hands where you are more likely to obtain the higher ranked hand. However, this cannot be avoided by any ranking.
DISCUSSION QUESTION.
Do you think that Scarne was correct that poker players are not about to change their rules?
<<<========<<
>>>>>==============>
Tom Moore sent us the following message from a friend who wrote:
I read this really funny article once that said that in boxes of animal crackers, the prey-type animals (penguins, pigs, and other cutesey ones) were found to be broken more often than predator- type animals (lions, tigers, and bears).
These cookies called "Barnum's Animals Crackers" were started in 1902. They've been immortalized in song (Animal Crackers in my soup"), film (the 1930 Marx Brothers movie "Animal Crackers") and in a poem by Christopher Morely which begins with "Animal crackers, and cocoa to drink. That is the finest of suppers, I think."
You can find more historical information as well as the current list of animals used from Nabisco.
Incidently, you can find an interesting and up-to-date discussion
of the M&M activity in the current issue of Chance Magazine: "The
mysterious case of the blue M&M's" by Ronald D. Fricker, Jr.,
Chance Magazine, Vol. 9, No. 4, Fall 1996.
<<<========<<
>>>>>==============>
Bob Griffin sent us two items. The first was a quote that he suggested we might use but confessed that it may be more mathematical than statistical. Yogi Berra, former Yankees catcher, when asked if he wanted his pizza cut into four or eight slices, said
"Better make it four. I don't think I can eat eight."
Bob writes:
I also ran across the following article in the latest Newsweek. Their tendency to use anecdotal evidence to try to demonstrate causality and trends provides an endless source of teaching materials.Conspiracy mania feeds our growing national paranoia.
Claiming that "conspiracy paranoia is surrounding us," this article gives readers a brief, skeptical tour of some contemporary and popular conspiracy theories. The "news peg" (i.e., that which makes the article timely and important in the eyes of the reporters and editors), as reflected in the headline, is that popular belief in conspiracies is growing. "This great nation has always had its share of conspiracy freaks....But the ranks of the darkly deluded may be growing," the authors state (p. 66). "Clearly, something is heating up in the more tropical climes of the American psyche," they conclude, based primarily on the following evidence cited in the article:
Discussion Questions:
- Three quarters of Americans believe that "the government is involved in conspiracy," according to a survey reported in George magazine.
- America On Line has begun a channel for fans of the paranormal and the paranoid.
- Mel Gibson will be starring in a movie called "Conspiracy Theory."
- The editor of The Skeptical Inquirer, a publi- cation that debunks the far-out, says that: "There certainly seems to be a resurgence in sympathy toward conspiracy theory and an increasing strain of paranoia."
Based on this evidence, would you conclude that popular paranoia and belief in conspiracy is growing in this country? Why or why not? If you are skeptical, what additional evidence would you seek to find out whether the conspiracy climate of public opinion is changing?
<<<========<<
>>>>>==============>
On the peculiar distribution of the U.S. stock indexes' digits.
American Statistician, Vol. 50, No. 4, Nov. 1996, pp. 311-313
Eduardo Ley
As we have discussed in previous issues of Chance News (see Chance News 4.10), Benford's distribution for leading digits is supposed to fit "natural data." The Benford distribution assigns probabilities log((i+1)/i) to digits i = 1,2,..,9. Ley asks if this distribution fits the leading digits of the one-day return on stock indexes defined as:
r(t) = 100*(ln(p(t+1))-ln(p(t)))/d(t)where p(t) is the value of the index on the tth trading day and d(t) is the time the tth and t+1st trading day -- usually 1. Since p(t+1) = p(t)exp{r(t)*d(t)/100}, r(t) is the continuous-time return rate for the period between tth and t+1st trading time.
Ley finds that the leading digits of r(t) fit the Benford distribution remarkably well for both the Dow-Jones and the Standard and Poor's index.
He obtained the following distributions for the leading digits for the one-day return rate for the Dow-Jones from January 1900 to June 1993 and the S&P from January 1926 to June 1993.
digit Dow-Jones S&P Benford 1 .289 .292 .301 2 .168 .170 .176 3 .124 .134 .125 4 .100 .099 .097 5 .085 .078 .079 6 .072 .071 .067 7 .062 .056 .058 8 .053 .054 .051 9 .047 .047 .046
As you can see, in both cases the approximation is very good.
Despite this apparent good fit, in both indicies a chi-squared test would reject Benford's distribution. Ley attributes this to the very large power of the test caused by so many samples. He remarks that if you take just the last ten years this does not happen and says:
If one takes models as mere approximations to reality, not as perfect data-generating mechanisms, then this can only be viewed as a weakness in the Neyman-Pearson theory (hypothesis testing).Ley's data is available from from http://econwpa.wustl.edu.
DISCUSSION QUESTIONS:
(1) Would you expect leading digits of the Dow-Jones values themselves to have Benford's distribution?
(2) Do you agree with Ley's remark about the weakness in the theory of hypothesis testing?
<<<========<<
>>>>>==============>
Judge rules breast implant evidence invalid.
The New York Times, 19 Dec. 1996, A1
Gina Kolata
In 1993, the Supreme Court considered how to determine what scientific evidence should be allowed in the courts. Rather than giving specific criteria, the court ruled that Federal Judges should use their judgment, based on the ways that scientific theories are evaluated, to determine the admissibility of scientific evidence. Federal District Court Judge Robert E. Jones has been overseeing breast implant cases in Oregon and accepted this responsibility.
He assembled a panel of disinterested scientists, asking them to survey the scientific evidence that had been submitted by the plaintiffs. He then held a four-day meeting in which 12 experts for the plaintiffs and the defense were questioned by lawyers for both sides, the court, and the panel of scientists. The panel then submitted its assessment of the plaintiff's scientific evidence. As a result of this, Judge Jones ruled that the evidence was not of sufficient scientific validity to be presented to the court.
The judge dismissed 70 cases, and, if his ruling is upheld on appeal, this will be a serious blow to the thousands of breast implant cases across the country awaiting trial.
In previous cases, women have won awards as high as $25 million by claiming silicone that leaked from the breast implant devices caused diseases including, classic auto-immune diseases as well as a new disease with symptoms of fatigue, headaches and muscle aches and pains. One major company, Dow Corning Corporation, was forced, by the litigations, to declare bankruptcy.
The story of how scientific evidence was ignored in these previous cases is told in a book described by the New York Times (Notable Books of the Year 1996, NYT 8 Dec. 1996, Section 7 p. 1.):
Science on Trial: The Clash of Medical Evidence and the Law in the Breast Implant Case. By Marcia Angell. (Norton, $27.50.) An accessible, passionate indictment of the ignorance, opportunism and social indifference that enriched lawyers and a few plaintiffs, though the available scientific evidence was against them.
What do you think is meant by a panel of "disinterested scientists?"
<<<========<<
>>>>>==============>
Kent Morrison sent the following article with the remark that he was asked to estimate the probability of the event described. Kent said that the Tribune chose not to use his thoughts on the matter and invites our readers to consider how they would have answered the newspaper's question "and what is the chance of that happening?"
Paso tale one for 'Believe it or not.'
Telegraph-Tribune, 8 Nov. 1996, B2
Jeff Ballinger
Last fall Johnathan Wince was a freshman in high school in Paso Robles California and his brother Christopher was a senior. While sitting in a ceramics class with Christopher, Johnathan signed and dated several dollar bills and these bills were later spent by the brothers.
In September, of this year, Christopher was a freshman at Morris Brown University in Atlanta. To his great surprise, Christopher got back one of the dollars in change while shopping in Atlanta's underground mall.
DISCUSSION QUESTION:
How would you answer the reporter's question: "and what is the chance of that happening?"
<<<========<<
>>>>>==============>
Selenium may reduce cancer risk, study says.
Los Angeles Times, 25 Dec. 1996, A1
Terence Monmaney
The role of the nutrient selenium in cancer has, for decades, been a mystery. Population studies have suggested that cancer rates are higher in areas where the soil is deficient in Selenium. Some animal studies have supported a connection with cancer but studies on humans failed to find a correlation between the levels of selenium in the blood and the risk of cancer.
A study, reported in the December 25 issue of JAMA, was designed to test the ability of selenium to prevent a recurrence of skin cancer for those who had once had this disease. The study, started in 1983, was a double-blind randomized trial carried out in seven dermatology clinics nation-wide. 653 of the subjects were given 200 milligram selenium supplement pills and 659 were given placebo pills. The study failed to show a beneficiary effect of selenium in preventing a recurrence of skin cancer.
However, a preliminary review in 1990 indicated an effect on the overall incidence of cancer. On the basis of this, it was decided to modify the study to include, as secondary end effects, total mortality, cancer mortality and incidence of lung, colon, rectum and prostate cancer. The study was continued until January 1996.
31 of those in the control group developed lung cancer (7.4%) as compared with 17 in the group receiving selenium supplements (2.6%). 35 of the controls developed prostate cancer (4.7%) as compared to 13 in the selenium group (2%). 19 controls got colon or rectum cancer(2.9%) as compared to 8 in the selenium group (1.2%).
The authors, and an accompanying editorial, stated that, while these results looked impressive, a single randomized trial was not sufficient for a change in prevention recommendations.
DISCUSSION QUESTIONS:
(1) What do you think the "end effect" of a randomized trial means? What about a "secondary end effect"?
(2) The incidence of cancer for several other kinds of cancer, including breast cancer, was not significantly lower in the selenium group than in the placebo group. What effect does this have in the evaluation of the study?
(3) The author of the editorial remarked that "interpretation of secondary end effects, like the interpretation of subgroups in randomized trials, requires caution." What did the writer mean by this?
<<<========<<
>>>>>==============>
Weather forecasts 'wrong half the time.'
Daily Telegraph, 9 Dec. 1996, p. 4
Robert Matthews
The Royal Meteorological Society (RMS) has been asked to investigate claims of the Met Office considered to be misleading. Evidently, the Met Office said that its basic forecast is about 84% accurate. This claim was challenged by a study by Dr. John Thornes who showed that some of their claims are correct less than half of the time. Thornes said that the 84% refers to the upper level of accuracy.
Thornes found that the Met predictions, in the summer, can be beat by predicting that the weather tomorrow will be the same as today. Thornes claims that, despite all the technological improvements in weather forecasting, the Met forecasts of rain at a specific time have fallen from an average of 89% correct in the mid-eighties to about 65% correct currently.
While the Met does provide statistics about its performance rate, Thornes feels that the office should be assessed by an outside body and recommended that the RMS act as an independent assessor of the Met and private forecasters.
You can learn about the methods used to assess weather predictors in the new book "Statistical Methods in the Atmospheric Sciences" by Daniel S. Wilks, Academic Press 1995.
DISCUSSION QUESTIONS:
(1) What do you think Thornes means by "the 84% refers to the upper level of accuracy?"
(2) If you want to assess the accuracy of the weather predictions on your favorite evening television news, how would you do this? Do you think you would find much difference between the predictions of the different stations?
(3) When giving the forecast for rain tomorrow, station WXYZ gives simply one of the two predictions "rain tomorrow" or "no rain tomorrow." Station ZYXW gives a probability for rain tomorrow. How might you compare the performance of these two stations over a year's time?
(3) Do you think that weather predictions in your area are better today that they were ten years ago?
<<<========<<
>>>>>==============>
Against the odds: the remarkable story of risk.
Wiley, New York, 1996
Peter L. Bernstein
Peter Bernstein is a well known investment advisor. He has written numerous popular books and articles on economics and finance. In this book, he gives us a history of the development of probability and its relation to the study of risk. His book reads like a novel, with a sprinkling of colorful anecdotes throughout.
Bernstein starts with an account of the early history of probability. After the usual speculation about why the Greeks did not develop probability, he describes how probability did get started, with the familiar cast of characters including: Cardano, Pascal and Fermat, de Moivre, Bernoulli and Laplace. Bernstein follows the classic history of probability by F. M. David: Games, Gods, and Gambling (Peter Griffin, London, 1962).
One of the earliest applications of probability to economics was the development of insurance. Bernstein begins with one of his anecdotes. He says it is believed that the first time coffee was drunk in England was when a Cretian scholar at Oxford made himself a strong cup of coffee. Evidently, this innocent act led to coffee houses springing up all over England. A coffee house, opened in 1687 by Edward Lloyd, was frequented by ship captains exchanging information of the hazards on new routes and other shipping information. This made the coffee houses a place to congregate for those who would offer merchants, for a fee, to cover the cost of a shipping loss should it occur. These one-man insurance operators came to be known as "underwriters." In 1771 seventy-nine of the underwriters, who did business at Lloyds coffee house, subscribed 100 pounds each and joined together, pledging all their worldly possessions to making good on their promises to cover losses. This group became the famous "Lloyds of London" insurance company.
Bernstein makes an excursion into statistics, commenting that the concept of regression to the mean "provides many decision-making systems with their philosophical underpinnings." He gives a nice historical account of how Darwin discovered regression to the mean. This account is based on the very complete treatment of this topic in Steven Stigler's 1986 book "History of Statistics."
The first attempt to model the stock market as a chance process was made by Louis Bachelier, a student of the famous French mathematician Poincaré. In his thesis, Bachelier anticipated the concept of an efficient market and modeled the stock market as a random walk (Brownian motion). This was five years before Einstein's development of Brownian motion as a model for the motion of an electron. Poincaré underestimated Bachelier's contribution and Bachelier was given a "mention honorable" for his thesis rather than "mention très honorable" essential for obtaining a decent job, which he never got.
Bernstein states that "classical economics had defined economics as a riskless system that always produced optimal results." This theory was challenged in the early 1900's by economists Frank Knight and John Maynard Keynes. They both felt that there were serious economic problems where standard probability analysis could not be applied. They felt that situations were so different from time to time that there was no hope in basing future predictions on past behavior. They argued that events like the time of the next world war or the next market crash were simply unknowns and standard probability could make no contribution to these. Keynes wrote an influential book on probability theory. In this book he states that probability theory has little relevance to real-life situations.
Bernstein believes that the distinction these authors made, between uncertainties that could be analyzed by probability methods and those that could not, led to the modern theory of the analysis of risk; but it was not clear to us what all this means. He seems to be saying that classical probability theory attributed future outcomes to a kind of fate, rather than individual decisions (see the quote above), and that Knight's and Keynes' ideas led to theories that took into account individual decisions.
Bernstein turns next to a subject that is more understandable to us: the theory of games introduced by von Neumann and Morgenstern in the forties. In this theory, rational behavior requires the introduction of probabilities in many decision-making situations.
Returning to the discussion of the stock market, Bernstein considers the work of Harry Markowitz, titled "Portfolio Selection", that appeared in 1952 in the "Journal of Finance." This was Markowitz's Phd thesis. He had been working on linear programming and knew nothing about stocks, but a chance meeting with a stock broker got him interested in the idea of controlling the risk in investing. The objective of his portfolio selection was to choose a collection of different kinds of stocks for investors who regard an expected return as desirable but variation of return as undesirable. He showed how to construct portfolios that minimize the variation as measured by the variance. It took two decades and the crash of 1973-74 to make investors appreciate the value of Markowitz's work. In 1990 he was awarded a Nobel prize for his work.
The idea that people make decisions about risk in a rational way implies that they use concepts of probability in a rational manner. The work of Kahneman and Tversky shattered this idea and led economists to become more interested in how people actually use probabilities in their decisions involving risk, rather than in how they should use them. This is a fascinating story that is just now unraveling, and Bernstein's account of it is up-to-date and well done.
Finally, Bernstein tells the story of derivatives as another attempt to control the volatility of investments. Derivatives are described as a way of making side bets. They come in two forms: futures and options. A future is an agreement between A and B that requires A to deliver a share of stock to B at a specified price and a specified time. An option is an agreement between A and B that allows, but does not require, A to buy a share of stock from B at a specified price anytime during a specified period.
Bernstein observes that these are old ideas going back to Aristotle. All that is new is the way they are currently carried out and used. For example, farmers have to invest in their crops well before they know what their crops will sell for. They have used future contracts to get enough for their products to cover the cost of producing them. To do this they might agree to sell to a food processor at a specified price sufficient to assure a profit on their crop. By such a contract the farmer avoids the risk that the price will go too low and the food processor avoids the risk that the price will go too high. In the seventeenth century Dutch tulip merchants used options to insure that they could replenish their stock at a reasonable price when necessary.
Future contracts are now bought on the various stock indices. For example, if the Standard and Poor Index is at 600 you might agree to purchase a "share" three months from now at 610. The deal is settled at the end of the three-month period by your paying the difference if the stock is below 610 and getting the difference if it is above 610.
If you buy an option at price X, and the price of stock goes up significantly beyond X in the period of your option you can make a profit by exercising your option. You have nothing to lose by buying the option. Thus you should have to pay a fee for the option and it is natural to ask what is a reasonable price to pay for an option.
In the early 70's, the problem of finding a formula for the value of an option was tackled three young researchers: Fischer Black, a physicist-mathematician working in Boston; Myron Scholes, a new faculty member at M.I.T. , and Robert C. Merton, a teaching assistant for Paul Samuelson.
These three established a formula for the price of an option that depended only on the price of the stock, the length of time the option is held, the interest rate during this period and, most importantly, an estimate of the volatility of the stock during that period. Their paper was rejected by two major journals. After the intervention of two senior researchers, it was finally accepted by a third journal. Unlike previous breakthroughs, this discovery had an immediate impact on the market -- including a rush on hand-calculators for use on the floor at Wall Street.
Bernstein limits himself to a brief discussion of the current interest in using chaos theory to understand the stock market and neural networks to make predictions.
There is a lot of interesting philosophy about the meaning of probability and risk and the relations between these concepts, that I have left out of this review which we suspect the reader will understand better than we did. We did poorly in philosophy.
DISCUSSION QUESTION:
Why do you think Knight and Keynes would say that you cannot assign a probability for the next world war occurring within ten years? (The Rand Corporation, in its early days, did assign such a probability).
<<<========<<
>>>>>==============>
Please send comments and suggestions for articles to
jlsnell@dartmouth.edu.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!