WORKING DRAFT

March 1, 1999

DO NOT QUOTE OR CIRCULATE

This version is intended for use by SRTL participants.  Please do not distribute without permission from the author.

 

 

Statistical literacy:

Conceptual and instructional issues

 

 

Iddo Gal

University of Haifa, Israel

 

To appear in J. O'Donoghue, D. Coben, & G. Fitzsimons (Eds.), Issues in adult numeracy education (tentative title). London: Kluwer.

 

 

1.  Framing the problem

 

One key declared goal of education programs at all levels is preparing learners to become more informed citizens and workers who can effectively function in an information-laden society. Towards that end, instruction in literacy and mathematics is usually provided, in part because adults will need to interact daily with quantitative situations, including many where quantitative information is embedded in text. This paper focuses on one critical but often neglected aspect of mathematics (numeracy) education, that of developing statistical literacy.

 

The term "statistical literacy" does not have an agreed-upon meaning. In public discourse, when “literacy” is combined with any term referring to a specific knowledge domain (e.g., “computer literacy”) it conjures up an image of the minimal subset of "basic skills" expected of all learners or citizens, as opposed to a more advanced set of skills and knowledge that only some people may achieve. In this sense, statistical literacy may be understood by some to denote a minimal knowledge of basic statistical concepts, tools, and procedures, possibly including some interpretive skills.

 

In this chapter “statistical literacy” describe people's ability to interpret and critically evaluate statistical information and data-based arguments appearing in diverse media channels (e.g., newspaper articles, TV and radio news and programs, publications of political groups, advertisements) and to their ability to discuss their opinions regarding such statistical information. This conception is related to a “minimal skills” conception, yet emphasizes sense-making and communicative capacities, more than formal statistical knowledge, assuming that most adults are consumers rather than producers of statistical information.

 

Statistical literacy is needed if adults are to be fully aware of trends and phenomena of social and personal importance, such as regarding crime, population growth, spread of diseases, industrial production, educational achievement, employment, and so forth, or to enable informed participation in public debate or action regarding national or community issues (Wallman, 1993). Similar needs arise in many workplaces, given increasing demands for quality and employee self-management (Carnevale, Gainer, & Meltzer, 1990; Packer, 1997).

 

Despite the importance of statistical literacy and its role in adults’ general numeracy (Gal, 1997), almost no in-depth discussions regarding statistical literacy and the educational processes needed to develop it have been published. This chapter aims to contribute to a needed dialogue among practitioners, educational planners, and researchers in order to promote statistical literacy. The chapter is organized in three parts. First, key contexts were statistical literacy is needed and can be developed are described. Next, key components of statistical literacy are outlined. Finally, some instructional dilemmas and implications for teaching and research are discussed.

 

2.  Types of interpretive contexts

 

A discussion of statistical literacy should first consider the contexts in which such “literacy” may both develop and be called for. We start by discussing the classroom or teaching context, since students should learn and know something about statistical and probabilistic concepts and procedures as a prerequisite for making sense of statistical messages.

 

Introducing statistical ideas

 

Various sources exist that discuss goals and approaches to instruction in introductory statistics at different levels of instruction (e.g., Friel, Russell & Mokros, 1990; Moore, 1992; Gal & Garfield, 1997; Lajoie, 1998). Such sources usually make quite similar recommendations, though the range of topics to be covered and the sequencing of topics and activities may vary depending on student, teacher, and context factors. An approach often suggested for almost any level of instruction is that teachers spend some time on basic concepts and procedures, and also take students on at least one complete cycle of a statistical investigation, where students engage all these phases:

            ·        Formulate a question or hypothesis of interest to the students

            ·        Plan the study (e.g., overall approach, sampling, how to measure target variables)

            ·        Collect data and organize it

            ·        Display, explore, and analyze data

            ·        Interpret findings (in light of the question)

            ·        Discuss conclusions and implications

 

The expectation is that by interweaving focused instruction with actual experiences in all phases of a statistical inquiry, students will gradually understand key goals, decisions, and dilemmas associated with designing a data-based study, become familiar with issues involved in making sense of data, and realize that statistical findings can be used to answer many questions and to support or reject hypotheses, depending on the quality and credibility of available information.

 

To be sure, the chosen learning process must also develop learners’ awareness of the “big ideas” that underlie all statistical thinking, such as: there is variation in the world and it has to be estimated in various ways by statistical methods; there is often a need to study samples instead of populations and to infer from samples to populations; errors are likely in measurement and inference, and there is a need to estimate errors; there is a need to reduce large amounts of raw data by noting trends and main features (see Moore, 1990; Gal & Garfield, 1997).

 

 

Interpretive contexts

 

The specific nature of interpretive skills will depend on the context(s) in which they are needed. Gal (1998) distinguishes two such contexts:

 

Reporting contexts emerge when learners are “data producers” and take part in all phases of a data-based study as described above. They have to interpret their own data/results in order to prepare a report, and discuss with others their findings, conclusions, implications, etc.

Listening (reading) contexts emerge when learners are “data consumers” and are encountered either (a) in the classroom, where learners have to listen to (or read) a report from other learners, or (b) in or outside the classroom, where people have to make sense of and possibly react to messages that contain statistical elements.

 

Listening contexts, especially those outside the classroom, are our main concern here when we speak about statistical literacy. In such contexts, people do not engage in generating any data or in making any computations. Their familiarity with the data-generation process (e.g., study design, sampling plan, questionnaires, etc), or with the procedures used to analyze the data, depend on the details and clarity of the information given by the message producer. Yet, they do have to comprehend the meaning of messages they are presented with, and be both interested and able to critically examine the reasonableness of such messages or claims, or reflect about different implications of findings being reported.

 

To illustrate some of the many ways in which statistical concepts and findings and statistics-related issues appear in the media, consider the following excerpts. These are taken from newspapers, which provide a prime example for a listening/reading context where statistical content is embedded in text.

 

“…The study found that women of average weight in the U.S. had a 50 per cent higher chance of heart attack than did women weighing 15 per cent below average.” (Watson, 1997, p. 109; from Hobart Mercury, New Zealand, February 10, 1995).

 

“Judges count out Census sampling:…at issue is far more than the accuracy of sampling in the Census held every 10 years: Billions of dollars in federal funds are allocated on the basis of how many people live in each state and city, and shifts in population can lead to the redrawing of House districts. A boost in the count of minorities would normally help Democrats.” (Philadelphia Inquirer, August 25, 1998).

 

“Poll backs limits on drinking by teens: The survey of more than 7000 adults… which has a margin of error of 2 percentage points, found that…More than half favored restrictions on alcohol advertising…More than 60% would ban TV ads for beer and wine.” (USA Today, October 5, 1998)

 

“The human race held this year many more sexual intercourses than last year; the world average was 112 per person this year, compared to 109 last year. This, according to a comprehensive survey initiated and funded, for the second year, by Durex, a manufacturer of prophylactics. The survey was held in 14 countries that according to experts represent all the world citizens…” (Yediot Aharonot, Israel, October 28, 1997).

 

The demands in listening contexts may differ from those in reporting contexts with regard to several related facets. (The reader is advised to apply points made below to the excerpts above).

 

1.  Literacy and background knowledge demands of the messages involved:  In the classroom, adult students will most likely use language that is quite simple, due to their still developing linguistic and statistical skills. In the real world, listeners will have to make sense of message of different degrees of complexity, created by journalists, officials, or others with diverse (and possibly advanced) linguistic and numeracy skills. Space limitations may make messages terse, choppy, or lack essential information.

 

2.  Range of statistical topics involved:  Messages in the media or other real-world contexts will cover a much broader range of statistical or probabilistic concepts, findings, and problems, compared to what learners usually bring up on their own in the classroom. For example, the design of controlled experiments, sampling techniques, correlations, trends over time, or risk assessment, are only some of the subjects that beginning learners of statistics are not likely to choose as a classroom project.

 

3.  Degree of familiarity with sources for variation and measurement error:  In listening contexts people are less aware of flaws in doing research or problems with collecting credible data when the given study is about an unfamiliar topic.

 

4.  Degree of need for critical evaluation of the source of a message: When an adult has to critically think about the validity of a message presented by a source such as a journalist, a politician, or an advertiser, there may be a greater need to suspect the message producer is trying to advance his or her agenda, than in classroom reporting.

 

Despite these differences, in both reporting and listening context the actors (learners) will be involved in the interpretation, creation, communication, or defense of opinions (for example, about the implications of a graph, the adequacy of a certain sample, or the validity of an argument). The development of students' ability to generate sensible and justifiable opinions should thus become a target area for instruction (Gal, 1998).

 

 

3. Building blocks of interpretive processes

 

What knowledge bases and other enabling processes should be in place so that people can come up with reasonable and if needed critical interpretations of messages encountered in statistical tasks? It is argued here that adults’ ability to effectively manage interpretive statistical tasks involve both a cognitive component (comprised of three elements: statistical knowledge bases, literacy skills, and a list of critical questions) as well as a dispositional component. (Gal, 1997, introduces the idea of management of quantitative situations). These four building blocks are described below separately, but are interdependent in operation and development.

 

Statistical knowledge bases

 

A prerequisite for comprehension and interpretation of statistical messages is that students posses knowledge of basic statistical and probabilistic concepts and procedures. Obviously, what constitutes “basic” will depend on the students’ level and context of learning. It is important to note that almost all authors who are concerned about the ability of adults or of school graduates to function in a statistics-rich society do not discuss what knowledge is needed to be statistically literate per se, but usually argue that all school (or college) graduates should master a wide range of statistical topics, assuming this will ensure learners’ ability to engage both in interpretive as well as in other statistical tasks as adults.

 

A recent example can be found in a comprehensive chapter by Scheaffer, Watkins, & Landwehr (1998), titled, “What every high-school graduate should know about statistics.” Based on their extensive prior work in the area of teaching statistics, and on reviewing various curriculum frameworks, these authors describe numerous areas as essential to include in a study of statistical topics towards a high-school degree:

                      ·        Number sense

                      ·        Understanding variables

                      ·        Interpreting tables and graphs

                      ·        Aspects of planning a survey or experiment, including what constitutes a good sample, methods of data collection and questionnaire design, etc.

                      ·        Data analysis processes, such as detecting patterns in univariate or two-way frequency data, summarizing key features with summary statistics, etc.

                      ·        Relationships between probability and statistics (e.g., in determining characteristics of random samples, background for significance testing)

                      ·        Inferential reasoning (including confidence intervals, testing hypotheses, etc.)

 

Clearly, few if any adult education programs cover such an agenda, due to time constraints but also given teachers’ lack of relevant knowledge and training. Watson (1997) is one of the very few sources that refer, albeit briefly, to the basic knowledge and skills needed specifically for statistical literacy. She suggests that these include percentage, median, mean, specific probabilities, odds, graphing, measures of spread, and exploratory data analysis.

 

It is proposed here that six different but related mathematical and statistical knowledge bases are needed for statistical literacy. These were identified on the basis of reviewing (a) writing by mathematics and statistics educators (examples are Moore, 1990; and chapters in Steen, 1997; Gal & Garfield, 1997; Lajoie, 1998), and (b) sources discussing mathematics and statistics in the news (such as, Huff, 1954; Hooke, 1983; Crossen, 1994; Paulus, 1995; Kolata, 1998). Given space limitations, these knowledge bases are described in broad strokes only; they are presented to suggest what kinds of “things” adults overall need to know and develop in a lifelong perspective, rather than determine what students are supposed to formally study at any specific educational stage, given that adult learners may bring with them diverse prior knowledge acquired both formally and informally, and given that people engage in multiple learning episodes throughout their lives. Also, these knowledge bases are described in absolute terms but have to be interpreted in light of people’s specific life contexts, keeping in mind that people’s numeracy (Gal, 1997) is not a fixed entity but a relative and dynamic set of skills and interrelated processes.

 

1.  Possess mathematical foundations:  Adults should posses not only number sense (pertaining both to numbers and probabilities), but also knowledge of proportionality concepts, especially of percent, which is a key concept used when reporting statistical results in the media. Familiarity with principles of reading graphs and tables and ability to integrate information from their different parts. Advanced arithmetical knowledge and multi-step problem-solving are needed mainly for learning data-analysis techniques, but are somewhat less important for functioning in listening contexts per se. That said, broader mathematical knowledge is overall essential, since statistical arguments may appear together with other mathematical issues that are not statistical in nature (Paulus, 1995).

 

2.  Know why data is needed and where it comes from:  Understand key “big ideas” behind statistical investigations in general, and have some technical knowledge of approaches to the design of different studies (survey, experiment, Census, exploratory), and the logic behind these, including some sampling and measurement methods and dilemmas. Understand the influence of sampling processes and sample size or composition on representativeness and ability to generalize or infer to a population.

 

3.  Know how data are processed and analyzed:  Know that the same data can be analyzed or displayed in multiple ways (e.g., by using different measures of central tendency, different graphs, etc.), and that different methods may lead to similar results but may also yield a different and at times conflicting view of the phenomena under investigation. Be aware that error are unavoidable but that they can be controlled at various stages of a research and be estimated statistically.

 

4.  Know how statistical conclusions are reached:  Be aware of the need to base claims or conclusions on credible empirical evidence. Know that observed differences, associations, or trends may exist but may not necessarily be large or stable enough to be meaningful or important. Realize that results of studies may fluctuate over episodes of data collection, but that such issues should diminish when studies are well-designed. Know that there are ways to determine the significance of a difference between groups, or of an association between variables, e.g., by comparing summary statistics.

 

5.  Understand basic notions of probability and risk:  Understand basic notions of probability and chance, and be familiar with the many ways in which they may be reported, such as percents, odds, or verbal estimates (see Wallsten, Fillenbaum, & Cox, 1986). Understand that statements of chance may come from diverse sources, both formal and subjective, with different degrees of credibility. Realize that judgments of chance may fluctuate when additional data is available.

 

6.  Know about typical flaws in executing studies, analyzing data, or interpreting results:  Be familiar with key “things that can go wrong” during the lifecycle of a study. Be aware of typical errors that researchers, public officials, and others make when reaching conclusions or conveying findings them to the public, such as confusing correlation with causality, making small differences loom large, ignoring significance of observed differences, etc.

 

This list, offered here in outline form, can be modified or expanded, depending on the level of the student and on the desired sophistication of statistical literacy. For practical reasons, this list is restricted here to domains that can reasonably be addressed (and in turn assessed) as part of ongoing educational processes. That said, we must keep in mind that numerous other knowledge elements that contribute to statistical literacy can be described but are beyond the scope of this paper. Some involve more literacy-related issues, such as having a sense for what good journalistic writing looks like (e.g., objective writing, presentation of two-sided arguments; provision of background information to orient readers to the context of a story, etc). Other involve broader cognitive and metacognitive capacities that can support statistically literate behavior, such as having a propensity for logical reasoning, curiosity, and open-minded thinking.

 

Literacy skills

 

The development of statistical literacy requires that people can comprehend text and the meaning and implications of the statistical information in it, in the context of the topic to which the reported statistical information pertains (Watson, 1997). The statistical portion of a message may sometimes not be large; comprehension of surrounding or knowledge of background information will be needed to enable a reader or listener to place the statistical part in a wider context. In addition, learners should also be able to present, orally or in writing, clear and well-articulated opinions, i.e., that contain enough information about the reasoning or evidence on which they are based so as to enable another listener to judge their reasonableness. Thus, statistical literacy and general literacy are intertwined.

 

Curriculum frameworks in mathematics now emphasize the need to develop all learners’ communication skills (e.g., NCTM, 1989; Curry, Schmidt, and Waldron, 1996). Such calls are also being echoed in statistics education circles (e.g., Samsa, & Oddone, 1994). However, as several authors have pointed out (e.g., Laborde, 1990), learning mathematical topics requires various types and degrees of interaction with learners’ literacy skills. Applied to learning statistics, such interaction may involve, for instance:

                   ·        learning new words or word clusters (e.g., median, standard deviation, statistically significant, margin of error) that have only a mathematical usage and meaning or where the cluster has a meaning different from those of its components.

                   ·        realizing that meanings of mathematical, statistical, or probabilistic terms used in a classroom context (e.g., random, table, association, average, sample, representative, error) may be different or more precise than those used in everyday speech.

                   ·        being aware that some quantitative information might be conveyed by terms or embedded in displays even if no numbers or formal statistical terms are used, e.g., information about degrees of uncertainty, trends or changes over time, and more.

                   ·        Realizing the need to apply a range of reading comprehension strategies to extract meaning from diverse texts which touch on statistical or probabilistic issues, from tersely-written mathematics or statistics textbooks, to journalistic texts, etc.

 

Such and related demands may affect learning, comprehension, and resulting real-world performance, not only of adults who are bilingual or otherwise have a weak mastery of English (Cocking, & Mestre, 1988), but also of people who have a relatively good command of the language. The results pertaining to Document Literacy, one of three facets of literacy assessed by the recent International Adult Literacy Survey (IALS), are interesting in this regard (Organisation for Economic Co-operation and Development & Human Resources Development Canada, 1997). The IALS employed functional, realistic tasks to assess literacy in large samples of adults in several industrialized countries, such as the U.S., U.K., Canada, Sweden, Germany and others. Literacy was viewed as the ability to understand and employ printed and written information in daily activities, in order to function in society, to achieve one’s goals, and to develop one’s knowledge and potential. Document Literacy was defined as the knowledge and skills required to locate and use information contained in various formats, including job applications, payroll forms, transportation schedules, maps, tables, and graphics.

 

Statistical literacy as described in this paper is related to Document Literacy (Kirsch and Mosenthal, 1990).The IALS Document Literacy scale included numerous statistics-related tasks, for instance making sense of graphs such as those that routinely appear in newspapers, interpreting statements with percents, or integrating information across graphs and tables. Such tasks require that adults not only locate specific information in given texts or displays, but cycle through various parts of diverse texts or displays, make inferences, or apply other reading comprehension strategies, quite often in the presence of irrelevant or distracting information. IALS results indicated that, roughly speaking, between one-third and two-thirds of all adults in most of the countries studied had difficulty with many Document Literacy tasks. While details of the IALS results are too complex to be described here, they do serve to demonstrate the many interdependencies between literacy and statistical knowledge and highlight the complexity of the notion of statistical literacy.

 

Critical questions

 

We would like adults to not only be able to comprehend graphical displays or statements and texts with embedded statistical terms or data-based claims (i.e., simply understand what is being said or shown), but also have "in their head" a list of critical questions which relate to things to worry about regarding that which is being communicated or displayed to them. Such a critical list should be a direct outgrowth of adults’ possession of the statistical knowledge bases outlined above, at least in a rudimentary form. When faced with an interpretive statistical task, we imagine people running through this mental list and asking for each item, "Is this question relevant for the situation/ message/ task I face right now?" Examples for possible questions include (but are not limited to):

 

1.  Where did the data (on which this statement is based) come from? What kind of study was it? Is this kind of study reasonable in this context?

2.  Was a sample used? How was it sampled? Is the sample large enough? Did the sample include people/things which are representative of the population? Overall, could this sample reasonably lead to valid inferences about the target population?

3.  How reliable or accurate were the measures used to generate the reported data?

4.  What is the shape of the underlying distribution of raw data (on which this summary statistic is based)? Does it matter how it is shaped?

5.  Are the reported statistics appropriate for this kind of data, e.g., was an average used to summarize ordinal data; is a mode a reasonable summary? Could outliers cause a summary statistic to misrepresent the true picture?

6.  Is a given graph drawn appropriately, or does it distort trends in the data?

7.  How was this probabilistic statement calculated, and are there enough credible data to justify such an estimate of likelihood?

8.  Overall, are the claims made here sensible? Are they supported by the data? (e.g., confusing correlation with causation)

9.  Should additional information or procedures be made available to enable me to evaluate the sensibility of these arguments? Is something missing?

10.  Are there alternative interpretations for the meaning of the findings, different explanations for what caused them, or additional or different implications?

 

Answers people generate to such and related questions can support the process of evaluating statistical messages and lead to the creation of more informed opinions. This list can of course be modified and expanded, depending on the level and functional needs of the learners and on the context and goals of instruction. For example, it can expand beyond basic statistical issues to cover statements of probability and risk, statistical significance, or more advanced or job-specific statistical topics, such as those related to statistical process control or quality management processes.

 

Dispositions and beliefs 

 

It is not enough that students have access to relevant knowledge bases or are aware of critical questions they could apply to a message at hand. To actually act in a statistically literate way, certain dispositions and beliefs need to be in place. First, and most importantly, learners should develop a propensity to spontaneously invoke, without external cues, the list of worry questions, and invest the mental effort needed to ask penetrating questions and try to answer them. Otherwise, they might learn to accept without questioning objectionable arguments, and may develop an incorrect world view.

 

Further, learners should uphold the belief that there may be alternative interpretations or implications to any finding which is based on statistical processes (e.g., a sample may be non-representative, an intervening variable affected the results of the study). They should feel comfortable with the role of being a critical evaluator of statistical claims, and accept that it is legitimate to have doubts and concerns about any aspect of the study or the interpretation of its results, and to raise questions about statistical information being communicated to them, even if they have not learned much formal statistics.

 

While we want students to develop a critical stance, we also want them to develop an appreciation for the power of statistical processes, and accept that the use of systematic data-gathering procedures, be they surveys, controlled experiments, or exploratory studies, often leads to conclusions that are better than those obtained by relying on anecdotal data or subjective experiences.

 

5.  Discussion and implications

 

Statistical literacy is part of both people's numeracy and literacy (Gal, 1999). A recent white paper of the European Commission argued that in a society in which the individual will have to understand complex situations and vast quantities of varied information, “there is a risk of a rift appearing between those who are able to interpret, those who can only use, and those who can do neither” (European Commission, 1996, p. 8). Yet, even though statistical literacy is an important area and included in the rhetoric of public officials and educators at all levels, especially those involved in numeracy education, it is hardly represented in existing textbooks or training materials for adult educators, and gets little attention in standard assessments.

 

The question is then—how should students' interpretive skills and statistical literacy be developed? Many teachers most likely will argue that learning to "do" statistics promotes achievement of statistical literacy. Many adult educators, however, visit only briefly the topic of statistics. They may teach their students, for instance, about bar graphs, how to calculate an average, or what a median means. This is important as a first step towards statistical literacy, but unfortunately, such topics are all too often taught in an isolated manner, without explicit connection to the way the underlying concepts appear in adults’ everyday lives. Fragmented teaching is not likely to contribute much to students' understanding of statistical concepts (Shaughnessy, 1992), or to their ability to make sense of statistical messages.

 

Students' statistical and mathematical knowledge bases, as described above, are a necessary component of statistical literacy, yet some of them are not at the heart of existing curricula in statistics. It is argued here that teachers must therefor also work directly towards statistical literacy, and emphasize broader aspects of research design and origins of data, interpretation issues, critical questions, and supporting dispositions and reasoning processes as described earlier. The development of statistical literacy requires work on broader interpretive questions that put into action concepts as well as critical perspectives learned before, in the context of authentic tasks. Texts of relevance to learners’ lives can be found in local newspapers or TV broadcasts, leaflets distributed by political candidates, advertisements, health education brochures from medical organizations, etc. Many resources for statistics education can be used to obtain examples for media-related classroom projects and ideas for class discussions, from the seminal Quantitative Literacy series (e.g., Landwehr, Swift, & Watkins, 1987), to the newspaper excerpts and accompanying discussion questions distributed regularly on the internet by the Chance Course (at: www.dartmouth.edu/~chance).

 

To support the development of adult learners’ statistical vocabulary and communication skills, including those needed for modern workplaces, better integration of numeracy-related and literacy activities is required. Many suggestions made towards developing communication aspects in mathematical classes can be adapted, such as having students keep journals of their work, prepare oral or written reports from statistical projects, make short presentations, or design their own math stories (Hicks and Wadlington, 1999).

 

Motivational barriers are likely to be an issue, as statistical work often ends with findings whose interpretation, meaning and quality are a matter of opinion. In contrast, much of school-type mathematics is often viewed by many adult learners, and sometimes by their teachers, as involving procedural learning and solutions that can be categorized as right or wrong. Students as well as teachers should realize that the "rules of the game" are different when it comes to thinking about statistical issues, and that they should take an active role in forming opinions and in explaining the reasoning behind them (Gal, 1998).

 

Extension of numeracy education to include an emphasis on statistics in general and on development of statistical literacy in particular may be also hampered by teachers' lack of knowledge, or concern about need to devote more time to topics that appear more central. However, work on statistical topics offers unique opportunities to enhance quantitative reasoning and important communication skills that mathematics educators have been struggling for years to advance, but with limited success. At a time when statistical knowledge, in both formal and informal forms, is increasingly being considered essential for effective citizenship and as a part of required workforce preparation, adult education systems should open a dialogue and seek strategies that can support the development of statistical literacy and thus help fulfill the promise of informed citizenship for all.

 

 

 

References

 

Carnevale, A. P., Gainer, L. J., & Meltzer, A. S. (1990). Workplace basics: The essential skills employers want. San Francisco: Jossey‑Bass.

 

Cocking, R. R., & Mestre, J. P. (Eds.) (1988). Linguistic and cultural influences on learning mathematics. Hillsdale, NJ: Lawrence Erlbaum.

 

Crossen, C. (1994). Tainted truth: The manipulation of fact in America. New York: Simon & Schuster.

 

Curry, D., Schmitt, M. J., & Waldron, W. (1996). A Framework for adult numeracy standards: The mathematical skills and abilities adults need to be equipped for the future. Final report from the System Reform Planning Project of the Adult Numeracy Network. Washington, DC: National Institute for Literacy.

 

European Commission (1996). White paper on education and training: Teaching and learning—towards the learning society. Luxembourg: Office for official publications of the European Commission.

 

Friel, S. N., Russell, S., & Mokros, J. R. (1990). Used Numbers: Statistics: middles, means, and in‑betweens. Palo Alto, CA: Dale Seymour Publications.

 

Gal, I. (1997). Numeracy: reflections on imperatives of a forgotten goal. In L. A. Steen (Ed.), Quantitative literacy (pp. 36-44). Washington, DC: College Board.

 

Gal, I. (1998). Assessing statistical knowledge as it relates to students' interpretation of data. In S. Lajoie (Ed.), Reflections on statistics: Learning, teaching, and assessment in grades K-12 (pp. 275-295). Mahwah, NJ: Lawrence Erlbaum.

 

Gal, I. (1999). The numeracy challenge. In I. Gal (Ed.), Developing adult numeracy: From theory to practice (pp. 1-25). Cresskill, NJ: Hampton Press.

 

Gal, I., & Garfield, J. (1997). Curricular goals and assessment challenges in statistics education. In I. Gal & J. B. Garfield (Eds.), The assessment challenge in statistics education (pp. 1-13). Amsterdam, the Netherlands: IOS Press.

 

Gal, I., & Garfield, J. (Eds.) (1997). The assessment challenge in statistics education (pp. 1-13). Amsterdam, the Netherlands: IOS Press.

 

Hicks, K., & Wadlington, B. (1999). Making life balance: Writing original math projects with adults. In I. Gal (Ed.), Developing adult numeracy: From theory to practice (pp. 145-160). Cresskill, NJ: Hampton Press.

 

Hooke, R. (1983). How to tell the liars from the statisticians. New York: Marcel Dekker.

 

Huff, D. (1954). How to lie with statistics. New York: W. W. Norton.

 

Kolata, G. (1997). Understanding the news. In L. A. Steen (Ed.), Why numbers count: quantitative literacy for tomorrow’s America (pp. 23-29). New York: The College Board.

 

Kirsch, I., & Mosenthal, P. (1990). Understanding the news. In L. A. Steen (Ed.), Reading research quarterly, 22(2), 83-99.

 

Laborde, C. (1990). Language and mathematics. In P. Nesher & J. Kilpatrick (Eds.), Mathematics and cognition (pp. 53‑69). New York: Cambridge University Press.

 

Lajoie, S. P. (Ed.) (1998). Reflections on statistics: learning, teaching, and assessment in grades K‑12 (pp. 63-88). Mahwah, NJ: Lawrence Erlbaum.

 

Landwehr, J. M., Swift, J., & Watkins, A. E. (1987). Exploring surveys and information from samples. (Quantitative literacy series). Palo Alto, CA: Dale Seymour publications.

 

Moore, D. S. (1990). Uncertainty. In L. A. Steen (Ed.), On the shoulders of giants: New approaches to numeracy (pp. 95-137). Mathematical Sciences Education Board. Washington, DC: National academy Press.

 

Moore, D. S. (1992). Teaching statistics as a respectable subject. In F. & S. Gordon (Eds.), Statistics for the twenty‑first century. (pp. 14‑25). Washington, DC: The Mathematical Association of America.

 

National Council of Teachers of Mathematics (NCTM) (1989). Curriculum and evaluation standards for school mathematics. Reston, VA: Author.

 

Organisation for Economic Co-operation and Development (OECD) and Human Resources Development Canada (1997). Literacy for the knowledge society: Further results from the International Adult Literacy Survey. Paris and Ottawa: OECD and Statistics Canada.

 

Paulus, J. A. (1996). A mathematician reads the newspaper. New York: Anchor books/Doubleday.

 

Packer, A. (1997). Mathematical Competencies that employers expect. In L. A. Steen (Ed.), Why numbers count: quantitative literacy for tomorrow’s America (pp. 137-154). New York: The College Board.

 

Samsa, G., & Oddone, E. Z. (1994). Integrating scientific writing into a statistics curriculum: A course of statistically based scientific writing. The American Statistician, 48(2), 117-119.

 

Shaughnessy, M. J. (1992). Research in probability and statistics: Reflections and directions. In D. A. Grouws, (Ed.), Handbook of research on mathematics teaching and learning (pp. 465-494). NY: Macmillan.

 

Scheaffer, R. L., Watkins, A. E., & Landwehr, J. M. (1998). What every high-school graduate should know about statistics. In S. O. Lajoie (Ed.), Reflections on statistics: learning, teaching and assessment in grades K-12 (pp. 3-31). Mahwah, New jersey: Lawrence Erlbaum.

 

Steen, L. A. (Ed.) (1997). Why numbers count: quantitative literacy for tomorrow’s America (pp. 137-154). New York: The College Board.

 

Wallman, K. K. (1993). Enhancing statistical literacy: Enriching our society. Journal of the American Statistical Association, 88, 1‑8.

 

Wallsten, T. S., Fillenbaum, S., & Cox, J. A. (1986). Base rate effects on the interpretations of probability and frequency expressions. Journal of Memory and Language, 25, 571‑587.

 

Watson, J. (1997). Assessing statistical literacy through the use of media surveys. In I. Gal & J. Garfield, (Eds.), The assessment challenge in statistics education. A publication of the International Statistical Institute. Voorburg, The Netherlands: IOS Press.