This blog is dedicated to sharing ideas and resources that can advance learning and democracy in the United States and elsewhere.

Saturday, June 28, 2014

What Is Normal? The Curse of the Bell Curve

Popularized ideas often end up misconstrued and misinterpreted. Consider the biblical allegory in Revelation 7:1, in which the narrator says, “I saw four angels standing on the four corners of the earth….” For centuries, and even within small pockets of believers today, biblical literalists have taken this statement as fact and thus concluded that the Earth is flat. After all, a spherical Earth would not have corners on which angels, which perforce must exist, could stand.

On a slightly less biblical scale, inamorati of the bell curve, formally the Gaussian function, have broadly misinterpreted the normal distribution in probability theory and then misapplied it to circumstances for which it was never intended. Perhaps the most egregious error committed by bell curve enthusiasts is the use of the normal distribution as a prescription—that is, to establish how variations ought to be distributed—rather than as a description of possible, or probable, distributions of variation. Bell curve prescriptionists are the flat-earthers of statistics, except that unlike actual flat-earthers who are relatively harmless, bell curve prescriptionists can do quite a bit of harm—especially in the present age of test mania.

High-stakes tests, in themselves, harm many students because the tests do not accurately or adequately capture a true portrait of students’ knowledge, understandings, or abilities. When an overlay of prescriptive “normality” is imposed, the results are even less reliable as indicators of, well, anything. And application of the bell curve to classroom practice is truly a curse worth lifting.

Consider, instead, O’Boyle and Aguinis’ (2012) “The Best and the Rest: Revisiting the Norm of Normality of Individual Performance.” These researchers studied the performance of individuals involved in four broad areas of human endeavor: academics writing papers, athletes at the professional and collegiate levels, politicians, and entertainers. Their findings challenge the “‘norm of normality’ where individual performance follows a normal distribution and deviations from normality are seen as ‘data problems’ that must be ‘fixed.’”

O’Boyle and Aguinis suggest, alternatively, that distributions of individual performance—such as the learning of students at various levels of schooling—do not follow a Gaussian distribution but, rather, a Paretian distribution (see illustration of a normal distribution overlaying one type of Paretian distribution). Named for Italian economist Vilfredo Pareto (1848 – 1923), this “power law” distribution, sometimes referred to as the “80/20 rule” was originally used to describe the allocation of wealth in Italian society—i.e., 80 percent of the wealth generally rests in the hands of 20 percent of the population. The distribution has broader applicability. The 80/20 rule is shorthand, not a fixed distribution; but it is consistent over many activities involving large groups of people and often fairly describes smaller groups as well. For example, in a given classroom a small percentage of students is often responsible for achieving a large percentage of the top marks, on a sports team a small percentage of players is often responsible for garnering a large percentage of goals or points, and so forth.

In education contexts the so-called Pareto Principle, rather than prescribing how students ought to perform, can be used to help students monitor their own learning. “Documenting a learner’s errors using Pareto charts is an interesting way for learners to see evidence of growth, especially when they are working on discrete skills,” according to staff development trainer Donna Curry (2001) at the EFF National Center.

As practice experience and research like the work done by O’Boyle and Aguinis continues to accumulate, it seems hopeful that thoughtful educators and education policy makers may eventually be able to throw off the curse of the bell curve and thereby move away from prescribing how students ought to perform—whether on high-stakes standardized tests or teacher-made, end-of-unit exams—in favor of examining how students actually do perform and how learning can be encouraged, supported, and expanded for all students. At the very least, notions like the Pareto Principle ought to help educators reconsider what constitutes “normal” when it comes to teaching and learning.


This summary is excerpted from a longer article titled, “The Curse of the Bell Curve,” which can be accessed at https://www.academia.edu/7488772/The_Curse_of_the_Bell_Curve. The full article contains the references omitted from this excerpt.

Friday, June 20, 2014

Test Blindness

A recent Chronicle of Higher Education headline read “Hampshire College Will Go ‘Test Blind’” (June 18, 2014). While many colleges are “test optional,” meaning that students may or may not submit SAT or ACT scores and institutions may or may not consider those scores in making admission decisions, Hampshire College in Amherst, Massachusetts, has become a standout by simply refusing even to consider such test scores. Officials of the college based the decision on research that indicated the scores, in the words of the article, “did not help identify applicants who were most likely to thrive on the campus.”

This research (unidentified in the article) reiterates other longstanding research findings that have concluded, literally over decades, that SAT scores are less reliable predictors of student success in college than the students’ high school grades—in other word, high school teachers’ face-to-face evaluations of their students.

Tests in general, whether the mass standardized variety now dominating public schooling, predictive tests such as the SAT, or garden variety teacher-made classroom quizzes, are by their nature mere snapshots. They are never fully fleshed, robust portraits of the test-takers. Well-trained, observant teachers may take the results of various tests into account but there is far more information than mere test results flowing into their judgments regarding the students they come to know firsthand—up close and personal, as a TV commentator might say. Test scores are no substitute for actual day-to-day interactions over time between teachers and students.

Today’s public schools are suffering from another type of “test blindness,” the limited vision of student learning ranging from potential to performance caused by an over-reliance on mass standardized tests to determine what should be taught, how well students are learning, and how well teachers are teaching—three things that testing cannot tell us except in very narrow, limited, snapshot kinds of ways.


A better course would be to take a leaf from Hampshire’s notebook and simply stop using tests. Mass testing is costing citizens millions of dollars, it’s costing students and teachers many hours of wasted learning time, and it’s enriching testing corporations way out of proportion to their tests’ value to education. Moreover, and most significantly, mass standardized testing is blinding schools and society to the realities of students’ needs, potential, and real performance.