Uneducated Guesses

Uneducated Guesses: Using Evidence to Uncover Misguided Education Policies

HOWARD WAINER
Copyright Date: 2011
Pages: 200
https://www.jstor.org/stable/j.ctt7rjsm
  • Cite this Item
  • Book Info
    Uneducated Guesses
    Book Description:

    Uneducated Guesseschallenges everything our policymakers thought they knew about education and education reform, from how to close the achievement gap in public schools to admission standards for top universities. In this explosive book, Howard Wainer uses statistical evidence to show why some of the most widely held beliefs in education today--and the policies that have resulted--are wrong. He shows why colleges that make the SAT optional for applicants end up with underperforming students and inflated national rankings, and why the push to substitute achievement tests for aptitude tests makes no sense. Wainer challenges the thinking behind the enormous rise of advanced placement courses in high schools, and demonstrates why assessing teachers based on how well their students perform on tests--a central pillar of recent education reforms--is woefully misguided. He explains why college rankings are often lacking in hard evidence, why essay questions on tests disadvantage women, why the most grievous errors in education testing are not made by testing organizations--and much more.

    No one concerned about seeing our children achieve their full potential can afford to ignore this book. With forceful storytelling, wry insight, and a wealth of real-world examples,Uneducated Guessesexposes today's educational policies to the light of empirical evidence, and offers solutions for fairer and more viable future policies.

    eISBN: 978-1-4008-3957-5
    Subjects: Education

Table of Contents

  1. Front Matter
    (pp. i-viii)
  2. Table of Contents
    (pp. ix-x)
  3. Preface
    (pp. xi-xviii)
  4. Introduction
    (pp. 1-7)

    It is rare to know exactly when and where an idea originates. But for me it was Monday, September 7, 1970, at 5:45 p.m. I was on the South Side of Chicago walking north on Dorchester Avenue, between Fifty-sixth and Fifty-seventh streets. I was a new assistant professor at the University of Chicago, and was walking home from my first day of work. Standing in the street was a very large, black motorcycle next to an even larger, fearsome looking, man. He was wearing motorcycle boots and a leather jacket, had long hair and a full beard, and was speaking...

  5. 1 On the Value of Entrance Exams: What Happens When the SAT Is Made Optional?
    (pp. 8-19)

    On my shelf is a well-used copy of James Thurberʹs (1939)Fables for Our Time. In it is the fable of ʺThe Glass in the Field.ʺ It seems that a builder had ʺleft a huge square of plate glass standing upright in a field one day.ʺ Flying at high speed, a goldfinch struck the glass and later told a seagull, a hawk, an eagle, and a swallow about his injuries caused by the crystallized air. The gull, the hawk, and the eagle laughed and bet the goldfinch a dozen worms that they could fly the same route without encountering crystallized...

  6. 2 On Substituting Achievement Tests for Aptitude Tests in College Admissions
    (pp. 20-28)

    Confucius has taught us that the first step toward wisdom is calling things by the right name. College admissions tests have historically been aptitude tests whose forebears were IQ tests. There are two principal reasons for this lineage:

    1. The tests were designed to be fair to applicants from high schools of widely varying curricula.

    2. Such tests have been remarkably successful in predicting success in undergraduate studies.

    Thus it isnʹt surprising that the name of one such test reflects its history and its content—the Scholastic Aptitude Test.

    Confucius would likely agree that a big step toward dishonesty is purposely naming...

  7. 3 On Rigid Decision Rules for Scholarships
    (pp. 29-31)

    The Preliminary SAT / National Merit Scholarship Qualifying Test (PSAT/NMSQT) is a program cosponsored by the College Board and the National Merit Scholarship Corporation. It’s a standardized test that gives firsthand practice for the SAT. It also provides an opportunity to enter National Merit scholarship programs and gain access to college and career planning tools. It is taken by more than 1.5 million students annually.

    The PSAT/NMSQT is designed to measure critical reading skills, math problem-solving skills, and writing skills. It is made up of relatively easy, retired SAT items and is dirt cheap to construct and administer. It also...

  8. 4 The Aptitude-Achievement Connection: Using an Aptitude Test to Aid in Allocating Educational Resources
    (pp. 32-56)

    In chapter 3 we learned that the PSAT, the shorter and easier version of the SAT, can be used effectively as one important part of selection decisions for scholarships. In this chapter we expand on this discussion to illustrate that the PSAT also provides evidence that can help us allocate scarce educational resources. We will look in a different, but allied arena: admission decisions for high school advanced placement courses.

    One of the lasting success stories in American education is the College Board’s Advanced Placement (AP) program. It originally allowed only a small elite to take advanced courses in high...

  9. 5 Comparing the Incomparable: On the Importance of Big Assumptions and Scant Evidence
    (pp. 57-72)

    In chapter 2, I alluded to the difficulty of obtaining comparable scores on two different achievement tests as part of the discussion of the recommendation to substitute achievement tests for aptitude tests in admissions decisions.¹ The details of equating two such tests, while important for the conclusions drawn in chapter 2, would have gotten in the way of the narrative flow. While postponing such an elaboration is acceptable, omitting it entirely is not. Now is the time.

    Making fair comparisons among groups of individuals who were administered different forms of a test is a task that ranges in difficulty from...

  10. 6 On Examinee Choice in Educational Testing
    (pp. 73-102)

    In chapters 2 and 5 we explored the difficulties encountered in making comparisons among candidates when the candidates themselves are the ones who decide which aspects of their experience and ability to present. The difficulties in making such comparisons fairly can be overwhelming. Let us now consider the more limited situation that manifests itself when the examination scores of students who are to be compared are obtained from test items that the students have chosen themselves. Such a situation occurred often in the beginning of the twentieth century, but gradually fell out of favor as evidence accumulated that highlighted the...

  11. 7 What If Choice Is Part of the Test?
    (pp. 103-109)

    In their search for the Holy Grail, both Walter Donovan and Indiana Jones arrived at the Canyon of the Crescent Moon with great anticipation. But after all of the other challenges had been met, the last test involved choice. The unfortunate Mr. Donovan chose first, and in the words of the Grail Knight, ʺHe chose poorly.ʺ The consequences were severe.

    In modern society we too must pass many tests, and we too must learn to choose wisely. The consequences are sometimes serious, although hopefully not as profound as for poor Walter Donovan.

    In the last chapter we discussed the issues...

  12. 8 A Little Ignorance Is a Dangerous Thing: How Statistics Rescued a Damsel in Distress
    (pp. 110-119)

    Since the passage of the No Child Left Behind Act the role of standardized test scores in contemporary education has grown. Students have been tested, and teachers, schools, districts, and states have been judged by the outcome. The pressure to get high scores has increased, and concern about cheating has increased apace. School officials desperately want high scores, but simultaneously do not want even the appearance of any malfeasance. Thus, after sixteen of the twenty-five students in Jenny Jonesʹs third-grade class at the Squire Allworthy Elementary School obtained perfect scores on the stateʹs math test, it was not entirely a...

  13. 9 Assessing Teachers from Student Scores: On the Practicality of Value-Added Models
    (pp. 120-138)

    In chapter 8 we saw how administrators of a school district, unused to the rigorous use of evidence in decision making, made a serious error in assessing the performance of one teacher. In this chapter we examine how the uncritical use of complex statistical machinery holds the promise of extending these sorts of errors nationwide.

    The No Child Left Behind Act (NCLB) has focused attention on having highly qualified teachers in every classroom. The determination of ʺhighly qualifiedʺ is based on such criteria as academic training, experience, and state licensure requirements. But there has been increased pressure to move beyond...

  14. 10 Shopping for Colleges When What We Know Ain’t
    (pp. 139-146)

    Will Rogersʹs well-known aphorism came to mind as I was reading a report by Eric Dash in theNew York Times¹ on a new and improved method for ranking colleges and universities, developed by Christopher Avery and his colleagues. Automobile commercials often tell us that we must choose cars wisely since, after a new home, it is the most expensive purchase we will make. And, because of this cost there are an unending number of reports comparing and measuring every aspect of each yearʹs crop of new vehicles, using a range of criteria fromConsumer Reportsʹ practical ones (comfort, reliability,...

  15. 11 Of CATs and Claims: The First Step toward Wisdom
    (pp. 147-155)

    The goal of this book parallels the principal goal of much of science—to try to understand the world through the integration of evidence and careful thought. I would like to end with a brief discussion of one instance where this tactic, which has always seemed completely convincing to me, was an utter failure in convincing some others. Such failures occur, alas, with disheartening frequency. A majority of this book has tried to establish that anecdotes are not data, and so basing decisions on anecdotes yields no guarantee of a positive outcome. As part of this chapter I try to...

  16. Epilogue
    (pp. 156-158)

    I began this book with an observation by Richard Feynman, one of the twentieth centuryʹs most revered scientists.

    It doesnʹt matter how beautiful your theory is, it doesnʹt matter how smart you are. If it doesnʹt agree with experiment, itʹs wrong.

    It seems to me that I am unlikely to go wrong ending with some allied observations.

    The title of this book was chosen purposely to be evocative; when we make guesses about how to improve any complex process—and education certainly falls into that category—we ought to use evidence to guide our guesswork. Such guesses I would class...

  17. References
    (pp. 159-164)
  18. Index
    (pp. 165-175)