Measuring Deeper Learning Through Cognitively Demanding Test Items

Measuring Deeper Learning Through Cognitively Demanding Test Items: Results from the Analysis of Six National and International Exams

Kun Yuan
Vi-Nhuan Le
Copyright Date: 2014
Published by: RAND Corporation
Pages: 119
https://www.jstor.org/stable/10.7249/j.ctt5vjvq3
Search for reviews of this book
  • Cite this Item
  • Book Info
    Measuring Deeper Learning Through Cognitively Demanding Test Items
    Book Description:

    Two test consortia are developing a next generation of tests to measure students’ attainment of the Common Core State Standards. These tests are expected to assess greater deeper learning skills than existing large-scale tests. A RAND study rated the cognitive demand of six nationally and internationally administered exams. The findings establish a benchmark for comparing how well the new generation of tests performs in assessing deeper learning.

    eISBN: 978-0-8330-8552-8
    Subjects: Education

Table of Contents

  1. Front Matter
    (pp. i-ii)
  2. Preface
    (pp. iii-iv)
  3. Table of Contents
    (pp. v-vi)
  4. Figures
    (pp. vii-viii)
  5. Tables
    (pp. ix-x)
  6. Summary
    (pp. xi-xviii)
  7. Acknowledgments
    (pp. xix-xx)
  8. Abbreviations
    (pp. xxi-xxii)
  9. 1. Introduction
    (pp. 1-8)

    In 2010, the William and Flora Hewlett Foundation’s Education Program launched its Deeper Learning Initiative, which emphasizes students’ mastery of core academic content and their development of deeper learning skills. The initiative focuses on enabling students to attain the following types of deeper learning skills:

    1.Master core academic content:Students will develop disciplinary knowledge, including facts and theories in a variety of domains—and the language and skills needed to acquire and understand this content.

    2.Think critically and solve complex problems:Students will know how and when to apply core knowledge by employing statistical reasoning and scientific inquiry to formulate...

  10. 2. Tests Included in This Study
    (pp. 9-18)

    In this chapter, we provide a brief description of each test, including its purpose, the framework used to guide its design, test item formats, subjects and grades tested, the number of released items analyzed, and the degree to which analyzed items were representative of their entire sample pools. We also present a review of prior research that compared benchmark tests.

    Ap offers college-level courses to advanced high school students around the world. Ap reaches a large number of schools and students worldwide, with an enrollment of more than 2 million students from 18,000 schools around the world. In the United...

  11. 3. Cognitive Demand Frameworks and Ratings for the Benchmark Tests
    (pp. 19-28)

    In this chapter, we describe the types of deeper learning skills that the six benchmark tests allowed us to analyze, how we chose the frameworks for our analysis of the extent to which these types of deeper learning skills were assessed by the benchmark tests, and the rating process.

    The Deeper Learning Initiative focuses on enabling students to emerge from their schooling with the ability to master core academic content knowledge, think critically and solve complex problems, work collaboratively, communicate effectively, and learn how to learn. Ideally, a deeper learning assessment would assess elements from each of the cognitive, intrapersonal,...

  12. 4. Findings
    (pp. 29-38)

    In this chapter, we present the results of our analysis. The findings represent the PARCC ratings under the modified weighting system, but the results and interpretations remained robust for the ratings obtained using the weighting system proposed by the PARCC developers.

    In total, we examined 790 mathematics items and 436 ELA items (418 reading and 18 writing items). Of the mathematics items analyzed, 45 percent were MC items, and 55 percent were OE items. The proportion of MC mathematics items varied by test, with 67 percent for AP and NAEP, 47 percent for TIMSS, 23 percent for PISA, and none...

  13. 5. Discussion and Implications
    (pp. 39-44)

    In this chapter, we summarize our results, discuss the implications for the Foundation’s Deeper Learning Initiative, and describe the limitations of this study.

    The overall composition pattern of cognitive demand in the six benchmark tests by subject and item format is similar to what was observed on state achievement tests (Yuan and Le, 2012). The cognitive demand of ELA tests is greater than that of mathematics tests. Format is associated with the cognitive demand of items, with OE items being more cognitively demanding than MC items.

    On average, these six benchmark tests had greater cognitive demand than did state achievement...

  14. Appendix A. Distributions of NAEP Items, by Grade and Specific Framework Dimension
    (pp. 45-46)
  15. Appendix B. Distributions of TIMSS Items, by Content and Cognitive Domain
    (pp. 47-48)
  16. Appendix C. Exemplary Test Items at Each DOK Level
    (pp. 49-58)
  17. Appendix D. Exemplary Test Items at Each PARCC Level, by Subject and Dimension
    (pp. 59-82)
  18. Appendix E. Distribution of Modified PARCC Ratings
    (pp. 83-84)
  19. Appendix F. Results for the Original PARCC Dimensions and Levels
    (pp. 85-92)
  20. References
    (pp. 93-97)