Value-Added Assessment in Practice

Value-Added Assessment in Practice: Lessons from the Pennsylvania Value-Added Assessment System Pilot Project

Daniel F. McCaffrey
Laura S. Hamilton
Copyright Date: 2007
Edition: 1
Published by: RAND Corporation
Pages: 128
https://www.jstor.org/stable/10.7249/tr506cc
  • Cite this Item
  • Book Info
    Value-Added Assessment in Practice
    Book Description:

    Value-added assessment (VAA) systems use statistical techniques to analyze test-score data; VAA data is intended to help educators make more informed decisions about curriculum and instruction. The authors examined the rollout of Pennsylvania's VAA program, and found that, in its pilot phase, the program had little effect on student achievement and received limited use by most principals and teachers at schools participating in the program.

    eISBN: 978-0-8330-4443-3
    Subjects: Education

Table of Contents

  1. Front Matter
    (pp. i-ii)
  2. Preface
    (pp. iii-iv)
  3. Table of Contents
    (pp. v-vi)
  4. Figures
    (pp. vii-viii)
  5. Tables
    (pp. ix-xii)
  6. Summary
    (pp. xiii-xviii)
  7. Acknowledgments
    (pp. xix-xx)
  8. Abbreviations
    (pp. xxi-xxii)
  9. CHAPTER ONE Introduction
    (pp. 1-14)

    The use of standardized test scores to inform and motivate instructional change is now a cornerstone of education policy in the United States. Systems that rely on test scores to hold schools, teachers, and students accountable for performance have been adopted in one form or another by all states over the past decade or more. The reauthorization of the Elementary and Secondary Education Act, the No Child Left Behind Act of 2001 (NCLB), made such test-based accountability the crux of national education policy as well. Furthermore, schools and school districts are increasingly using assessment data as a tool for local...

  10. CHAPTER TWO Methods and Data
    (pp. 15-30)

    The primary goal of the study is to estimate the causal effects of VAA on student achievement and educational practice using the PVAAS pilot program. Because the pilot districts were not randomly chosen, the study uses a quasi-experimental design. We use matching methods to identify comparison districts that are similar to the pilot districts so that differences in student outcomes and educational practice can be treated as estimates of the causal effects of the pilot program.

    The study uses two types of outcomes: standardized achievement test scores to estimate the effects on student achievement, and survey responses from district administrators,...

  11. CHAPTER THREE Effects of PVAAS on Student Achievement
    (pp. 31-36)

    The primary question that motivated this study is whether district participation in the PVAAS pilot affects student achievement. As discussed in Chapter Two, the matching of PVAAS districts with similar districts that were not participating in PVAAS provides a quasi-experimental design that can support causal inferences about PVAAS effects. This chapter summarizes the results of the achievement analysis.

    Table 3.1 provides means and standard deviations of individual student-level PSSA test scores for the pilot and control districts by cohort, subject, and grade level for the 2003–2004, 2004–2005, and 2005–2006 school years. Cohort 1 school districts received their...

  12. CHAPTER FOUR Superintendents’ Responses to PVAAS
    (pp. 37-52)

    In this chapter, we present the results from the superintendent surveys. We begin with a comparison of PVAAS and non-PVAAS districts to a set of questions about data use. The second part of this chapter examines the responses of PVAAS participants to a set of questions about their understanding and use of the PVAAS program and the reports it generates. Because the pilot included only a relatively small sample of districts, the analyses have only limited power to detect differences between PVAAS and comparison district administrators. For PVAAS to affect student outcomes would probably require fairly substantial changes in actions...

  13. CHAPTER FIVE Principals’ Responses to PVAAS
    (pp. 53-70)

    This chapter presents results from the principal surveys. One of the goals of the principal surveys was to examine how participating in PVAAS affects principals’ attitudes and opinions about standardized test-score data and use of that data. The second goal was to understand how principals use the PVAAS data. However, one of the key findings from our surveys of principals from schools in PVAAS districts is that although nearly all of them reported having heard of PVAAS, large numbers of these principals reported that they had not seen the PVAAS reports, and a small percentage also incorrectly reported that their...

  14. CHAPTER SIX Teachers’ Responses to PVAAS
    (pp. 71-84)

    PVAAS is intended to facilitate effective use of achievement data for instructional decisionmaking. To the extent that providing PVAAS information leads to improved student achievement, it is likely that this effect occurs in large part as a result of actions taken by teachers. This chapter summarizes information from the teacher surveys. The surveys were designed to elicit information about teacher support for the PVAAS program and their understanding and use of the information that the program provides. As noted in Chapter Five, principals’ survey responses suggest that in many schools the information has not yet been used by or disseminated...

  15. CHAPTER SEVEN Summary and Implications
    (pp. 85-90)

    The goal of this study was to examine the utility of a value-added assessment system for promoting educational improvement. The primary research question focused on the effects of VAA program participation on student achievement. The study was also designed to improve our understanding about how to use VAA to improve schools. The questions about student achievement effects and educators’ practices are related: Both are intended to examine the promise of VAA for promoting improved student outcomes.

    The test-score results show no difference in the average performance of students from districts participating in the Pennsylvania Value-Added Assessment System pilot program and...

  16. APPENDIX Matching Results Summary Tables and Figures
    (pp. 91-102)
  17. References
    (pp. 103-106)