Evaluating the Impact of Prevention and Early Intervention Activities on the Mental Health of California’s Population

Evaluating the Impact of Prevention and Early Intervention Activities on the Mental Health of California’s Population

Katherine E. Watkins
M. Audrey Burnam
Edward N. Okeke
Claude Messan Setodji
Copyright Date: 2012
Published by: RAND Corporation
Pages: 286
https://www.jstor.org/stable/10.7249/j.ctt3fh03j
  • Cite this Item
  • Book Info
    Evaluating the Impact of Prevention and Early Intervention Activities on the Mental Health of California’s Population
    Book Description:

    This report describes development of a statewide framework for evaluating and monitoring the short- and long-term impact of prevention and early intervention funding for mental health services on the California population. It details the approach, the data sources, and the frameworks developed: an overall approach framework and outcome-specific frameworks.

    eISBN: 978-0-8330-7818-6
    Subjects: History, Psychology

Table of Contents

  1. Front Matter
    (pp. [i]-i)
  2. Table of Contents
    (pp. ii-iii)
  3. Executive Summary
    (pp. iv-viii)

    In 2004, California voters passed the Mental Health Services Act. The Act was intended to transform California’s community mental health system from a crisis-­driven system to one that included a focus on prevention and wellness. The vision was that prevention and early intervention (PEI) services marked the first step in a continuum of services designed to identify early symptoms and prevent mental illness from becoming severe and disabling. Twenty percent of the Act’s funding was dedicated to PEI services. The Act identified seven negative outcomes that PEI programs were intended to reduce: suicide, mental health–related incarcerations, school dropout, unemployment,...

  4. Acknowledgments
    (pp. ix-ix)
  5. Abbreviations
    (pp. x-x)
  6. Chapter One Background
    (pp. 1-3)

    The Mental Health Services Act (hereafter, the Act), passed by California voters in 2004, called for transforming California’s community mental health system from a crisis-driven system to one that included a focus on prevention and wellness. Transformation was to be accomplished in part by dedicating a portion of the Act’s revenues to Prevention and Early Intervetion (PEI) services. The focus on prevention and wellness represented a historic change in the way that California addressed the problem of serious mental illness and the consequences of mental illness for individuals, families, and communities.

    The Act was intended to convert the public mental...

  7. Chapter Two Goals and Approach
    (pp. 4-5)

    A first step of the project was to more fully develop the goals for the evaluation framework. To accomplish this, we conducted interviews with 48 key stakeholders, as described in Chapter Three. During the discussions, many stakeholders observed that the seven negative outcomes identified in the Act are typically not directly and immediately affected by individual PEI programs; rather, these outcomes should be reduced over the long run if the entire system (the continuum of prevention, early intervention, and treatment) is strengthened. There was broad recognition that system changes take time and that the benefits of PEI efforts are likely...

  8. Chapter Three Methods
    (pp. 6-8)

    In this chapter, we describe how we developed and refined our evaluation frameworks and how we identified the databases that would be relevant for a statewide PEI evaluation.

    To develop the overall evaluation framework, we first needed to understand the goals of the legislation, how the goals were implemented, who the target population for PEI program activities was, and how the results would be used. We began by conducting key informant interviews with 48 individuals. Half were subject-matter experts with academic credentials in evaluation research or in measuring the key outcomes; the rest were either consumers or state or county...

  9. Chapter Four Evaluation Frameworks
    (pp. 9-13)

    We created two types of frameworks: an overall approach framework, as described in Chapter One and shown in Figure 4.1, and specific frameworks for each of the key outcomes specified by the Act, examples of which appear in Figures 4.2 (suicide prevention) and 4.3 (reduced suffering). As noted earlier, the key outcomes are broad social outcomes with multiple determinants. Therefore, in addition to looking at specific measures of each outcome, the frameworks also identified antecedent factors that were either known to, or that we hypothesized would, affect each outcome and that we posited to be influenced by PEI funding. That...

  10. Chapter Five Data Sources and Measure Specifications
    (pp. 14-14)

    As noted above, the data used to measure where funding from the Act is going (the second box) and how it is being used (the third box) will come primarily from programs and counties. Some programs and counties are already collecting this information; however, it is not collected in uniform ways across programs and counties, and counties do not provide these data to the state for analysis. One of the recommendations we make is that program-level data be collected using a uniform template so that the information can be aggregated and used for comparisons.

    In some cases, data not currently...

  11. Chapter Six Analytic Approaches to Evaluating the Impact of Prevention and Early Intervention Programs
    (pp. 15-20)

    The standard program evaluation framework considers the effect of a particular intervention or “treatment” on one or more outcomes. The challenge for the program evaluator is usually threefold: (1) to determine which outcomes are expected to be affected by the intervention, (2) to detect and measure changes in the outcomes of interest, and (3) to credibly attribute cause to effect (in other words, to determine how much of the observed change in the outcomes can be attributed to the intervention). The use of appropriate conceptual frameworks, theories of change, or more-complex theoretical models can help the evaluator in defining the...

  12. Chapter Seven Conclusions
    (pp. 21-25)

    In this chapter, we provide concluding comments on the utility of the evaluation framework if it were to be implemented using existing data sources and core program-level data, and we discuss the applicability of the framework to the broader evaluation of the Mental Health Services Act. We make recommendations for additional data development to support the evaluation framework. We also identify some areas in which supplemental evaluation activities could address important system evaluation priorities that cannot feasibly be addressed as part of an ongoing statewide data monitoring and evaluation capability. We conclude by recommending next steps for developing and implementing...

  13. Appendix A. Framework Logic Models
    (pp. 26-35)
  14. Appendix B. Database Descriptions
    (pp. 36-149)
  15. Appendix C Measures Descriptions
    (pp. 150-267)
  16. Appendix D Technical Approach
    (pp. 268-273)
  17. References
    (pp. 274-274)