A Systematic Process to Facilitate Evidence-Informed Decisionmaking Regarding Program Expansion

A Systematic Process to Facilitate Evidence-Informed Decisionmaking Regarding Program Expansion: The RAND Toolkit, Volume 3

Laurie T. Martin
Coreen Farris
David M. Adamson
Robin M. Weinick
Copyright Date: 2014
Published by: RAND Corporation
Pages: 86
https://www.jstor.org/stable/10.7249/j.ctt5vjvn4
  • Cite this Item
  • Book Info
    A Systematic Process to Facilitate Evidence-Informed Decisionmaking Regarding Program Expansion
    Book Description:

    Despite supporting more than 200 psychological health and traumatic brain injury programs, the Department of Defense lacks a way to develop, track, and assess the performance of this portfolio. RAND researchers developed a potential model and tools to support a centralized, systematic, and ongoing process to aid in making decisions around continued program support and expansion.

    eISBN: 978-0-8330-8483-5
    Subjects: Psychology, Management & Organizational Behavior, Health Sciences

Table of Contents

  1. Front Matter
    (pp. i-ii)
  2. Preface
    (pp. iii-iv)
  3. Table of Contents
    (pp. v-viii)
  4. Summary
    (pp. ix-x)
  5. Acknowledgments
    (pp. xi-xii)
  6. Abbreviations
    (pp. xiii-xiv)
  7. Glossary
    (pp. xv-xvi)
  8. CHAPTER ONE Introduction
    (pp. 1-4)

    Since 2001, more than 2.2 million service members have been deployed in support of military operations in Iraq and Afghanistan. Although most have coped well with deployment-related stresses, the high operational tempo of the past decade, longer deployments, and frequent redeployments have resulted in significant psychological health problems among service members and their families. To meet the growing need for services to support psychological health and care for traumatic brain injury (TBI), a wide range of programs have been developed and implemented throughout the Department of Defense (DoD) to prevent mental health problems, improve troops’ resilience, identify service members experiencing...

  9. CHAPTER TWO Assessing Programs for Possible Expansion
    (pp. 5-10)

    As noted in Chapter One, RAND’s report on DoD programs that support psychological health and care for TBI recommended creating a single decisionmaking entity within DoD that is charged with regular, ongoing data collection from all relevant programs (Weinick et al., 2011). This chapter describes the potential makeup of this entity, which we will refer to as a decisionmaking board, and a process that this board could follow. The chapter also presents criteria for rating programs and a decision tree for applying these criteria to program expansion decisions.

    One potential model for ongoing program reviews is the Human Research Protection...

  10. CHAPTER THREE Quality of Evaluation
    (pp. 11-14)

    Of the four criteria to guide program expansion, the quality of program evaluation(s) conducted to date is perhaps the most important. Without a good quality evaluation, it is difficult, if not impossible, to know whether the program in question is effective and should be considered for expansion. Assessing the quality of any evaluation is thus a necessary first step in determining whether a program ought to be considered for expansion. If a program has not yet conducted an evaluation of its effectiveness, it should be encouraged to do so to ensure that it is producing desired results.

    Our approach to...

  11. CHAPTER FOUR Program Effectiveness
    (pp. 15-18)

    The second criterion is the program’s effectiveness in achieving its intended outcomes. Provided that the quality of the evaluation design is adequate, program effectiveness—the extent to which a program influences the targeted outcome—can be assessed. Using information provided in the Program Abstraction Form about the evaluation design and program’s effectiveness, the reviewer rates the program as effective (moderate to large improvement), effective (small improvement), not effective, or harmful. Note that this decision should consider not only statistical significance, but also clinical or “meaningful” improvement. What constitutes a small, moderate, or large improvement is best left to the experts...

  12. CHAPTER FIVE Population and Policy Priorities
    (pp. 19-20)

    The last two criteria include the population and policy priorities. Programs serving high-priority populations and those addressing or responding to major policy priorities may warrant additional consideration in the decisionmaking process. It is expected that only a very small number of populations and priorities will be rated as high priority.

    A population may be considered higher priority if there is an identified need or gap in services for that population or if the population is at particularly high risk for a given outcome. Although the Program Abstraction Form (see Appendix A) will provide some information about the priority of the...

  13. CHAPTER SIX Decisionmaking Regarding Program Expansion
    (pp. 21-26)

    Once the RPET has been completed, summary ratings from the tool can be used to support and facilitate DoD decisions around program support and expansion. The RPET was designed to support, not supplant, human decisionmaking and as such does not provide a definitive answer to whether a program should be expanded. Nor does it provide insight into the best approach for expanding promising programs. This chapter addresses two critical questions: (1) “Should the program be expanded or not?” and (2) “If the program should be expanded, then how?” by revisiting the decision tree presented in Chapter Two, discussing a number...

  14. APPENDIX A Program Abstraction Form
    (pp. 27-38)
  15. APPENDIX B The RAND Program Expansion Tool and Instruction Manual
    (pp. 39-46)
  16. APPENDIX C Instruction Manual for the RAND Program Assessment Tool
    (pp. 47-68)
  17. References
    (pp. 69-70)