Academic and Entrepreneurial Research

Academic and Entrepreneurial Research: Consequences of Diversity in Federal Evaluation Studies

Ilene Nagel Bernstein
Howard E. Freeman
Copyright Date: 1975
Published by: Russell Sage Foundation
Pages: 206
https://www.jstor.org/stable/10.7758/9781610448253
  • Cite this Item
  • Book Info
    Academic and Entrepreneurial Research
    Book Description:

    As social action programs in health, education, and welfare have expanded, interest has grown in evaluating their implementation and effectiveness. Policymakers and social planners--at all levels of government and in the private sector--are currently confronted with the problem of evaluating the large number of human service programs that compete for available resources.

    Academic and Entrepreneurial Researchpresents a systematic study of the expenditure of federal funds for evaluation research. It reviews federally-supported evaluations of programs, including evaluations of social change experiments and research-demonstration programs funded by the various executive departments of the federal government. Evaluation studies of these large-scale programs vary in scope, quality, and potential utility. Bernstein and Freeman examine all projects initiated during fiscal year 1970 in order to understand better the methods employed, the types of persons engaged in such research, and expectations regarding the utilization of findings.

    The book provides data about "high" and "low" quality evaluation research and contains recommendations for restructuring the entire evaluation research enterprise in light of the findings.

    eISBN: 978-1-61044-825-3
    Subjects: Population Studies, Political Science, Sociology

Table of Contents

  1. Front Matter
    (pp. i-vi)
  2. Table of Contents
    (pp. vii-viii)
  3. Preface
    (pp. ix-xiv)
    Howard E. Freeman
  4. CHAPTER ONE The Evaluation-Research Endeavor
    (pp. 1-22)

    There is no commonly accepted definition of the term “evaluation research.” As we use the term here, it is the systematic study of the operation of social-action, treatment, or intervention programs, and of their impact. Ideally, the goals of evaluation research are twofold: first, to supply information that allows policy-makers, planners, and professionals to make rational decisions about social-development and human-resource programs and thus to maximize the expenditure of economic and human resources; and, second, to add to knowledge available about social and interpersonal behavior and the social environment, and to explicate and refine the practice principles that underlie programming...

  5. CHAPTER TWO Design of the Study
    (pp. 23-32)

    This study was developed, as stated in the Preface, as part of a broad program at Russell Sage Foundation directed toward the improvement of evaluation-research activities. During the early stages of the development of the overall program, staff at the Foundation spent considerable time engaged in discussions with persons involved in the administration and funding of evaluation-research activities in the various departments of the executive branch of the federal government. In addition, government officials in the Office of the President responsible for domestic programs and officials in the Office of Management and Budget were contacted. Their views on the current...

  6. CHAPTER THREE Federally Supported Evaluation Research
    (pp. 33-64)

    Despite the mounting interest in evaluation research, the expansion of federal funds for studies and the proliferation of organizations engaged in investigations, as indicated previously, little is known about the sources of support, the background of persons involved in the enterprise, the organizational arrangements which surround activities, the methodological techniques employed, or the dissemination of research results. The research enterprise is inextricably bound up with the way the complex federal human-resource establishment is structured and operates. Thus, the organizational arrangements in evaluation research and the nature of activities undertaken need to consider the relations between funding groups and awardees, as...

  7. CHAPTER FOUR Measurement of Process in Evaluation Research
    (pp. 65-82)

    It is our contention, as discussed in Chapter 1, that evaluation-research efforts should contain procedures to measure process if the evaluation of a social-action program or intervention is to be comprehensive and have utility. Hyman and Wright (1967, p. 744) underscore this by stating, “Taking the work for the deed, an evaluator may try to observe the effects of nonexistent treatment, or a treatment very different from the one he thought was being examined.” A noteworthy case of this kind was made famous by the same two authors, who recount the tale of the evaluation of a propaganda campaign based...

  8. CHAPTER FIVE Measurement of Impact in Evaluation Research
    (pp. 83-98)

    The measurement of impact requires procedures that allow the evaluator (1) to document the extent to which the social-action program has or has not achieved its stated goals; (2) to attribute any effects or changes that are discovered to the implementation of the action program, ruling out rival hypotheses which might alternatively explain identified changes; (3) to delineate, if possible, the conditions or combination of conditions under which the program is most effective (i.e., those that yield maximum benefits and minimum costs); and (4) to delineate, if possible, any unanticipated consequences or side effects of the implementation of the program....

  9. CHAPTER SIX Academic and Entrepreneurial Research: A Multivariate Analysis
    (pp. 99-134)

    Even when self-reports of investigators on the procedures employed in evaluation research are the data base, there is great variation in design, sampling, measurement, and data-analysis procedures. Moreover, although many in the field assert that evaluations should be comprehensive (e.g., Suchman 1967 and Greenberg 1968), in many cases they are not.

    The preceding two chapters discussed some of the basic procedures employed and ordered studies according to variables which we maintain reflect differential quality of evaluation research. We related the ordering of studies on these indicators of quality to a series of measures that describe the evaluation-research enterprise. Our analysis...

  10. CHAPTER SEVEN Recommendations for a National Policy
    (pp. 135-152)

    From the standpoint of research quality, there is an obvious solution for improving evaluation research: have all studies undertaken in academic research centers, by Ph.D. psychology professors and those with similar training and orientation, include a commitment that the research results must be published in refereed social science journals, provide funds only as grants and have them awarded on the basis of peer-review committee judgments, allow a time period of from three to five years for the planning and conduct of the research, have the grants monitored by federal officials with a high degree of social-science graduate training and with...

  11. APPENDIX I: Agencies Audited
    (pp. 153-154)
  12. APPENDIX II: Survey on Evaluation Studies of Social Action Programs
    (pp. 155-168)
  13. APPENDIX III: Matrix of Chi Square Significance Tests for Associations between Selected Independent Variables
    (pp. 169-170)
  14. APPENDIX IV: Amount of Variance Explained by Each Variable on the Indexes of Research Quality
    (pp. 171-178)
  15. Bibliography
    (pp. 179-184)
  16. Index
    (pp. 185-187)