Skip to Main Content
Have library access? Log in through your library
Surveying Subjective Phenomena, Volume 1

Surveying Subjective Phenomena, Volume 1

Copyright Date: 1984
Published by: Russell Sage Foundation
Pages: 512
  • Cite this Item
  • Book Info
    Surveying Subjective Phenomena, Volume 1
    Book Description:

    In January 1980 a panel of distinguished social scientists and statisticians assembled at the National Academy of Sciences to begin a thorough review of the uses, reliability, and validity of surveys purporting to measure such subjective phenomena as attitudes, opinions, beliefs, and preferences. This review was prompted not only by the widespread use of survey results in both academic and non-academic settings, but also by a proliferation of apparent discrepancies in allegedly equivalent measurements and by growing public concern over the value of such measurements.

    This two-volume report of the panel's findings is certain to become one of the standard works in the field of survey measurement. Volume I summarizes the state of the art of surveying subjective phenomena, evaluates contemporary measurement programs, examines the uses and abuses of such surveys, and candidly assesses the problems affecting them. The panel also offers strategies for improving the quality and usefulness of subjective survey data. In volume II, individual panel members and other experts explore in greater depth particular theoretical and empirical topics relevant to the panel's conclusions.

    For social scientists and policymakers who conduct, analyze, and rely on surveys of the national state of mind, this comprehensive and current review will be an invaluable resource.

    eISBN: 978-1-61044-538-2
    Subjects: Sociology

Table of Contents

Export Selected Citations Export to NoodleTools Export to RefWorks Export to EasyBib Export a RIS file (For EndNote, ProCite, Reference Manager, Zotero, Mendeley...) Export a Text file (For BibTex)
  1. Front Matter
    (pp. i-vi)
  2. Table of Contents
    (pp. vii-x)
  3. Contents, Volume 2
    (pp. xi-xii)
  4. Preface
    (pp. xiii-xvi)
    Charles F. Turner and Elizabeth Marlin
  5. Part I Background

    • [Part I Introduction]
      (pp. 1-2)

      Hasn’t it often occurred that instruments originally invented for record and computation have inadvertently so extended the concepts of the entity they were invented to measure (concepts of space, etc.) in the mind and imagination that employed them, that they may metaphorically be said to have extended the original boundaries of the entity measured?

      Primary responsibility for original drafting of this part of this panel’s report was assumed by Duncan (1.2, 1.3), Fischhoff (1.4), and Turner (1.1, 1.4, 1.5). Sec Note on Authorship, page vi....

    • 1 Introduction
      (pp. 3-22)

      Questions about the reliability, validity, and relevance of survey measurements are not new, and we are not the first committee to study those questions. In 1945, the National Research Council (NRC) joined with the Social Science Research Council to establish a Joint Committee on the Measurement of Opinion, Attitudes and Consumer Wants. With financial support from the Rockefeller Foundation, a committee of distinguished social scientists, statisticians, and practitioners was established under the chairmanship of Samuel Stouffer.¹ Among the major topics identified by this group at its first meeting was “the validity of statements, opinion and information furnished by respondents.” The...

  6. Part II Uses and Abuses of Surveys

    • [Part II Introduction]
      (pp. 23-24)

      Once upon a time, runs an old Wendish legend, there was a peasant girl of Dehsa who lay on the grass at midday and slept. Her betrothed sat by her.… Then the noon-woman came and questioned him. Each time he answered her she put fresh questions to him. As the clock struck one his heart stopped beating. The noon-woman had questioned him to death.

      Primary responsibility for original drafting of this part of the panel’s report was assumed collaboratively by DeMaio, Marsh, and Turner. Section 2.1 was contributed by Duncan. See Note on Authorship, page vi....

    • 2 The Development and Contemporary Use of Subjective Surveys
      (pp. 25-60)

      At least two rather distinct origins of modern polls or surveys can be discerned. The term “survey” was used in the nineteenth and early twentieth centuries for any kind of (supposedly impartial) “first-hand investigation” of a community or group. “Social Surveys” were given this broad definition in an article in the 1934 edition of theEncyclopaedia of the Social Sciences, which refers to Le Play’s family budget studies, Booth’s monumental survey of poverty and working-class life in London, and subsequent studies in this vein. That encyclopedia also includes an article on “straw vote,” defined as “an unofficial canvass of an...

    • 3 Standards of Practice and the Misuse of Surveys
      (pp. 61-94)

      The practice of survey research is constrained by the limits of knowledge, by the resources available for a given study, and by the norms of the research community as to what constitutes adequate practice. Several attempts have been made to codify the latter in terms of standards for the conduct and reporting of survey research. While these standards are not always met, they do establish a lower limit for what is considered to be acceptable practice and, in some cases, provide mechanisms for the public exposure and censure of deficient practices.

      In this chapter we first briefly outline some relevant...

  7. Part III Measurement Issues

    • [Part III Introduction]
      (pp. 95-96)

      All measurement is befuddled with error. About this the scientist can and does do something; he ascertains the possible extent of the error, determines whether it is constant (biasing) or variable, or both, and ever strives to improve his instruments and techniques.

      Primary responsibility for original drafting of this part of the panel’s report was assumed by Abelson (4.3), Bailar (4.2), Duncan (6.1–6.4), Frankel (4.2), Kruskal (4.1), Schuman (5.1), Smith (5.2), and Turner (5.2, 5.3). See Note on Authorship, page vi....

    • 4 Measurement and Error: An Introduction
      (pp. 97-128)

      “What is truth? said jesting Pilate; and would not stay for an answer.” Thus Francis Bacon opens his essay “Of Truth.” Yet we cannot scoff at truth in trying to deal with measurement and error. We must begin with some sense or understanding or even faith that there is a true quantity, a true value under measurement, for error means discrepancy from truth.¹

      In the physical sciences, it is usually reasonable to think of real lengths, times, massesout there, so to speak, that are measured better and better as measuring equipment improves and as more resources are expended on...

    • 5 Variability in Survey Measurements of Subjective Phenomena: Empirical Results
      (pp. 129-178)

      The fundamental unit of meaning in a survey is the single question. There are smaller units to be sure, individual words being the most obvious, and in this chapter we will give examples showing how their variation can affect meaning. There are also larger units, primarily the scales and indices frequently employed in academic research, though much less often in poll reports by commercial organizations. But the discrete question plays a role in survey research analogous to that of atoms in chemistry; it is the distinctive unit of interaction in the course of administering a survey and the necessary building...

    • 6 Measurement and Structure: Strategies for the Design and Analysis of Subjective Survey Data
      (pp. 179-230)

      In this chapter we do not treat the topic of survey analysis comprehensively, but rather call attention to some approaches to the scientific analysis of survey data that are either novel or underused. We hope that the following material will stimulate readers to consider some of the less obvious possibilities for detecting and analyzing structure in survey data. Here, as in Part IV (where we review selected approaches that may lead to improvement in survey design), we identifY some lines of work that seem promising, without prejudice to many other approaches that may also merit consideration.¹

      We believe that, in...

  8. Part IV The Survey Interview Process

    • [Part IV Introduction]
      (pp. 231-234)

      … one has only learnt to get the better of words For the thing one no longer has to say, or the way in which One is no longer disposed to say it. And so each venture Is a new beginning, a raid on the inarticulate With shabby equipment always deteriorating In the general mess of imprecision of feeling.…

      Primary responsibility for original drafting of this part of the panel’s report was assumed by Abelson (9.3), Fischhoff (7.2), MacKuen (7.1), Martin (8.1–8.5, 9.1, 9.2, 9.4), and Nerlove (7.3). See Note on Authorship, page vi.

      In this part, we deal...

    • 7 Conceptual Ambiguity in Surveys
      (pp. 235-256)

      Concepts used in social science theories have meaning not only in theory, but also in the real world. The theoretically appropriate definition of a social construct—such as public opinion, or risk, or unemployment—thus depends not only on scientific and measurement criteria, but also on how the concept is understood and used in the world.

      Lay and scientific (or technical) concepts do not necessarily match. For example, lay definitions of unemployment probably do not correspond exactly to the definition that is used in the Current Population Survey. When there is disjuncture between concepts as they are used by survey...

    • 8 The Role of the Respondent
      (pp. 257-278)

      The social context can constrain or facilitate the expression of attitudes and opinions, and therefore the feelings expressed in different social situations are not necessarily consistent. For example the question “How are you?” is likely to elicit different answers when asked by a friend in casual conversation or by a psychiatrist in a therapy session, and appropriately so; the feelings that are real and relevant in the two contexts are different.

      The influence of social context on attitude measurement is illustrated by an early analysis of public opinion in a small rural community by Schanck (1932). He found that churchgoers...

    • 9 The Question-and-Answer Process
      (pp. 279-302)

      Although question wording and question order sometimes have substantial effects on response distributions, survey researchers often cannot predict when such effects will occur or explain them when they do occur. This lack of knowledge suggests that the process by which respondents interpret survey questions and formulate answers to them is not yet well understood. Furthermore, it is probable that many survey artifacts represent systematic psychological phenomena that do not exist only in surveys. We therefore consulted relevant psychological literature in order to formulate hypotheses about psychological sources of bias in the question-and-answer process. In this chapter we review four analytical...

  9. Part V Improving Survey Measurement of Subjective Phenomena

    • [Part V Introduction]
      (pp. 303-304)

      This collection of rules will perhaps appear needlessly complicated if one compares it with the procedures in current use. All this apparatus of precautions may seem very laborious for a science which, up to this point, scarcely demanded from those who devoted themselves to it more than general and philosophical training. It is certain, indeed, that the practice of such a method cannot have for its effect the popularization of interest in sociological matters. When … we ask men to discard the concepts they are accustomed to apply to an order of facts, in order to reexamine the latter in...

    • 10 Recommendations
      (pp. 305-334)

      We come now to the portion of our report in which we must translate our work into advice. We cannot offer a series of dramatic and novel recommendations; too many capable critics and practitioners have been over the ground before us for this to be a realistic aspiration. Moreover, almost every recommendation must carry a tacit disclaimer: this advice has not been adequately tested. But we can offer a series of recommendations that could significantly improve the survey enterprise.¹

      Our recommendations are intended to advance three goals:

      —To create an improved climate for the conduct of polls and surveys;


  10. Appendixes

    • A Separate Statements
      (pp. 337-340)
    • B Notes on the Size and Scope of the Survey Industry
      (pp. 341-352)
    • C Details of Newspaper Study
      (pp. 353-368)
    • D On the Review of Federal Surveys
      (pp. 369-370)
    • E Details of Survey Results Cited in Union Carbide Example
      (pp. 371-376)
    • F Intersurvey Comparisons: Early Work
      (pp. 377-390)
    • G Intersurvey Comparisons: Recent Work
      (pp. 391-406)
    • H Scheme for Classifying Survey Questions According to Their Subjective Properties
      (pp. 407-432)
  11. References
    (pp. 433-472)
  12. Name Index
    (pp. 473-482)
  13. Subject Index
    (pp. 483-495)