Skip to Main Content
Have library access? Log in through your library
Surveying Subjective Phenomena, Volume 2

Surveying Subjective Phenomena, Volume 2

Copyright Date: 1984
Published by: Russell Sage Foundation
Pages: 636
  • Cite this Item
  • Book Info
    Surveying Subjective Phenomena, Volume 2
    Book Description:

    In January 1980 a panel of distinguished social scientists and statisticians assembled at the National Academy of Sciences to begin a thorough review of the uses, reliability, and validity of surveys purporting to measure such subjective phenomena as attitudes, opinions, beliefs, and preferences. This review was prompted not only by the widespread use of survey results in both academic and non-academic settings, but also by a proliferation of apparent discrepancies in allegedly equivalent measurements and by growing public concern over the value of such measurements.

    This two-volume report of the panel's findings is certain to become one of the standard works in the field of survey measurement. Volume I summarizes the state of the art of surveying subjective phenomena, evaluates contemporary measurement programs, examines the uses and abuses of such surveys, and candidly assesses the problems affecting them. The panel also offers strategies for improving the quality and usefulness of subjective survey data. In volume II, individual panel members and other experts explore in greater depth particular theoretical and empirical topics relevant to the panel's conclusions.

    For social scientists and policymakers who conduct, analyze, and rely on surveys of the national state of mind, this comprehensive and current review will be an invaluable resource.

    eISBN: 978-1-61044-700-3
    Subjects: Sociology

Table of Contents

Export Selected Citations Export to NoodleTools Export to RefWorks Export to EasyBib Export a RIS file (For EndNote, ProCite, Reference Manager, Zotero, Mendeley...) Export a Text file (For BibTex)
  1. Front Matter
    (pp. i-viii)
  2. Table of Contents
    (pp. ix-xii)
  3. Contents, Volume 1
    (pp. xiii-xiv)
  4. Preface
    (pp. xv-xvi)
    Charles F. Turner and Elizabeth Martin
  5. Part I Measurement of Subjective Phenomena in the Social Sciences

    • [Part I Introduction]
      (pp. 1-2)

      The idea that people can be asked direct questions as a way of measuring their attitudes, opinions, and other subjective states is fundamental to most survey research. Despite the fact that all the social sciences draw upon data gathered in this way, the survey method of direct measurement is viewed with skepticism and ambivalence by many social scientists. The first part of Volume 2 offers an historical perspective on the ways social scientists have approached the problem of measuring subjective phenomena.

      In Chapter 1, Jean Converse traces the development of the attitude concept in psychology and sociology, and recounts the...

    • 1 Attitude Measurement in Psychology and Sociology: The Early Years
      (pp. 3-40)
      Jean M. Converse

      The study of attitudes began to flourish in academic social psychology in the early 1920s, well before the advent of the new opinion polls of 1935, and attitudes became a key piece of the mosaic that later became survey research. In later years, survey research became omniverous, taking in everything it could get—opinions, facts, information, behavior, preferences, beliefs, experiences, measures of personality and even of physiology. But attitudes were the first content. The quantification of attitudes represented in part an effort to apply to subjective phenomena in social science some of the precision of measurement of physical science. The...

    • 2 Utility in Economics: A Survey of the Literature
      (pp. 41-92)
      J. G. Tulip Meeks

      Whatisutility? To answer, “Happiness,” is to invite the further question what happiness is. Is it pleasure, satisfaction, contentment, absence of suffering, freedom, self-fulfillment, a high quality of life, serving others, obedience to God? Or all or none or some of these? Is it a sensation—as a tickle is—or is it rather, as the philosopher Ryle (19.54) suggested of pleasure, to do with putting your whole heart into what you are doing?

      Economists have not on the whole been troubled by definitional questions of this sort in their extensive use of the concept of utility.¹ At first...

    • 3 The Use of Survey Data in Basic Research in the Social Sciences
      (pp. 93-114)
      Stanley Presser

      Few innovations in the social sciences rival the importance of the modern survey. Both survey practitioners and their critics have suggested that research questions, concepts, and findings have all been shaped by the survey method. It is probably no coincidence that the blossoming of empirical social science and the development of survey research occurred nearly simultaneously. Yet the precise nature of the survey’s impact on the social sciences has received little systematic attention. Indeed, setting aside the question of influence, there is surprisingly little quantitative evidence simply about the extent and character of the survey’s use. Despite an essentially universal...

  6. Part II Quasi-Facts

    • [Part II Introduction]
      (pp. 115-116)

      Perhaps the most common distinction made among survey questions is the one between fact and opinion, where opinion includes attitudes, beliefs, values, preferences, and similar notions. Facts are often assumed to be straightforward (in terms of both their existence and measurement), while opinion is usually thought to be much less certain in either sense. Few doubt the value of surveys aimed at factual matters, but criticism of opinion surveys, or more broadly of surveys of subjective phenomena, extends to doubts about whether such reports are possible, meaningful, or worthwhile.

      The criticism might be avoided by restricting surveys to factual information—...

    • 4 The Subjectivity of Ethnicity
      (pp. 117-128)
      Tom W. Smith

      Basic background variables are commonly seen as concrete and objective factors. In fact they often have a large subjective component. Ethnicity is a prime example. First there is the difficult problem of defining what ethnicity is. It is most frequently seen as some form of a cultural heritage or identification that is defined by some combination of nationality, language, religion, and race (Isajiw, 1974). We will not even try to disentangle how one plucks “ethnicity” out of these and related factors. Are Jews from Poland Jews, Poles, Polish Jews, or Jewish Poles? If we look at Catholic and Protestant Germans,...

    • 5 Measuring Employment and Unemployment
      (pp. 129-142)
      Barbara A. Bailar and Naomi D. Rothwell

      Whether a person is employed, unemployed, or out of the labor force may seem at first a factual item, and one that can easily he verified. There are people, however, for whom labor force status is an attitude that cannot be verified from records. Whether or not a person will be reported in statistical tabulations as employed, unemployed, or out of the labor force may depend on various factors such as the weather or the interviewer’s skill; who responds for the person; the respondent’s interest, mood, or perceptions about the purposes and uses of the interview; the time of day;...

    • 6 Housing Research: Conceptual and Measurement Issues
      (pp. 143-156)
      Sandra J. Newman

      The impetus for all federal housing legislation and housing programs in this century has been the belief that housing plays an important role in promoting the health, safety, and well-being of the individual and the growth, wealth, and security of the nation. Since the 72nd Congress during the Great Depression, every successivc Congress has enacted legislation or conducted studies dealing with housing (Bureau of the Census, 1957). Yet decades after the first federal intervention in the housing sector, long after the first recognition of improved housing as an important and legitimate goal of government policy, and after numerous large-scale population...

  7. Part III Nonsampling Sources of Variability

    • [Part III Introduction]
      (pp. 157-158)

      Although commonly used standards of statistical inference take account of variability due to sampling, it is nonsampling factors inherent in the measurement process that sometimes account for the lion’s share of the variability in survey measurements. The authors in this section demonstrate that we are still a long way from fully understanding and controlling nonsampling sources of variability.

      In Chapter 7, Charles Turner reports several sets of anomalous survey findings that came to light serendipitously in the course of other investigations. These findings indicate that different survey organizations, asking the same (or almost the same) questions at the same time,...

    • 7 Why Do Surveys Disagree? Some Preliminary Hypotheses and Some Disagreeable Examples
      (pp. 159-214)
      Charles F. Turner

      This chapter reports a number of anomalies in the survey measurement of subjective phenomena. Initial examples are drawn from recent attempts to use survey measures as social indicators of phenomena such as the well-being of the population, public confidence in national institutions, and public support for science. Observed discrepancies between supposedly equivalent measures of the same phenomena indicate that survey measures of many subjective phenomena have large nonsampling variances that may contaminate both their univariate and multivariate response distributions. These nonsampling variances may confound attempts to measure population change in instances where complete survey replication is not performed. This is...

    • 8 Nonattitudes: A Review and Evaluation
      (pp. 215-256)
      Tom W. Smith

      Studies of voting behavior and other political matters in the fifties developed a picture of the American electorate that was startlingly at odds with the basic assumption of a rational citizenry as formulated in classic democratic theory. John Q. Voter was found to have (1) low levels of conceptualization with a limited and distorted ideological comprehension of issues; (2) little information about procedural details of the government, the identity or party of officeholders, and topical issues of the moment; (3) minimal political participation, with voting being the only political activity engaged in by a notable number of people; (4) weakly...

    • 9 Social Desirability and Survey Measurement: A Review
      (pp. 257-282)
      Theresa J. DeMaio

      Social desirability is generally considered to be a major source of response bias in survey research. Given the frequency with which it is mentioned as an explanation for particular results, definitions of the term are surprisingly rare and uninformative. Generally speaking, it refers to “a tendency to give a favorable picture of oneself” (Selltiz, Wrightsman, and Cook, 1976). A more specific definition (albeit somewhat circular) is included in the proceedings of a Health Survey Research Methods conference (U. S. Department of Health, Education, and Welfare, 1977b:47):

      … social desirability bias: answers which reflect an attempt to enhance some socially desirable...

    • 10 The Manner of Inquiry: An Analysis of Survey Question Form Across Organizations and Over Time
      (pp. 283-316)
      Jean M. Converse and Howard Schuman

      Nationwide sample polls and surveys have flourished in the United States for the last 40 years, and certain major survey organizations have been scrupulous in building a historical record of their work. The oldest and best-known survey, the Gallup Poll, has syndicated its questions and answers continuously since 1935 and has recently collected all its results in bound volumes (Gallup, 1972a).¹ The Harris Survey is of more recent vintage (1956), yet Harris and Associates has also begun to publish its syndicated findings in a permanent archive (Harris, 1971). Two academic organizations have been conducting surveys since the 1940s: the National...

  8. Part IV Some Statistical Models for Error and Structure in Survey Data

    • [Part IV Introduction]
      (pp. 317-318)

      The branch of statistical methods most relevant to the data collected in subjective suveys—methods for analyzing categorical data—was markedly underdeveloped until recently. In the last decade, however, there has been an impressive series of developments in this area (summarized in major textbooks by Bishop, Fienberg and Holland; Goodman; and Haberman). Dissemination of these new methods to survey practitioners—together with a heightened awareness of the need for such procedures—present the possibility of significant advances in the scientific use of subjective survey measurements.

      In contrast to procedures such as factor analysis and Jöreskog’s LISREL these new methods do...

    • 11 Some Statistical Models for Analyzing Why Surveys Disagree
      (pp. 319-366)
      Clifford C. Clogg

      In order to understand why different surveys disagree in the measurement of subjective phenomena, it is necessary to consider the various contexts under which responses are elicited. Question wording, question placement (in relationship to other questions), the use of “filters,” method used (telephone, mail, direct interview), interviewer quality, questionnaire length, and even time period are some of the possible contexts of interest. It is imperative that principles of experimental design be utilized in order to study context effects, involving at the very least the random assignment of contexts to individual respondents (see, for example, Cochran and Cox, 1957). (The proper...

    • 12 Rasch Measurement in Survey Research: Further Examples and Discussion
      (pp. 367-404)
      Otis Dudley Duncan

      This chapter continues the exposition in Section 6.4 of Volume 1, which should be read first. Reviews of work on Rasch models in psychometrics should also be consulted (Lumsden, 1976; Rasch, 1960/1980; Weiss and Davison, 1981; Wright, 1977), since it is assumed that the reader is acquainted with the rationale and motivation of Rasch’s approach. Application of it to survey data has only begun, although educational psychologists report some work with attitude questionnaires (Andrich, 1978a; 1978d; Wright and Masters, 1982). An alternative approach to latent-trait models applicable in survey work is presented by Reiser (1981). Lazarsfeld and Henry (1968:223-225) give...

    • 13 Measurement Error in Surveys
      (pp. 405-440)
      Judith T. Lessler

      This chapter summarizes the results of a taxonomy project established, in part, to review the literature on the terminology and methods used in dealing with errors in survey measurements. The project was entitled,A Taxonomy of Survey Errors¹ (Lessler, Kalsbeek, and Folsom, 1981), and focuses on four separate types of errors that can occur in surveys. The division into four categories of error corresponds to the four basic components of a survey:

      1. Definition of a target population and construction of some means for identifying and accessing this population;

      2. Specification of a sample design and selection of a sample;

      3. Solicitation of...

  9. Part V Putting Survey Measurements in Context

    • [Part V Introduction]
      (pp. 441-442)

      It is common to interpret survey and poll results in the wider context of events and trends in society at large. Such interpretations are usuallyad hoc, resting on a commentator’s intuitions and insights about social change. Chapters 14 to 16, in contrast, illustrate ways that survey data might be supplemented by auxiliary data to illuminate historical trends and variations in public attitudes. Each of these chapters analyzes trends over time in subjective measurements of an aspect of public opinion and charts the interrelation between such subjective survey measurements and independent measurements of objective phenomena.

      In Chapter 14, MacKuen tests...

    • 14 Reality, the Press, and Citizens’ Political Agendas
      (pp. 443-474)
      Michael B. MacKuen

      The independence of individuals in deciding what sorts of matters are politically important is an essential element in most democratic theory. After all, political decisions can only be made in subject areas that are in public view, and the ability to put matters on the political agenda is an important source of political power. An essential result of the last few years’ examination of the empirical record, however, is a skepticism about the autonomy ordinary citizens exercise in matters of public affairs in general and, specifically, in choosing among political priorities. The importance of setting a democratic agenda has been...

    • 15 Mass Media, Contraceptive Behavior, and Attitudes on Abortion: Toward a Comprehensive Model of Subjective Social Change
      (pp. 475-500)
      James R. Beniger

      Past work on monitoring subjective social change by means of objective indicators, usually survey measures of public attitudes and opinion, has tended to be narrowly focused. Virtually ignored is the broader context of news events, mass media reporting, and behavioral change that has long been known to influence subjective measures. At least three causal components that are necessary for a full understanding of subjective social change are usually overlooked. Those components are:

      1.The symbolic environment, that is, the context in which public attitudes and opinions are formed and changed. This involves both discrete cultural events such as marketing decisions and...

    • 16 The Popularity of Presidents: 1963–80
      (pp. 501-546)
      Michael B. MacKuen and Charles F. Turner

      For four decades, polls have sought to measure the popularity of incumbent presidents, and the results of these polls have been widely reported in the media. It is claimed that “amidst the avalanche of polling data filling our newspapers and airwaves daily, none commands more attention than the periodic reports of presidential popularity (Orren, 1978:35).”¹ Anecdotes of President Lyndon Johnson bandying about poll reports of his popularity are well known, and it is widely believed that the behavior of politicians is influenced by the behavior of these popularity measurements. As Brody and Page (1975:136-137) observe,

      We can be confident that...

    • 17 Cultural Indicators and the Analysis of Public Opinion
      (pp. 547-564)
      Elizabeth Martin

      The premise of this chapter is that participation in popular culture is an important vehicle by which beliefs, attitudes, and commitments held by members of a collectivity are created and communicated. Cultural symbolism represents a language in whose terms members of a culture may debate and affirm abstract issues of value and commitment. Often, the significance of seemingly mundane activities as a means of affirming common value orientations is not fully appreciated by either participants or observers. As Wittgenstein (1953: No. 129) notes, “The aspects of things that are most important for us are hidden because of their simplicity and...

    • 18 Do Polls Affect What People Think?
      (pp. 565-592)
      Catherine Marsh

      The idea that public opinion itself might affect what people think is not new. In their different ways, Rousseau’s notion of a general will and Durkheim’s conception of a “conscience collective” were both intended to sum up a force in society that was outside of and a constraint upon the individual members of the society. However, this conception of public opinion as the property of a society rather than of individuals within society proved rather difficult to measure, and the flowering of empirical social science in the twentieth century brought about an atomization of this concept, as it did so...

  10. Name Index
    (pp. 593-606)
  11. Subject Index
    (pp. 607-617)
  12. Back Matter
    (pp. 618-618)