Questions About Questions

Questions About Questions: Inquiries into the Cognitive Bases of Surveys

JUDITH M. TANUR EDITOR
Copyright Date: 1992
Published by: Russell Sage Foundation
Pages: 328
https://www.jstor.org/stable/10.7758/9781610445269
  • Cite this Item
  • Book Info
    Questions About Questions
    Book Description:

    The social survey has become an essential tool in modern society, providing crucial measurements of social change, describing social life, and guiding government policy. But the validity of surveys is fragile and depends ultimately upon the accuracy of answers to survey questions. As our dependence on surveys grows, so too have questions about the accuracy of survey responses. Authored by a group of experts in cognitive psychology, linguistics, and survey research, Questions About Questions provides a broad review of the survey response problem. Examining the cognitive and social processes that influence the answers to questions, the book first takes up the problem of meaning and demonstrates that a respondent must share the survey researcher’s intended meaning of a question if the response is to be revealing and informative. The book then turns to an examination of memory. It provides a framework for understanding the processes that can introduce errors into retrospective reports, useful guidance on when those reports are more or less trustworthy, and investigates techniques for the improvement of such reports. Questions about the rigid standardization imposed on the survey interview receive a thorough airing as the authors show how traditional survey formats violate the usual norms of conversational behavior and potentially endanger the validity of the data collected. Synthesizing the work of the Social Science Research Council’s Committee on Cognition and Survey Research, Questions About Questions emphasizes the reciprocal gains to be achieved when insights and techniques from the cognitive sciences and survey research are exchanged.

    eISBN: 978-1-61044-526-9
    Subjects: Sociology, Psychology

Table of Contents

  1. Front Matter
    (pp. i-iv)
  2. Table of Contents
    (pp. v-viii)
  3. Preface: A Brief History of the Movement to Study Cognitive Aspects of Surveys and the Social Science Research Council (SSRC) Committee
    (pp. ix-xiv)
  4. Acknowledgments
    (pp. xv-xv)
  5. Members of the Social Science Research Council Committee on Cognition and Survey Research
    (pp. xvi-xvi)
  6. WORKSHOPS AND SEMINARS OF THE SOCIAL SCIENCE RESEARCH COUNCIL COMMITTEE ON COGNITION AND SURVEY RESEARCH
    (pp. xvii-xxii)
  7. PART I INTRODUCTION
    • 1 Cognitive Aspects of Surveys and This Volume
      (pp. 3-12)
      JUDITH M. TANUR

      Since sample survey technology originated in studies of London’s poor in the late nineteenth century, it has undergone continual refinements. Many of these refinements have improved sampling and estimation procedures. In the 1930s the United States government agencies adopted probability sampling methods as a means of ensuring the representativeness of a sample. But probability sampling did not replace the convenience and quota sampling strategies of prior decades in many other survey organizations until these strategies were proved untrustworthy when the pre-election polls of 1948 “elected” Thomas Dewey but the voters chose Harry Truman (see Fienberg and Tanur, 1989). By now,...

  8. PART II MEANING
    • 2 Asking Questions and Influencing Answers
      (pp. 15-48)
      HERBERT H. CLARK and MICHAEL F. SCHOBER

      On the face of it, survey interviews are simple. An interviewer steps into the home of a randomly selected member of the public, asks a series of questions, records the answers, and departs with new facts or opinions to add to her collection. (For convenience let us think of the interviewer as female and the respondent as male.) The information she takes away is determined by the questions she asks—how they are worded and what they require of the respondent. Properly designed, they will give her the facts and opinions she wants.

      But this view of survey interviews is...

    • 3 Direct Questioning About Comprehension in a Survey Setting
      (pp. 49-62)
      ROBERT M. GROVES, NANCY H. FULTZ and ELIZABETH MARTIN

      Survey questionnaires are useful measurement instruments to the extent that questions convey to the respondent the desired intent of the researcher. For much of survey research this has been assumed to be an unproblematic part of the measurement process. Yet throughout the history of survey research, there has been a stream of methodological research, now grown large with the application of concepts from cognitive psychology to surveys, which has challenged that assumption.

      One of the initial cognitive acts by a survey respondent upon hearing a question is the attribution of meaning to the oral presentation by the interviewer. This step...

  9. PART III MEMORY
    • 4 Personal Recall and the Limits of Retrospective Questions in Surveys
      (pp. 65-94)
      ROBERT W. PEARSON, MICHAEL ROSS and ROBYN M. DAWES

      Retrospective, or memory-based, responses to interview questions provide an indispensable window onto our past. Often, retrospective questions are the only means available to monitor individual or social states, or their change. The responses to such questions are used, for example, to:

      Estimate the nation’s monthly unemployment rate. (Respondents who are not employed are asked to report whether they looked for work within the last four weeks, to differentiate between the unemployed and those not in the labor force.)

      Estimate the lifetime prevalence rate of depression. (Respondents are asked whether they ever had a continuous period of two weeks or more...

    • 5 Improving Episodic Memory Performance of Survey Respondents
      (pp. 95-101)
      ROBERT T. CROYLE and ELIZABETH F. LOFTUS

      Many surveys challenge the cognitive abilities of respondents by asking them to recall the date, frequency, or characteristics of personal events. According to Tulving’s (1983) framework, this is a task of episodic memory retrieval. Chapter 4 of this volume demonstrates the inherent difficulties of this task. Individuals can adopt a number of different strategies in order to recall the distant past, and these efforts are frequently characterized by bias and error. The following contributions in chapters 6 to 8 describe recent investigations of methods for improving the episodic memory performance of survey respondents. Two of the contributions examine reports of...

    • 6 Memory and Mismemory for Health Events
      (pp. 102-137)
      ELIZABETH F. LOFTUS, KYLE D. SMITH, MARK R. KLINGER and JUDITH FIEDLER

      “During the past 12 months, about how many times did you see or talk to a medical doctor?” This question is one of many posed to respondents who participate in the National Health Interview Survey, a major government-sponsored sample survey designed to obtain information on the health of Americans. In this survey 50,000 people are asked each year to recall the occurrence of health-related conditions and their consequences, such as days lost from work. The resulting survey estimates of health problems and the utilization of healthcare services are used to formulate legislation relating to health programs. But surveys are not...

    • 7 Attempts to Improve the Accuracy of Self-Reports of Voting
      (pp. 138-153)
      ROBERT P. ABELSON, ELIZABETH F. LOFTUS and ANTHONY G. GREENWALD

      An important self-reported behavior for which survey data are beset with inaccuracies is that of voting. Comparisons of actual vote counts with retrospective answers to the question, “Did you vote?” consistently find a tendency toward overreporting of voting. This is the case both in studies that simply compare aggregate real voting percentages with aggregate claimed voting percentages and in studies that validate voting claims separately for each individual survey respondent. The latter procedure is troublesome to carry out, necessitating a good deal of search of public records, but it yields more sensitive information. In this chapter we shall confine our...

    • 8 Applying Cognitive Theory in Public Health Investigations: Enhancing Food Recall with the Cognitive Interview
      (pp. 154-170)
      RONALD P. FISHER and KATHRYN L. QUIGLEY

      Public-health epidemiologists rely heavily on histories of food consumption to investigate foodborne outbreaks. Those histories, usually obtained between two to seven days after the suspect meal, are used to identify specific foods consumed by ill and well persons present at the implicated meal, to determine which foods are associated with the illness (Bryan, 1973). If a laboratory examination of the foods served at the meal is impossible or inadequate, the food-consumption histories may be the only source of information available to link a foodborne outbreak with a specific food (Mann, 1981). In such cases it is particularly important to obtain...

  10. PART IV EXPRESSION:: THE CASE OF ATTITUDE MEASUREMENT IN SURVEYS
    • 9 Opportunities in Survey Measurement of Attitudes
      (pp. 173-176)
      ROBERT P. ABELSON

      A large proportion of the questions asked in surveys concern the attitudes of the respondents: attitudes toward controversial issues, political candidates, racial groups, consumer products, and so on. How attitudes relate to each other and how they depend on the demographic characteristics of the respondents are the stuff of much survey analysis. In laboratory studies, and in social psychology as well, attitude measures are often the dependent variables. The importance of the methodology of attitude measurement can hardly be exaggerated.

      In the following two chapters in this part, two key problem areas in the survey measurement of attitudes are discussed,...

    • 10 The Case for Measuring Attitude Strength in Surveys
      (pp. 177-203)
      JON A. KROSNICK and ROBERT P. ABELSON

      Attitude measurement is one of the most common goals of surveys. In the news media, for example, we frequently read reports of survey results revealing the proportions of Americans who approve and disapprove of the president’s performance in office, or the numbers of citizens who favor and oppose legislation outlawing abortion, or the percentages of people who prefer particular candidates running for public office. As this is true for the popular press, so it is true for surveys conducted by academics and by government. Many surveys are designed to measure behaviors as well, but attitude assessment has been, since the...

    • 11 New Technologies for the Direct and Indirect Assessment of Attitudes
      (pp. 204-238)
      JOHN F. DOVIDIO and RUSSELL H. FAZIO

      Recent theoretical and empirical advances in the domain of attitudes and social cognition offer considerable food for thought for the survey researcher. Basic theory and research have provided new insights regarding such matters as the structure of attitudes, their activation from memory, and the processes by which they guide the individual’s behavior. The theoretical distinctions that have been offered are ones that the survey researcher may find helpful when constructing a survey instrument. In addition, the methods by which these conceptual questions have been addressed are relatively novel to the field of social psychology, and these new “technologies” may themselves...

  11. PART V SOCIAL INTERACTION
    • 12 Validity and the Collaborative Construction of Meaning in Face-to-Face Surveys
      (pp. 241-268)
      LUCY SUCHMAN and BRIGITTE JORDAN

      For statistically based social science, survey research is the principal means of obtaining data about the social world. The interview from this point of view is a standardized data collection procedure that uses a questionnaire as its instrument of measurement. The interview is also, however, an essentially interactional event. From the moment that the interviewer sits down across from the respondent and begins to talk, the survey interview assumes and relies upon a wealth of conventions and resources from ordinary conversation. At the same time, the concern with standardized procedures and the statistical notion of error that standardization is intended...

  12. PART VI GOVERNMENT APPLICATIONS
    • 13 A Review of Research at the Bureau of Labor Statistics
      (pp. 271-290)
      CATHRYN S. DIPPO and JANET L. NORWOOD

      Federal statistical agencies have a long history of research on data quality and collection methods. Much of the early work was on improved methods of sampling and error calculation, but in recent years research has focused increasingly on methods for reducing nonsampling error. In fact, early research on the influence of the interviewer on the accuracy of survey data was done at the Bureau of the Census (Hanson and Marks, 1958), and Neter and Waksberg’s (1964) study of response errors in expenditure data collected in the Survey of Residential Alterations and Repairs was the first comprehensive investigation of the effects...

  13. Name Index
    (pp. 291-298)
  14. Subject Index
    (pp. 299-306)