Skip to Main Content
Have library access? Log in through your library
Handbook of Meta-analysis in Ecology and Evolution

Handbook of Meta-analysis in Ecology and Evolution

Julia Koricheva
Jessica Gurevitch
Kerrie Mengersen
Copyright Date: 2013
Pages: 592
  • Cite this Item
  • Book Info
    Handbook of Meta-analysis in Ecology and Evolution
    Book Description:

    Meta-analysis is a powerful statistical methodology for synthesizing research evidence across independent studies. This is the first comprehensive handbook of meta-analysis written specifically for ecologists and evolutionary biologists, and it provides an invaluable introduction for beginners as well as an up-to-date guide for experienced meta-analysts.

    The chapters, written by renowned experts, walk readers through every step of meta-analysis, from problem formulation to the presentation of the results. The handbook identifies both the advantages of using meta-analysis for research synthesis and the potential pitfalls and limitations of meta-analysis (including when it should not be used). Different approaches to carrying out a meta-analysis are described, and include moment and least-square, maximum likelihood, and Bayesian approaches, all illustrated using worked examples based on real biological datasets. This one-of-a-kind resource is uniquely tailored to the biological sciences, and will provide an invaluable text for practitioners from graduate students and senior scientists to policymakers in conservation and environmental management.

    Walks you through every step of carrying out a meta-analysis in ecology and evolutionary biology, from problem formulation to result presentationBrings together experts from a broad range of fieldsShows how to avoid, minimize, or resolve pitfalls such as missing data, publication bias, varying data quality, nonindependence of observations, and phylogenetic dependencies among speciesHelps you choose the right softwareDraws on numerous examples based on real biological datasets

    eISBN: 978-1-4008-4618-4
    Subjects: Ecology & Evolutionary Biology, Statistics

Table of Contents

  1. Front Matter
    (pp. i-vi)
  2. Table of Contents
    (pp. vii-x)
    (pp. xi-xvi)
    Julia Koricheva, Jessica Gurevitch and Kerrie Mengersen
  4. Section I: Introduction and Planning

    • 1 Place of Meta-analysis among Other Methods of Research Synthesis
      (pp. 3-13)
      Julia Koricheva and Jessica Gurevitch

      In the most general terms, meta-analysis is one method of research synthesis. Research synthesis may be defined as a review of primary research on a given topic with the purpose of integrating the findings (e.g., for creating generalizations or resolving conflicts). Research synthesis is central to the scientific enterprise. Without it, the evidence for various alternative hypotheses cannot be properly evaluated and generalizations cannot be reached, thus the advance of the scientific field as well as any potential practical applications are inhibited. Research synthesis can be performed either qualitatively, in the form of a narrative review, or quantitatively, by employing...

    • 2 The Procedure of Meta-analysis in a Nutshell
      (pp. 14-24)
      Isabelle M. Côté and Michael D. Jennions

      It is said that a picture is worth a thousand words. Taking this at face value, we offer two figures to summarize the entire meta-analytic process. In Figure 2.1 we cover Part I, the initial stage of formulating a question and systematically searching the primary literature for suitable studies. In Figure 2.2 we cover Part II, the stage at which you extract data from publications, run statistical tests, and present and interpret your results. The two figures explicitly link each step to the relevant chapters (indicated by a circled number). These indicate where Chapters 3 to 27 fit into the...

  5. Section II: Initiating a Meta-analysis

    • 3 First Steps in Beginning a Meta-analysis
      (pp. 27-36)
      Gavin B. Stewart, Isabelle M. Côté, Hannah R. Rothstein and Peter S. Curtis

      This chapter is concerned with initiating the process of systematic research synthesis. Whymper’s advice following the death of his companions on the first ascent of the Matterhorn is as relevant to those synthesizing mountains of data as to those climbing real peaks. Failure to carefully define the problem and methods can result in serious, sometimes fatal errors and terminal falls for meta-analyses. Without a systematic approach to defining, obtaining, and collating data, meta-analyses may yield precise but erroneous results, with different types of sampling error (biases) and excess subjectivity in choice of methods and definition of thresholds; these devalue the...

    • 4 Gathering Data: Searching Literature and Selection Criteria
      (pp. 37-51)
      Isabelle M. Côté, Peter S. Curtis, Hannah R. Rothstein and Gavin B. Stewart

      People sometimes think that doing a meta-analytic review will give them an easy or rapid publication. Think again! There is no such thing as a good, quick and dirty meta-analysis, particularly when carrying out a systematic review. Gathering data for such a project can be a long and tedious affair—the most arduous and time-consuming stages of the review begin once you have formulated your question clearly and developed your protocol (Chapter 3). These stages consist of identifying and retrieving relevant sources, evaluating the retrieved information against specified selection criteria, and extracting and appraising data from the studies that survive...

    • 5 Extraction and Critical Appraisal of Data
      (pp. 52-60)
      Peter S. Curtis, Kerrie Mengersen, Marc J. Lajeunesse, Hannah R. Rothstein and Gavin B. Stewart

      The efficient and accurate extraction of data from primary studies is an important component of successful research reviews. It is one of the most time-consuming parts of a research review and should be approached with the goal of repeatability and transparency of results. Careful definition of the research question (Chapter 3) and identification of the effect size metric(s) to be used (Chapters 6 and 7) are prerequisite to efficient data extraction. The components of the data extraction process may be simpler or more complex, depending on the scope of the meta-analysis (e.g., how many studies are to be included, how...

    • 6 Effect Sizes: Conventional Choices and Calculations
      (pp. 61-71)
      Michael S. Rosenberg, Hannah R. Rothstein and Jessica Gurevitch

      One of the fundamental concepts in meta-analysis is that of the effect size. It is this concept that allowed the development of modern meta-analysis. An effect size is a statistical parameter that can be used to compare, on the same scale, the results of different studies in which a common effect of interest has been measured. There is no universal effect size measure in meta-analysis; it depends on a number of considerations, including the nature of dependent variables (e.g., binary or continuous) and whether we are comparing two groups or looking at one group (e.g., over time). In experimental studies,...

    • 7 Using Other Metrics of Effect Size in Meta-analysis
      (pp. 72-86)
      Kerrie Mengersen and Jessica Gurevitch

      Meta-analysis in ecology and evolutionary biology has generally been used to synthesize the results of independent experiments in order to assess overall results across studies, and to examine the causes of heterogeneity in those results due to modifying characteristics of the studies. The ability to do this is based on using standardized effect sizes that express the effect of interest on the same scale or in comparable terms across studies. In combining them, the effect sizes are weighted by their inverse sampling variances to account for differences in the precision of their estimates of the effect of interest, as well...

  6. Section III: Essential Analytic Models and Methods

    • 8 Statistical Models and Approaches to Inference
      (pp. 89-107)
      Kerrie Mengersen, Christopher H. Schmid, Michael D. Jennions and Jessica Gurevitch

      We come now to the statistical aspects of meta-analysis— namely,

      (1) the statistical model that describes how the study-specific estimates of interest will be combined;

      (2) the key statistical approaches for meta-analysis; and

      (3) the corresponding estimates, inferences, and decisions that arise from a meta-analysis.

      The technical details of the statistical approaches for analysis and inference appear in later chapters. Here, the focus is on providing an introduction and overview of these three components. First, we describe common statistical models used in ecological meta-analyses and the relationships between these models, showing how they are all variations of the same general...

    • 9 Moment and Least-Squares Based Approaches to Meta-analytic Inference
      (pp. 108-124)
      Michael S. Rosenberg

      Chapter 8 introduced variance and structural models and various statistical inference approaches used in meta-analysis. This chapter describes the basic details behind the moment and least-squares approach to meta-analysis. This approach represents “classic” meta-analysis; it is the one most frequently found in meta-analytic introductions and used in ecological meta-analyses to date. As discussed in Chapter 8, this approach to meta-analytic inference has the advantage of using fairly simple formulas (for basic structural models) that can be easily calculated, and it is clearly and directly comparable to common statistical concepts, such as weighted means and sums of squares. The disadvantages of...

    • 10 Maximum Likelihood Approaches to Meta-analysis
      (pp. 125-144)
      Kerrie Mengersen and Christopher H. Schmid

      In Chapter 9, moment-based and least squares approaches to estimating the parameters of the meta-analysis model were described and illustrated. This approach is appropriate if the aim of the analysis is simply to estimate the means and variances (that is, the moments of the model) and if these moments are well defined (and easily computed). For example, if the data are normally distributed, the first two moments (the mean and variance) completely describe the distribution and are easily calculated, as described in Chapter 9. Thus in a simple fixed or random-effects model, the method of moments (MM) is adequate.


    • 11 Bayesian Meta-analysis
      (pp. 145-173)
      Christopher H. Schmid and Kerrie Mengersen

      In this chapter, we introduce and describe a Bayesian approach to meta-analysis. We discuss the ways in which a Bayesian approach differs from the method of moments and maximum likelihood methods described in Chapters 9 and 10, and summarize the steps required for a Bayesian analysis. We will find that Bayesian methods provide the basis for a rich variety of very flexible models, explicit statements about uncertainty of model parameters, inclusion of other information relevant to an analysis, and direct probabilistic statements about parameters of interest. In a meta-analysis context, this allows for more straightforward accommodation of study-specific differences and...

    • 12 Software for Statistical Meta-analysis
      (pp. 174-192)
      Christopher H. Schmid, Gavin B. Stewart, Hannah R. Rothstein, Marc J. Lajeunesse and Jessica Gurevitch

      To conduct a meta-analysis, a researcher will need to use computer software to perform all but the simplest calculations. There are three types of software that can be used, depending upon the needs of the user. The first option is a spreadsheet, the second is a general purpose statistical package, and the third option is a program developed expressly to carry out meta-analysis.

      The most basic meta-analytic tools, such as weighted averages for fixed-and random-effects models, can be programmed by a knowledgeable user in a spreadsheet, such as Microsoft Excel. In the past, spreadsheets have often been used to carry...

  7. Section IV: Statistical Issues and Problems

    • 13 Recovering Missing or Partial Data from Studies: A Survey of Conversions and Imputations for Meta-analysis
      (pp. 195-206)
      Marc J. Lajeunesse

      Meta-analysis uses summary statistics like effect sizes to combine information from multiple studies. Yet a common problem encountered when collecting information for calculating effect sizes is the absence of data from published studies. The incomplete reporting of means, correlations, variances, and sample sizes can bias meta-analysis in many ways: reviews will have smaller sample sizes because studies with missing data are often excluded (Orwin and Cordray 1985, Follmann et al. 1992); effect size metrics like Hedges’dare disfavored because they require too many within-study statistics; approaches to pooling effect sizes will use less restrictive statistical models such as unweighted...

    • 14 Publication and Related Biases
      (pp. 207-236)
      Michael D. Jennions, Christopher J. Lortie, Michael S. Rosenberg and Hannah R. Rothstein

      Increased use of meta-analysis in ecology and evolution has stimulated greater consideration of the occurrence of publication bias in the scientific literature. A search of Web of Science showed that prior to 1995 no papers in ecology or evolution journals used the term “publication bias” in their title, abstract, or keywords (Fig. 14.1). More frequent occurrence of this term since 1995 seems to coincide with the promotion of meta-analysis in widely read ecology and evolution journals (e.g., Arnqvist and Wooster 1995a) and a subsequent increase in the publication of meta-analyses (Chapter 25). It is noteworthy that of the 84 papers...

    • 15 Temporal Trends in Effect Sizes: Causes, Detection, and Implications
      (pp. 237-254)
      Julia Koricheva, Michael D. Jennions and Joseph Lau

      The general aim of meta-analysis, as well as of any other form of research synthesis, is to combine scientific evidence scattered through a number of individual studies addressing the same topic. Evidence, however, is not static and tends to evolve over time due to changes in research methods, changes in the characteristics of the subjects being studied, and so forth. New studies might either strengthen or challenge the conclusions of previous reports, resulting in changes in the mean effect size and its variance over time. The magnitude and direction of the mean effect size, and the breadth of its confidence...

    • 16 Statistical Models for the Meta-analysis of Nonindependent Data
      (pp. 255-283)
      Kerrie Mengersen, Michael D. Jennions and Christopher H. Schmid

      In previous chapters we considered meta-analysis in which each primary study contributes one estimate of an effect size to the analysis. Each study and its corresponding estimate were treated as statistically independent. In many meta-analyses, however, independence is questionable because there are several effect estimates per study and/or some of the individual studies included in the meta-analysis might not provide independent estimates of the effect (e.g., if the studies have been conducted at the same site). Within-study nonindependence can arise due to multiple measures of the same effect on the same experimental units being made over time, multiple treatments being...

    • 17 Phylogenetic Nonindependence and Meta-analysis
      (pp. 284-299)
      Marc J. Lajeunesse, Michael S. Rosenberg and Michael D. Jennions

      An important statistical assumption of meta-analysis is that effect sizes are independent (Landman and Dawes 1982, Hedges and Olkin 1985, Gleser and Olkin 1994). This statistical independence means that the collection of effect sizes pooled in a meta-analysis does not have a correlated structure, and that each effect size (or sample) represents an independent piece of information. There are several reasons why a data set might have a correlated structure—for example, when multiple effect sizes are extracted from a single experiment or from different time points throughout a study. These forms of nonindependence are reviewed in Chapter 16. Here...

    • 18 Meta-analysis of Primary Data
      (pp. 300-312)
      Kerrie Mengersen, Jessica Gurevitch and Christopher H. Schmid

      The statistical methods that receive most focus in this book are those that allow the combination of summary data from each study. This chapter addresses the situation in which the primary data from each study are available for inclusion in the meta-analysis. This type of meta-analysis is appealing for a number of reasons. Most importantly, each study can be analyzed in a consistent manner, thus producing directly comparable effect estimates that have been similarly controlled for potential biases and other study-specific issues. This avoids, for example, the problem of attempting to combine published effect estimates that have been adjusted for...

    • 19 Meta-analysis of Results from Multisite Studies
      (pp. 313-320)
      Jessica Gurevitch

      Research synthesis in ecology has typically been based on literature reviews, as is also common in other fields. That is, a search is conducted for relevant data addressing a particular research question, the utility of published and unpublished data is assessed, and the results are synthesized to address questions based on all of the available evidence. However, another not uncommon scenario in ecological research occurs when a group of researchers wishes to address the same or similar questions, using similar methodology, and is subsequently interested in synthesizing their results both to determine overall answers to those questions and to determine...

  8. Section V: Presentation and Interpretation of Results

    • 20 Quality Standards for Research Syntheses
      (pp. 323-338)
      Hannah R. Rothstein, Christopher J. Lortie, Gavin B. Stewart, Julia Koricheva and Jessica Gurevitch

      What makes a quantitative research synthesis good or flawed? How can authors improve the quality of their review at various stages in the process of planning and carrying out a research synthesis? What criteria can editors and reviewers use to assess whether a quantitative synthesis should be accepted for publication, revised, or rejected? How can readers of published syntheses determine how to evaluate the quality of what they are reading, and in doing so decide whether or not to trust its results and their interpretation? In this chapter, we present guidelines to address these questions. We outline these guidelines in...

    • 21 Graphical Presentation of Results
      (pp. 339-347)
      Christopher J. Lortie, Joseph Lau and Marc J. Lajeunesse

      Data are the currency of science. Visualizations of data are thus one of the most compelling means to effectively communicate ideas in science (Cleveland 1985, Ellison 2001, Tufte 2001). Graphs present data in a visual form enabling the reader to read values, identify patterns, assess the outcome of a statistical technique, or analyze relationships within or between variables (Tukey 1972, Higgins and Green 2011). Not every graph has to serve all these functions; however, most sets of best practices for visualization in science also apply to meta-analyses. Unfortunately, there is a tendency to underreport data in ecology, erring on the...

    • 22 Power Statistics for Meta-analysis: Tests for Mean Effects and Homogeneity
      (pp. 348-363)
      Marc J. Lajeunesse

      A common justification for meta-analysis is the increased statistical power to detect effects over what is obtained from individual studies (Miller and Pollock 1994, Arnqvist and Wooster 1995a). A classic example in the medical sciences illustrates this advantage. Multiple independent studies were conducted to determine the effect of a blood-clot medication on the likelihood of surviving a heart attack; however, only 6 of 33 studies detected a statistically significant effect of this medication on patients. Pooling these studies with meta-analysis, however, Lau et al. (1992) found a significant and important overall effect: patients treated with this medication had a 20%...

    • 23 Role of Meta-analysis in Interpreting the Scientific Literature
      (pp. 364-380)
      Michael D. Jennions, Christopher J. Lortie and Julia Koricheva

      Scientists often deal with vast amounts of data, and the ability to summarize this information effectively is a major asset. Therefore, researchers make a considerable effort to acquire the necessary statistical skills to rigorously analyze each empirical data set that they collect. The same thoroughness should occur when writing up work for publication, which ideally requires synthesis of the scientific literature for each question that is answered (i.e. statistical test conducted) to place the results in context. This synthesis is a real challenge. One thing that makes it challenging for ecologists and evolutionary biologists to stay up to date with...

    • 24 Using Meta-analysis to Test Ecological and Evolutionary Theory
      (pp. 381-404)
      Michael D. Jennions, Christopher J. Lortie and Julia Koricheva

      The use of meta-analys is by ecologists and evolutionary biologists to tackle both large and small controversies was established from the beginning of the method’s application. In two of the earliest ecological meta-analyses, Järvinen (1991) quantified how female age affects laying date and clutch size in two bird species (a very focused question), while Gurevitch et al. (1992) asked the “big picture” question of what evidence there was that competition shapes communities. Subsequent promotion of meta-analysis (Arnqvist and Wooster 1995a, Gurevitch et al. 2001) and the development of software aimed at biologists (Rosenberg et al. 2000) led to rapid growth...

  9. Section VI: Contributions of Meta-analysis in Ecology and Evolution

    • 25 History and Progress of Meta-analysis
      (pp. 407-419)
      Joseph Lau, Hannah R. Rothstein and Gavin B. Stewart

      Meta-analysis was first introduced in medicine and the social sciences, and was used extensively in these fields decades earlier than in ecology and evolutionary biology. In this chapter we review the development of meta-analysis in medicine and the social sciences in order to illustrate its background and compare its application in these fields to those in ecology and evolution. For the purpose of this chapter, by “medicine,” we mean all aspects of health care and the biomedical sciences, including the diagnosis and treatment of individual patients, public health policy, health care financing and decision making, and basic and clinical biomedical...

    • 26 Contributions of Meta-analysis to Conservation and Management
      (pp. 420-425)
      Isabelle M. Côté and Gavin B. Stewart

      Meta-analysis was first brought to the attention of conservation biologists by Fernandez-Duque and Vallegia (1994). Syntheses of conservation-related literature had largely been, until then, narrative reviews. Fernandez-Duque and Vallegia (1994) outlined several advantages of meta-analytical techniques in terms of guiding conservation decisions. In particular, they highlighted the fact that committing a type II error (e.g., assuming that a particular human action has no effect when it in fact does) can have more serious consequences for conservation than making a type I error (e.g., assuming that an action has an effect when it really does not). When the action is potentially...

    • 27 Conclusions: Past, Present, and Future of Meta-analysis in Ecology and Evolution
      (pp. 426-432)
      Jessica Gurevitch and Julia Koricheva

      Meta-analysis has become established and widely applied in ecology, evolutionary biology, and conservation ecology (Gurevitch et al. 2001, Stewart 2010) since being first introduced in the early 1990s, as we have seen throughout this volume. During the past two decades since its introduction, methodology has been developed and refined, standards have become established, and the utility of these techniques has become more widely known among scientists in these fields. As in other disciplines, there was initially considerable controversy and resistance to the implementation of meta-analysis in ecology and evolution, followed by more general recognition of its value. We are now...

    (pp. 433-440)
    (pp. 441-446)
    (pp. 447-486)
    (pp. 487-488)
    (pp. 489-498)