“Excellence” is the holy grail of academic life. Scholars strive to produce research that will influence the direction of their field. Universities compete to improve their relative rankings Students seek inspiring mentors. But if excellence is ubiquitously evoked, there is little cross-disciplinary consensus about what it means and how it is achieved, especially in the world of research. “The cream of the crop” in an English or anthropology department has little in common with “the best and the brightest” in an economics department. This disparity does not occur because the academic enterprise is bankrupt or meaningless. It happens because disciplines...

The institutional framework of evaluation that structures funding decisions in the academic world is not secret. Nevertheless, most of this “nuts and bolts” information—ranging from funding programs’ objectives and formal criteria of evaluation to how panels are formed and what panelists are asked to do—is not widely known. In this chapter, then, I describe the objectives of the five funding agencies studied and the formal evaluation criteria associated with those objectives; the structure of the evaluation process, including the role of the program officer, the selection of panelists and screeners, and the pre-deliberation ranking work; and, lastly, the...

The “gulf of mutual incomprehension” that Sir Charles Percy Snow famously posited as separating “scientists” from “literary intellectuals” also separates many social scientists from humanists, as well as many interpretative from more positivist researchers.¹ Long before they come to sit on funding panels, scholars absorb a variety of beliefs and perceptions about disciplinary cultures, especially each field’s approach to producing and evaluating knowledge.² They become familiar with these differences through intellectual activity—graduate training, mentoring, reading within and outside their fields, and so on—as well as through the formal and informal activities of everyday life at colleges and universities....

Almost without exception, the panelists I talked with consider their deliberations fair and their panel able to identify the top proposals.¹ Like the economist quoted earlier, the evaluators do not believe that their panel did a perfect job, but they do maintain that they identified the best proposals “on average.” They agree that meritocracy guides the process of selection and that unfettered market mechanisms generally determine the outcome of the competition. Some qualify their views by referring to the “role of chance and passion” in the process, and some acknowledge that “mistakes are made.” Overall, however, they are confident that...

InThe University in Ruins,the literary scholar Bill Readings remarks that “the idea of excellence” is ubiquitously evoked in academic contexts, yet little consensus exists concerning its meaning.

As a term, it “has the singular advantage of being entirely meaningless, or to put it more precisely, non-referential.”¹ As panelists are very much aware, excellence is a quintessential polymorphic term. A sociologist notes, “There are different standards of excellence, different kinds of excellence,” yet is nevertheless “pretty confident that I’d know it when I see it.” This chapter spells out what this “it” is that panelists easily recognize but cannot...

Although the criteria of interdisciplinarity and diversity are used to distinguish one proposal from another, they do not speak to quality per se. Instead, they concern characteristics of proposals and applicants that may push a very good but not perfect project or candidate over the proverbial bar. As such, diversity in particular can act as an additive, rather than as an alternative, standard of evaluation.

The discussion focuses first on the distinct challenges raised by the evaluation of interdisciplinary proposals and explores how panelists define “good interdisciplinarity” given the general lack of consensus in this area. That the necessary types...

In American higher education, excellence, merit, and quality are often captured by quantitative measures such as GRE and SAT scores (if you are a student) or number of citations (if you are a researcher). But when it comes to evaluating the proposed work of academics across disciplines, simple measures like these usually will not do. Instead, scholars are brought together to deliberate the merits of these proposals. As historians, political scientists, anthropologists, and literary scholars weigh in with their particular expertise, they also learn from one another, improvise, opine, convince, and attempt to balance competing standards. They strategize, high-ball, and...