Handling Digital Brains

Handling Digital Brains: A Laboratory Study of Multimodal Semiotic Interaction in the Age of Computers

Morana Alač
Copyright Date: 2011
Published by: MIT Press
Pages: 218
  • Cite this Item
  • Book Info
    Handling Digital Brains
    Book Description:

    The results of fMRI brain scanning require extensive analysis in the laboratory. In Handling Digital Brains, Morana Alac shows that fMRI researchers do not sit passively staring at computer screens but actively involve their bodies in laboratory practice. Discussing fMRI visuals with colleagues, scientists animate the scans with gestures, and talk as they work with computers. Alac argues that to understand how digital scientific visuals take on meaning we must consider their dynamic coordination with gesture, speech, and working hands. These multimodal actions, she suggests, are an essential component of digital scientific visuals. A semiotician trained in cognitive science, Alac grounds her discussion in concepts from Peirce's semiotics and her methodology in ethnography and multimodal conversation analysis. Basing her observations on videotaped records of activity in three fMRI research labs, Alac describes scientists' manual engagement with digital visuals of the human brain. Doing so, she turns her attention to the issue of practical thinking. Alac argues that although fMRI technology directs scientists to consider human thinking in terms of an individual brain, scientific practices in the fMRI lab demonstrate thinking that engages the whole lived body and the world in which the body is situated. The turn toward the digital does not bring with it abstraction but a manual and embodied engagement. The practical and multimodal engagement with digital brains in the laboratory challenges certain assumptions behind fMRI technology; it suggests our hands are essential to learning, and the making of meaning.

    eISBN: 978-0-262-29547-5
    Subjects: Technology, Health Sciences

Table of Contents

  1. Front Matter
    (pp. i-viii)
  2. Table of Contents
    (pp. ix-x)
  3. Acknowledgments
    (pp. xi-xii)
  4. 1 In the fMRI Laboratory
    (pp. 1-22)

    It is 2002, and we are in a cognitive neuroscience laboratory at the University of California, San Diego. There, we encounter two researchers seated in front of a computer screen (figure 1.1). One of them, the laboratory director, Paul (a professor with a distinguished record of publishing and teaching in the field of cognitive neuroscience), is talking with a graduate student named Jane (a promising Ph.D. candidate in cognitive science) seated next to him (featured on the right in figure 1.1). The two researchers are engaged in the practice offunctional magnetic resonance imaging(fMRI).

    fMRI, together with its forerunner,...

  5. 2 fMRI Brain Visuals as Fields for Interaction
    (pp. 23-48)

    fMRI brain visuals are signs in a very straightforward sense: Cognitive neuroscientists observe the human brain and its processes by consulting its fMRI renderings. Alan Gross (2008: 281) has suggested that the character of fMRI brain visuals should be understood in terms of indexical signs. In his proposal, Gross refers to Peirce’s¹ famous distinction betweenicon,index, andsymbol,² articulated with respect to the relationship between the sign and its object (Peirce, C.P.: 4.531). For Peirce, whereas symbol, most closely related to the Saussurian language-like sign, is a conventional sign denoting its object with respect to a rule, iconic and...

  6. 3 fMRI Brain Imaging and the Experience of Sound
    (pp. 49-66)

    In cognitive neuroscience laboratories fMRI visuals are engaged in a multimodal manner. This multimodal engagement regards the bodily conduct involved in working and interacting with digital screens. fMRI practitioners do not only inspect brain scans by passively posing their gazes on the surfaces of the visuals; they also modify the aspects of the visuals by working on computer keyboards, gesturing in front of digital displays, touching the visuals, listening to the sound of the scanning machine, and moving their semiotic bodies while coordinating with each other.Seeingin the laboratory—intended as recognizing meaningful patterns in experimental data—thus refers...

  7. 4 fMRI Brain Visuals and Semiotic Bodies
    (pp. 67-92)

    To continue to unpack the claim that digital scientific visuals are fields for interaction, this chapter focuses on semiotic bodies. The discussion is grounded in two excerpts from an interaction where the fMRI practitioners are involved in reading brain visuals as a part of work and apprenticeship activity. Whereas the previous chapter described sound, this chapter focuses on bodily conduct. It indicates how practitioners engage their gesturing hands, nodding heads, moving necks, hunching shoulders, and bending torsos to make sense of their experimental data.

    With respect to earlier approaches in the scientific study of the human mind, fMRI is considered...

  8. 5 The Semiotic Mind in the fMRI Laboratory
    (pp. 93-120)

    Reconceptualizing fMRI brain visuals as fields for interaction has consequences for the understanding of thinking in the laboratory. The fMRI researchers’ skill to see certain concrete and spatially represented biological phenomena is not confined to an individual brain but is a gradual achievement that involves a fine coordination between digital screens, researchers, and their hands, and an array of graphical inscriptions. Far from being only an intellectual operation, the newcomer learns to see by looking at brain visuals but also by touching the digital screen and coordinating it with charts, maps, sketches, and other paper-based visuospatial inscriptions. The details of...

  9. 6 Materiality of Digital Brains
    (pp. 121-146)

    Rather than dealing with biological matter, fMRI practitioners spend long hours in front of computer screens working with “digital brains.” This allows them to engage with their experimental data in an embodied manner. Because fMRI scans function as Peirce’s iconic signs, they generateeffects of similarity(Eco, 1999) not because they simplylook likethe brains, but because they afford certain actions that are in some ways analogous to the engagement with physical objects. Through this embodied engagement, the scans participate in what practitioners experience as the objects of their practice. This means that when practitioners account for features in...

  10. 7 Publishing fMRI Visuals
    (pp. 147-158)

    Throughout this book I have been suggesting that fMRI visuals are fields scientists actively engage with, rather than pictures and images they simply look at. By foregrounding the human hands as essential elements of scientific visuals, I have described how, during data analysis sessions, fMRI visuals do notstand fororindicatesomething but participate in enacting hybrid forms that practitioners experience as objects of their practice. This, however, does not mean that the visual character and the cultural shaping of fMRI evidence are of lesser relevance. In fact, the multimodal approach needs its visual component to be complete, and...

  11. 8 Conclusion
    (pp. 159-168)

    This book has showed how, by turning attention to the skillful (but habitually overlooked) orchestration of working hands, multimodal semiotic acts, and digital technology, we can get a deeper sense for how science is actually done. Digital video records, augmented with extensive fieldwork, despite its limitations, have provided a way to describe how practical thinking is accomplished, how professional skills are acquired, and how objects of enquiry are experientially understood. The intricacies that constitute such events remain invisible when practice is narrated to interviewers or when scientists report their results to larger audiences.

    By attending to how fMRI practitioners modify...

  12. Notes
    (pp. 169-178)
  13. References
    (pp. 179-194)
  14. Index
    (pp. 195-200)
  15. [Illustrations]
    (pp. None)