The editors of [in]Transition are pleased to present a special issue of our journal dedicated to "The Poetics of Eye Tracking." The issue was proposed and edited by Tessa Dwyer (Monash University), Claire Perkins (Monash University), and Sean Redmond (Deakin University). Warm thanks to them and the issue participants, as well as to all the peer-reviewers for their work. Drew Morton steered the production of the issue on behalf of [in]Transition. 

The Poetics of Eye Tracking

by Tessa Dwyer, Claire Perkins, and Sean Redmond

The four videographic essays that make up this special edition have developed out of the work pioneered by the Eye Tracking and the Moving Image Research group. ETMI was set up in late 2012 by Jodi Sita, a neuroscientist, and Sean Redmond, a film and television theorist, with two central goals in mind: one, they wanted to utilise eye tracking technology more centrally in the analysis and examination of the moving image; and two, they wanted to draw together scholars and practitioners from the Sciences and the (Creative) Arts and Humanities so that different modes of enquiry, and theoretical and methodological apparatuses, were placed in the same analytical arena. They felt that by having a room full of filmmakers, artists, film and cultural theorists, screenwriters, visual ethnographers, vision scientists and neuroscientists, new and exciting conversations and deliberations about how viewers engage with the moving image would be born.  This would be in stark contrast to what they felt had gone before where eye tracking analysis of film had been dominated by cognitive, neo-formalist approaches (see Redmond, and Batty, 2015). The formation of the group, then, created a strong commitment to inter-disciplinary and cross-institutional relationships, and to what was considered a necessary dialogue between different disciplines united by a shared desire: to investigate vision regimes in relation to the affecting power and beauty of the moving image. From its inception ETMI was also committed to the poetics of eye tracking and its potential to recast the way research could and should be carried out (see Redmond, 2016). The very first seeds of ETMI involved the blossoming talk of how the videographic essay would best capture its always exciting and equally vexing findings.

In this issue the convergence comes to life, with four essays that each explore and demonstrate the natural fit of the videographic format for eye tracking research. The two phenomena converge around the problem of substitution: eye tracking research works with predominantly visual data that is then translated into prose, and the video essay interferes in the ekphrastic challenge of expressing spectatorship through the written medium. When drawing, as the ETMI group does, upon the former to examine the latter, the ability of the video essay to conduct analysis on the terms of its object – i.e., moving images and sounds – is uniquely valuable (see Keathley and Mittell, 2016). Preserving the live data of audiences watching images as they move enables the dynamic illustration that eye tracking as technology and methodology requires. Further, though, the video essay showcases the uncanny nature of eye tracking research, where researchers are compelled to watch the traces of their subjects watching. By putting the audience into this vital position of spectatorship, the video essay primes its viewer to experientially consider the poetic and phenomenological pathways that eye tracking can open up.

This collection showcases these diverse ways in which eye tracking can inform and support both critical and creative screen analysis, starting with Unseen Screens: Eye Tracking, Magic and Misdirection by Tessa Dwyer and Jenny Robinson. This piece introduces some key concepts and visualisation techniques in moving-image eye tracking research including attentional synchrony, edit blindness, heat mapping, and aggregate gaze plot data. Navigating through these fixtures, Unseen Screens drills down into misdirection, foregrounding continuities between seeing and not seeing, illusion and transparency, film and magic. This piece is followed by two interconnected works that unfold around the thematic of sound: Materialisation, Emotion & Attention: Tracking Sound’s Perceptual Effects in Film by Darrin Verhagen, and The Ear that Dreams: Eye Tracking Sound in the Moving Image by Sean Redmond. With humour and insight, Verhagen explores and debunks a range of sound/image theories and principles to argue for the ability of sound design to transform attention by deepening levels of engagement. Redmond further probes the affordances of film sound – its immersive, bi-sensorial and synesthetic effects – to explore the intensities that eye tracking data cannot image but only imagine. In this way, The Ear That Dreams prepares ground for embodied eye tracking methodologies. These tandem works on sound are followed by Dead Time, a piece by Catherine Fowler, Claire Perkins, and Andrea Rassell that institutes a dramatic change of pace, dramatising and massaging the tensions and challenges inherent within ‘slow cinema’. Dead Time reverses the usual tendency in eye tracking research to examine data through a quantitative lens. Instead, Fowler, Perkins, and Rassell ‘unbundle’ eye tracking to focus on the experiential idiosyncrasies of individualised viewing patterns. The order in which these works are presented moves from exposition to abstraction, orienting viewers initially through a mapping of misdirection and screen/sound relations before leading into an exploration of emotion, engagement, and affective poetics to settle finally upon an unflinching moment of meditative self-reflexion. Together these works chart eye tracking methods, approaches and data sets, yet they also acknowledge how charts and maps are themselves representations —part fiction, part fact. In doing so, they question and broaden understandings of eye tracking and the arts/science nexus.

References

Keathley, Christian and Mittell, Jason (2016). The Videographic Essay: Criticism in Sound and Image.

Redmond, Sean and Batty, Craig (2015). ‘Seeing into Things: Eye Tracking the Moving Image’. Refractory 25.

Redmond, Sean (2016). ‘The Love Particles of Eye Tracking’. brief 54: 111-120.