Interaction in 4-second bursts
The fragmented nature of attentional resources in mobile HCI

Antti Oulasvirta¹, Sakari Tamminen², Virpi Roto³, Jaana Kuorelahti²

¹Helsinki Institute for Information Technology HIIT

²Helsinki University of Technology TKK

³Nokia Research Center NRC

New! Please see our CHI Video Figure (16.8MB .mov) by Tuomo Nyyssönen of NRC

Problem: Why interaction breaks down in certain mobile situations? People seem to have tremendous capabilities for utilizing mobile devices in various innovative and fulfilling ways while on the move. However, there are times when the fluency of interaction breaks down dramatically. We all have experiences where we have to slow down, to postpone, or to stop interaction with a device entirely because of a cognitively taxing situation. And we sometimes have to invent novel ways of interaction or workarounds on the spot. Why does this happen and how does this happen? Theory: Cognitive Resource Competition We approached the question from the perspective of resource competition, a notion inspired by the Multiple Resources Theory of C.D. Wickens (1984, 1985). Different mobile situations trigger different tasks that we have to manage. For example, waiting for a metro to arrive is not simply about sitting idly with all cognitive resources free for time killing activities, but calls for action: estimating when the metro arrives, moving to a position where it can be perceived, continuously interpreting auditory sense data, monitoring how personal space is perhaps intruded by by passers, occasionally glancing the environment to see if the metro is coming etc. At times, when the tasks tap a same limited resource, resource-depletion might emerge, and the use of the resource has to be managed more carefully. Division and sequencing of the tasks emerges, which leads to fragmentation of attention and interaction with the mobile device. This paper analysed mobile situations from this perspective of cognitive resource competition, which in certain contexts is argued to lead to heavy multitasking and task-switching, which eventually breaks the fluent flow of interaction with mobile devices to fragments of just few seconds. The key notions are the capacities that the competition is for, their division and sequencing, and different types of interference that can occur. The Resource Competition Framework allows for making task-analysis based comparisons between mobile situations regarding the fragmentation of attention and interaction. Our results demonstrate that attentional limitations are very real and seriously constrain mobile interaction. Method: A semi-naturalistic field study
RCF predictions were tested in a semi-naturalistic field study measuring attention during the performance of assigned Web search tasks on mobile phone while moving through nine varied but typical urban situations. In the study, participants’ behavior, action, and context were recorded during Web search taking place in various mobile situations. Because uncontrollable aspects of the environment and participant behavior, special arrangements were needed in terms of both study design and apparatus. In operationalizing resource competition, our key measure was the deployment of visual gaze during Web search. Several others indicators were measured as well, such as the speed of walking and interaction with the device. We aimed at making the equipment setup as unobtrusive (for the user and other people) as possible. 30 g (Watek WAT 230A) minicams were used for video recording. This video stream was sent wirelessly to a receiver in the participant’s backpack. Since we knew that wireless video is susceptible to distractions, we backed up this view onto a tape carried by the experimenter. The participant carried most of the equipment in a backpack. It contained a microphone, a video camcorder, batteries, a wireless link receiver, and a 4-channel quad processor for building up one video from the four video streams.
Figure 2. Configuration of recording equipment.	Figure 3. Output video data integrated on-the-fly.

Results: Witnessing the Fragmentation in several mobile situations

The data conveys the impulsive, fragmented, and drastically short-term nature of attention in mobile interaction. Continuous attention to the mobile device fragmented and broke down to bursts of just 4 to 8 seconds from the 16 seconds of the laboratory, and attention to the mobile device had to be interrupted several task-switches: by glancing the environment up to 8 times during a page loading!

We also made observations on rather complex, adaptive strategies to compensate for resource-depletion:

- Reducing resources from secondary tasks

- Postponing tasks

- Calibration

- Brief sampling

- Turntaking capture

Figure 4. Duration of continuous attention to the mobile
device during page loading. Error bars denote 95 % CIs.

Figure 5. Number of attention-switches away from the mobile device during
page loading. Error bars denote 95 % CIs.

Discussion: The cost of mobility to human-computer interaction

The cost of mobility becomes indisputable when comparing it to the single-user, non-social laboratory condition. The difference is striking, as revealed by the almost eight-fold differences between the lab and the street conditions. Others’ recent results support the claim that attention in the office is much less fragmented. For example, looking at office multitasking, Czerwinski et al. (2004) reported only 0.7 interruptions during a task lasting 53 minutes on average. However, their results, relying on self-reporting and self-generated categories (of e.g. task, switch, and interruption), may underestimate the frequency. Indeed, González and Mark’s (2004) observation study revealed that information workers spent, on average, three minutes working on one event before switching to another. Going mobile takes multi-task processing to an extreme where interaction breaks down to bursts of just few seconds. These observations are also strong evidence for the importance of testing and experimenting in the field.

Last updated on 11th April 2005 by Antti Oulasvirta (HIIT/UH)

This is a web summary of Oulasvirta, A., Tamminen, S., Roto, V., & Kuorelahti, J. (2005). Interaction in 4-second bursts: The fragmented nature of attentional resources in mobile HCI. In Proceedings of SIGCHI Conference on Human Factors in Computing Systems, ACM Press, New York, pp. 919-928. The original paper can be found here.

<<Back to my homepage