Publication Title
Multimodal learning with audio description. An eye tracking study of children's gaze during a visual recognition task
Publication Type
Book chapter
Editor(s)
Title of edited book
Proceedings of the ACM Symposium on Applied Perception (SAP ‘12)
Year of publication
2012
Pages
83-90
City
Language(s)

English

Modalities
Source
BITRA
Abstract
The paper explores the effects of adding audio description to an educational film on children's learning behaviour, manifested by a visual recognition task. We hypothesize that the multimodal educational setting, consisting of both verbal (film dialogue and audio description) and non-verbal (motion pictures) representations of knowledge, fosters knowledge acquisition as it provides information via multiple channels, which in turn strengthens memory retrieval. In the study we employ eye tracking methodology to examine the recognition of previously seen film material, testing whether audio description promotes recognition- rather than elimination-based decision-making in the visual recognition task. The analysis of first fixation duration and first run fixation count measures in the experimental and control groups partially confirmed our hypotheses. Children in the experimental group generally looked longer at the scenes they had seen, which supports the hypothesis that their decision was based on recognition, whereas children in the control group had longer fixations on scenes they were unfamiliar with, suggesting a decision based on elimination.
Submitted by Miguel Llanos on Wed, 26/06/2019 - 07:16