Attention and dual coding theory. A study on the interaction.

We have conducted two experiments on subtitling and its presentation as part of the HBB4ALL European project. Nowadays, the tendency for content consumption is shifting towards an “on demand” dynamic performed on an array of different devices. With that in mind, not only did we explore the effect the audio language has on the way we process content but also the ways in which it is affected when using different devices to consume content. In the first experiment we explored the relation between attention, subtitles and information processing. We presented the participants with two sets of clips, one with audio in English and the other with audio in Norwegian in order to explore the effect on the reading of subtitles (in Polish). As predicted by the Dual Coding Theory (Paivio, 1986) multiple channel information processing is found to be a strong facilitator towards better performance in content comprehension. Moreover, attention resource distribution is also found to be altered by the experimental conditions. Resource allocation and demand evaluation mechanisms (Kahneman, 1973) are considered to be the main causes of those differences. In the second experiment the focus is more applied. A comparison was made between information processing on different devices in order to explore how reading and processing behaviour changes from platform to platform. To assess whether or not there are differences between the different platforms, we presented a set of clips in an unknown language and compared the results of comprehension and visual questionnaires as well as eye tracking readings. No significant differences were found across devices in content comprehension. It is not necessary to adapt subtitles to fit these devices as we are capable of compensating for any differences in their characteristics. The following general discussion considers the findings of the two experiments to be relevant to the understanding of our attentional and learning system. In both experimental situations, our cognitive processes are able to retrieve the necessary mechanisms to compensate for the differences made in the experimental manipulation.