The heartbeat of multimodal creativity: A pilot heart rate study on objectivity and subjectivity in audio description
DOI:
https://doi.org/10.5007/2175-7968.2025.e106867Keywords:
multimodality, audio description, creativity, heart rate, experimental researchAbstract
The appropriateness of subjectivity or objectivity in audio description (AD) is an ongoing debate where the professional and the academic perspectives seem to collide. Whereas some professional guidelines support the view that an objective AD is the advisable option, some scholars have pointed out the advantages of more creative alternatives. Meanwhile, part of AD research is making inroads into experimental methodologies: AD and eye-tracking, text-to-speech AD, the automatic generation of AD, and physiological measurements applied to AD, such as heart rate, cortisol levels, or skin conductance response, among others. In this vein, our proposal presents an experimental pilot study using a heart rate sensor to measure partially sighted participants’ response to different combinations of subjective and objective multimodal components within the AD script. To do so, objective and subjective multimodal components from an AD clip from Netflix’s Money Heist were analysed. Afterwards, objective components were turned subjective (and the other way around), which resulted in two versions of the clip in subjective-objective terms: the original and the creative one. Both versions were recorded by a professional studio, participants were randomly assigned one of the clips, and they orally completed a demographic survey, a presence test, and a comprehension test followed by a semi-structured interview. Moreover, the UPTIVO Belt-D heart rate sensor was used to measure beats per minute, average heart rate, maximum heart rate, minimum heart rate, etc. Preliminary results were triangulated with the tests and the interview to obtain empirical data about the combination of subjective and objective multimodal components in AD that induces the highest heart rate and presence levels, enables better comprehension and is best regarded by users. Future research following this methodological design and involving an appropriate number of participants could provide empirical evidence about the immersive combination of objectivity and subjectivity in AD to be considered by guidelines.
References
ABNT. (2018). NBR 16452. Acessibilidade na comunicação: audiodescrição. ABNT (Associação Brasileira de Normas Técnicas).
ADLAB. (2013). Report on testing. Report no. 3, ADLAB (Audio Description: Lifelong Access to the Blind) Project.
AENOR. (2005). Norma UNE 153020. Audiodescripción para personas con discapacidad visual. Requisitos para la audiodescripción y elaboración de audioguías [The Standard UNE 153020. Audio description for visually impaired people. Guidelines for audio description procedures and for the preparation of audio guides]. AENOR.
Audio Description Coalition. (2009). Standards for Audio Description and Code of Professional Conduct for Describers Based on the Training and Experience of Audio Describers and Trainers from across the United States. Audio Description Coalition.
Baños, R. M., Botella, C., Alcañiz, M., Liaño, V., Guerrero, B., & Rey, B. (2004). Immersion and emotion: their impact on the sense of presence. CyberPsychology & Behavior, 7(6), 734–741. https://doi.org/10.1089/cpb.2004.7.734
Bardini, F. (2020). La transposició del llenguatge cinematogràfic en l’audiodescripció i l’experiència fílmica de les persones amb discapacitat visual [Cinematic language transposition in audio description and visually impaired people’s filmic experience]. [Doctoral Dissertation]. Universitat de Vic. https://www.tesisenred.net/handle/10803/669901#page=1
Cabeza-Cáceres, C. (2013). Audiodescripció i recepció: efecte de la velocitat de narració, l’entonació i l’explicitació en la comprensió fílmica. [Doctoral Dissertation]. Universitat Autònoma de Barcelona. http://hdl.handle.net/10803/113556
California Audio Describers Alliance. (2009). Audio Description Standards. California Audio Describers Alliance.
Cavallo, A., & Fryer, L. (2022). Integrated Access in Live Performance. Routledge.
Chaume, F. (2004). Cine y Traducción. Cátedra.
Csikszentmihalyi, M. (1988). The Flow Experience and its Significance for Human Psychology. In M. Csikszentmihalyi & I. S. Csikszentmihalyi (Eds.), Optimal experience: Psychological Studies of Flow in Consciousness (pp. 15–35). Cambridge University Press.
Delgado, H., Matamala, A., & Serrano, J. (2015). Speaker Diarization and Speech Recognition in the Semi-automatization of Audio Description: An Exploratory Study on Future Possibilities? Cadernos de Tradução, 35(2), 308–324. https://doi.org/10.5007/2175-7968.2015v35n2p308
Diaz Cintas, J., & Szarkowska, A. (2020). Experimental Research in Audiovisual Translation – Cognition, Reception, Production. The Journal of Specialised Translation, (33), 3–16. https://doi.org/10.26034/cm.jostrans.2020.542
Di Giovanni, E. (2014). Visual and Narrative Priorities of the Blind and Non-blind: Eye-tracking and Audio Description. Perspectives: Studies in Translatology, 22(1), 136–153. https://doi.org/10.1080/0907676X.2013.769610
Di Giovanni, E. (2020). Reception Studies and Audiovisual Translation. In Ł. Bogucki & M. Deckert (Eds.), The Palgrave Handbook of Audiovisual Translation and Media Accessibility (pp. 397–413). Palgrave Macmillan. https://doi.org/10.1007/978-3-030-42105-2_20
Dillon, C. (2006). Emotional Responses to Immersive Media. University of London.
Fels, D. I., Udo, P. J., Diamond, J. E., & Diamond, J. (2006). A Comparison of Alternative Narrative Approaches to Video Description for Animated Comedy. Journal of Visual Impairment & Blindness, 100(5), 295–305.
Frazier, G. (1975). The Autobiography of Miss Jane Pittman: An All-Audio Adaptation of the Teleplay for the Blind and Visually Handicapped. [Master’s Thesis]. San Francisco State University.
Fresno, N. (2017). Approaching Engagement in Audio Description. Rivista Internazionale di Tecnica della Traduzione, (19), 13–32. https://doi.org/10.13137/2421-6763/17349
Fryer, L. (2013). Putting It into Words: The Impact of Visual Impairment on Perception, Experience and Presence. [Doctoral Dissertation]. University of London. https://research.gold.ac.uk/id/eprint/10152
Fryer, L., & Freeman, J. (2012). Cinematic Language and the Description of Film: Keeping AD Users in the Frame. Perspectives: Studies in Translatology, 21(3), 412–426. https://doi.org/10.1080/0907676X.2012.693108.
Fryer, L., & Freeman, J. (2014). Can You Feel What I'm Saying? The Impact of Verbal Information on Emotion Elicitation and Presence in People with a Visual Impairment. In A. Felnhofer & O. D. Kothgassner (Eds.), Challenging Presence: Proceedings of the 15th International Conference on Presence (pp. 99–107). WUV Universität Wien.
González, C., & Jankowska, A. (2024). Measuring Psychological Immersion through Cardiovascular Response Measures in Subtitled Films. Translation, Cognition & Behavior, 7(1), 90–115. https://doi.org/10.1075/tcb.00090.gon
Greco, G. M., & Jankowska, A. (2020). Media Accessibility Within and Beyond Audiovisual Translation. In Ł. Bogucki & M. Deckert (Eds.), The Palgrave Handbook of Audiovisual Translation and Media Accessibility (pp. 57–81). Palgrave Macmillan.
Hasegawa-Johnson, M. A., Black, A. W., Ondel, L., Scharenborg, O., & Ciannella, F. (2017). Image 2 Speech: Automatically Generating Audio Descriptions of Images. In K. Smaïli, M. Diouri & K. Benzakour (Eds.), Proceedings of the International Conference on Natural Language, Signal and Speech Processing (pp. 1–5). ICNLSP. https://hdl.handle.net/2066/244344
Heeter, C. (1992). Being There: The Subjective Experience of Presence. Presence: Teleoperators and Virtual Environments, 1(2), 262–271. https://doi.org/10.1162/pres.1992.1.2.262
Holsanova, J. (2016). A Cognitive Approach to Audio Description. In A. Matamala & P. Orero (Eds.), Researching Audio Description: New Approaches (pp. 49–73). Palgrave Macmillan.
Hyks, V., & Blokland, B. (2023). European Blind Union Handbook for High Quality Audio Description on Screen. European
Iglesias Fernández, E., Martínez, S. M., & Núñez, A. J.C. (2015). Cross-fertilization between Reception Studies in Audio Description and Interpreting Quality Assessment: The Role of the Describer’s Voice. In R. Baños-Piñero & J. Díaz-Cintas (Eds.), Audiovisual Translation in a Global Context (pp. 72–95). Palgrave Macmillan. https://doi.org/10.1057/9781137552891_5
ITC [Independent Television Commission]. (2000). Guidance on Standards for Audio Description. ITC.
Iturregui-Gallardo, G. (2019). Audio Subtitling: Voicing Strategies and their Effect on Emotional Activation. [Doctoral Dissertation]. Universitat Autònoma de Barcelona. https://www.tdx.cat/handle/10803/667158
Jankowska, A. (2015). Translating Audio Description Scripts: Translation as a New Strategy of Creating Audio Description. Peter Lang.
Jankowska, A., Pilarczyk, J., Wołoszyn, K., & Kuniecki, M. (2022). Enough is Enough: How Much Intonation is Needed in the Vocal Delivery of Audio Description? Perspectives, 31(4), 705–723. https://doi.org/10.1080/0907676X.2022.2026423
Krejtz, K., Rutkowska-Siuda, D., & Krejtz, I. (2024). Gaze-Led Audio Description (GLAD). Concept and Application to Accessibility of Architectural Heritage. In A. Marcus-Quinn, K. Krejtz & C. Armando Duarte (Eds.), Transforming Media Accessibility in Europe (pp. 53–72). Springer.
Krejtz, I., Szarkowska, A., Walczak, A., Krejtz, K., & Duchowski, A. T. (2012). Audio Description as an Aural Guide of Children’s Visual Attention: Evidence from an Eye-tracking Study. In S. N. Spencer (Ed.), Proceedings of the Symposium on Eye Tracking Research and Applications (pp. 99–106). Association for Computing Machinery.
Kruger, J. L. (2010). Audio Narration: Re-narrativising Film. Perspectives: Studies in Translatology, 18(3), 231–249. https://doi.org/10.1080/0907676X.2010.485686
Kruger, J. L. (2012). Making Meaning in AVT: Eye Tracking and Viewer Construction of Narrative. Perspectives: Studies in Translatology, 20(1), 67–86. https://doi.org/10.1080/0907676X.2011.632688
Kruger, J., & Doherty, S. (2018). Triangulation of Online and Offline Measures of Processing and Reception in AVT. In E. Di Giovanni & Y. Gambier (Eds.), Reception Studies and Audiovisual Translation (pp. 91–109). John Benjamins.
Lessiter, J., Freeman, J., Keogh, E., & Davidoff, J. (2001). Cross-Media Presence Questionnaire: The ITC-Sense of Presence Inventory. Presence: Teleoperators and Virtual Environments, 10(3), 282–297. https://doi.org/10.1162/105474601300343612
Lombard, M., & Ditton, T. (1997). At the Heart of It All: The Concept of Presence. Journal of Computer-Mediated Communication, 3(2). https://doi.org/10.1111/j.1083-6101.1997.tb00072.x
López Rubio, M. (2024). Descripción y recepción de la audiodescripción de comedias en Netflix España para personas con discapacidad visual: una propuesta de mejora. [Doctoral Dissertation]. Universitat de València. https://hdl.handle.net/10550/98663
Mangiron, C. (2022). Audiovisual Translation and Multimedia Localization. In F. Zanettin & C. Rundle (Eds.), The Routledge Handbook of Translation and Methodology (pp. 410–423). Routledge.
Matamala, A., Soler-Vilageliu, O., Iturregui-Gallardo, G., Jankowska, A., Méndez-Ulrich, J.-L., & Serrano Ratero, A. (2020). Electrodermal activity as a measure of emotions in media accessibility research: methodological considerations. The Journal of Specialised Translation, (33), 192–151. https://doi.org/10.26034/cm.jostrans.2020.551
Mazur, I. (2020). Audio Description: Concepts, Theories and Research Approaches. In Ł. Bogucki & M. Deckert (Eds.), The Palgrave Handbook of Audiovisual Translation and Media Accessibility (pp. 227–247). Palgrave Macmillan.
Mazur, I. & Chmiel, A. (2016). Should Audio Description Reflect the Way Sighted Viewers Look at Films? Combining Eye-tracking and Reception Study Data. In A. Matamala & P. Orero (Eds.), Researching Audio Description: New Approaches (pp. 97–121). Palgrave Macmillan.
Morisset, L., & Gonant, F. (2008). Charte de l’audiodescription. Principes et orientations [Audio description charter. Principles and orientations]. Ministère des Affaires Sociales. https://www.alain-bensoussan.com/wp-content/uploads/255205.pdf
Naves, S. B., Mauch, C., Alves, S. F., & Araújo, V. L. S. (Orgs.). (2016). Guia para Produções Audiovisuais Acessíveis. Ministério da Cultura, Secretaria do Audiovisual.
Netflix (2024). Audio Description Style Guide (v2.5). https://partnerhelp.netflixstudios.com/hc/en-us/articles/215510667-Audio-Description-Style-Guide-v2-5
Ofcom. (2024). Ofcom’s Guidelines on the Provision of Television Access Services.
Orero, P., & Vilaró, A. (2012). Eye Tracking Analysis of Minor Details in Films for Audio Description. MonTi. Monographs in Translation and Interpreting, (4), 295–319. http://dx.doi.org/10.6035/MonTI.2012.4.13
Orrego-Carmona, D. (2019). Audiovisual Translation and Audience Reception. In L. Pérez-González (Ed.), The Routledge Handbook of Audiovisual Translation (pp. 367–382). Routledge.
Pedersen, J. (2020). Audiovisual Translation Norms and Guidelines. In Ł. Bogucki & M. Deckert (Eds.), The Palgrave Handbook of Audiovisual Translation and Media Accessibility (pp. 417–436). Palgrave Macmillan.
Perego, E., & Pacinotti, R. (2020). Audiovisual Translation through the Ages. In Ł. Bogucki & M. Deckert (Eds.), The Palgrave Handbook of Audiovisual Translation and Media Accessibility (pp. 33–56). Palgrave Macmillan.
Ramos Caro, M. (2013). El impacto emocional de la audiodescripción. [Doctoral Dissertation]. Universidad de Murcia. http://hdl.handle.net/10201/36475
Ramos Caro, M. (2015). The Emotional Experience of Films: Does Audio Description Make a Difference? The Translator, 21(1), 68–94. https://doi.org/10.1080/13556509.2014.994853
Ramos Caro, M. (2016). Testing Audio Narration: The Emotional Impact of Language in Audio Description. Perspectives, 24(4), 606–634. https://doi.org/10.1080/0907676X.2015.1120760
Remael, A., Reviers, N., & Vercauteren, G. (2014). Pictures Painted in Words. ADLAB Audio Description Guidelines. ADLAB Project.
Richardson, D. C., Griffin, N. K., Zaki, L., Stephenson, A., Yan, J., Curry, T., Noble, R., Hogan, J., Skipper, J. I., & Devlin, J. T. (2020). Engagement in Video and Audio Narratives: Contrasting Self-report and Physiological Measures. Sci Rep, (10), 2020. https://doi.org/10.1038/s41598-020-68253-2
Riva, G. (2011). Presence, Actions and Emotions: A Theoretical Framework. Annual Review of CyberTherapy and Telemedicine, 9, 2–5.
Rizzo, A., & Spinzi, C. G. (2023). Authorial (Audio) Description: Creativity in the Transfer of CSRs in Squid Game. Translation and Translanguaging in Multilingual Contexts, 9(3), 419–449. https://doi.org/10.1075/ttmc.00121.riz
Rojo López, A. M., Ramos Caro, M., & Espín López, L. (2021). Audio Described vs. Audiovisual Porn: Cortisol, Heart Rate and Engagement in Visually Impaired vs. Sighted Participants. Frontiers in Psychology, 12, 1–15. https://doi.org/10.3389/FPSYG.2021.661452
Romero-Fresco, P. & Chaume, F. (2022). Creativity in Audiovisual Translation and Media Accessibility. The Journal of Specialised Translation, (38), 75–101. https://doi.org/10.26034/cm.jostrans.2022.084
Romero-Muñoz, A. (2023). Multimodal Analysis as a Way to Operationalise Objectivity in Audio Description: A Corpus-based Study of Spanish Series on Netflix. Journal of Audiovisual Translation, 6(2), 8–32. https://doi.org/10.47476/jat.v6i2.2023.251
Romero-Muñoz, A. (2025). The Audio Description Script through the Lens of Multimodality: A Qualitative and Quantitative Analysis of the Meaning Codes in Elite. Parallèles, 1–15. https://doi.org/10.17462/para.2025.02.03
Romero-Muñoz, A. (in press). Towards an experimental multimodal configuration. A methodological proposal for eye-tracking the audio description script. In F. Chaume & L. Kostopoulou (Eds.), Translation as Experimentation. Audiovisual and Interactive Media. (forthcoming).
Saldanha, G., & O’Brien, S. (2013). Research Methodologies in Translation Studies. Routledge.
Sherry, J. J. L. (2004). Flow and Media Enjoyment. Communication Theory, 14(4), 328–347. https://doi.org/10.1111/j.1468-2885.2004.tb00318.x
Snyder, J. (2014). The Visual Made Verbal: A Comprehensive Training Manual and Guide to the History and Applications of Audio Description. American Council of the Blind.
Soler Gallego, S., & Luque Colmenero, M. O. (2023). Increased Subjectivity in Audio Description of Visual Art: A Focus Group Reception Study of Content Minimalism and Interpretive Voicing. Journal of Audiovisual Translation, 6(2), 55–76. https://doi.org/10.47476/jat.v6i2.2023.248
Szarkowska, A. (2011). Text-to-speech audio description. Towards wider availability of AD. The Journal of Specialised Translation, (15), 142–162. https://doi.org/10.26034/cm.jostrans.2011.509
Szarkowska, A. (2013). Auteur Description: From the Director’s Creative Vision to Audio Description. Journal of Visual Impairment & Blindness, 107(5), 383–387. https://doi.org/10.1177/0145482X1310700507
Szarkowska, A., & Jankowska, A. (2012). Text-to-speech Audio Description of Voiced-over Films: A Case Study of Audio Described Volver in Polish. In E. Perego (Ed.), Emerging Topics in Translation: Audio Description (pp. 81–94). Edizioni Università Di Trieste.
Szarkowska, A., & Mączyńska, M. (2011). Text-to-speech Audio Description with Audio Subtitling to a Nonfiction Film: A Case Study Based on La Soufriere by Werner Herzog. University of Warsaw.
Szarkowska, A., & Wasylczyk, P. (2014). Audiodeskrypcja autorska [Auteur audio description]. Przekładaniec, 28, 48–62. https://doi.org/10.4467/16891864PC.14.004.1711
Technical specification ISO/IEC TS 20071-21. (2015). Information technology, user interface component accessibility. Part 21. ISO.
Udo, J. P., & Fels, D. I. (2009). From the Describer’s Mouth: Reflections on Creating Unconventional Audio Description for Live Theatre. In A. Şerban, A. Matamala & J.-M. Lavaur (Eds.), Audiovisual Translation in Close-up (pp. 257–278). Peter Lang.
Walczak, A. (2017). Immersion in Audio Description: The Impact of Style and Vocal Delivery on Users’ Experience. [Doctoral Dissertation]. Universitat Autònoma de Barcelona]. https://www.tdx.cat/handle/10803/402401#page=1
Walczak, A., & Fryer, L. (2017). Creative Description: The Impact of Audio Description Style on Presence in Visually Impaired Audiences. British Journal of Visual Impairment, 35(1), 6–17. https://doi.org/10.1177/0264619616661603
Wilken, N., & Kruger, J. L. (2016). Putting the Audience in the Picture: Mise-en-shot and Psychological Immersion in Audio Described Film. Across Languages and Cultures, 17(2), 251–270. https://doi.org/10.1556/084.2016.17.2.6
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Cadernos de Tradução
This work is licensed under a Creative Commons Attribution 4.0 International License.
Copyright Notice
Authors hold the copyright and grant the journal the right for their articles' first publication, being their works simultaneously licensed under the Creative Commons Attribution License (CC BY), which allows the sharing of such works with its authorship acknowledged and its initial publication in this journal.
Authors are allowed to enter into separate additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or as a book chapter, with an acknowledgment of its initial publication in this journal).
