Multimodality and Augmented Reality (AR) books for children: an analysis of the literary work “Mur”

Authors

DOI:

https://doi.org/10.5007/1807-9288.2021.e81171

Abstract

In this article, I present an analysis of the children’s book of Augmented Reality (AR) Mur, by the Finnish Kaisa Happonen and Anne Vasko, based on the theory of multimodality. My main goal is to describe the media and modes used in the construction of this children's book of Augmented Reality and in so doing understand how these semiotic resources produce an organic sense of unity as well as aesthetic effects in this work. After a brief introduction on Augmented Reality and the work by Happonen and Vasko, there is a reflection on the field of studies of multimodality, with emphasis on its main operational concepts: semiotic resources, means and modes. In the second part, I mobilize these concepts to explain how these technological resources are used in each medium. In this same section, two main modes of the work are addressed: image and writing. Finally, in the conclusion, there is a reflection on how the modes and media are articulated in this work to produce a meaningful reading experience.

Author Biography

Edgar Roberto Kirchof, Universidade Luterana do Brasil

Possui graduação em Letras (Portugês/Alemão) pela Universidade do Vale do Rio dos Sinos (1995), graduação em Teologia pela Escola Superior de Teologia (1998), mestrado em Ciências da Comunicação pela Universidade do Vale do Rio dos Sinos (1997) e doutorado em Lingüística e Letras pela Pontifícia Universidade Católica do Rio Grande do Sul (2001), tendo realizado um Pós-Doutorado em Estética e Biossemiótica na Universidade de Kassel, Alemanha (2005). É bolsista produtividade pelo CNPq, membro do CA de Ciências Humanas e Sociais da FAPERGS e editor da Revista Textura. Atualmente é professor adjunto da Universidade Luterana do Brasil, atuando, como docente e pesquisador, no Programa de Pós-Graduação em Educação (PPGEDU) e como docente no Curso de Letras. Seus principais temas de pesquisa são as relações entre a Literatura, a Cultura Digital e a questão das Diferenças, com foco na Literatura infanto-juvenil. Atua nas seguintes áreas: Teoria da Literatura, Semiótica, Estudos Culturais e Educação.

References

AARSETH, Espen J. Cybertext: Perspectives on Ergodig Literature. Baltimore & London: John Hopkins, 1997.

AKHTAR, Omar. Understanding Use Cases for Augmented, Mixed and Virtual Reality. Research Report Altimeter, 2018.

CORRERO, Cristina; GIL, M. Rosa; JUAN, Anna; REAL, Neus. Juego, lengua y literatura: de la oralidad a la multimodalidade. In: EDO, Mequè; BLANCH, Sílvia; ANTON, Montserrat (eds.). El juego en la primera infância. Barcelona: Ediciones OCTAEDRO, S.L., 2016, p. 113-130

BATEMAN, John; WILDFEUER, Janina; HIIPPALA, Tuomo. Multimodality: A Problem-Oriented Introduction - Foundations, Research and Analysis. Berlin/Boston: Walter de Gruyter GmbH, 2017.

BELMIRO, Celia Abicalil. A multimodalidade na literatura infantil e a formação de professores leitores. RBLA, Belo Horizonte, v. 10, n. 2, 2010, p. 403-420.

BERRY, Rodney; WYSE, Lonce. The Music Table Revisited. Problems of Changing Levels of Detail and Abstraction in a Tangible Representation. In: O’HALLORAN, Kay L.; SMITH, Bradley A.. Multimodal Studies: Exploring Issues and Domains. London & New York, 2011, p. 76-94.

CRAIG, Alan B. Understanding Augmented Reality: Concepts and Applications. Elsevier Inc. Morgan Kaufmann, 2013. (e-book)

FREDERICO, Aline. Lendo um aplicativo: dimensões da construção de sentido na leitura literária digital na primeira infância. PERSPECTIVA, Florianópolis, v. 39, n. 1, jan./mar. 2021, p. 1-25.

GRANSTRÖM, B.; HOUSE, D.; KARLSSON, I. (eds). Mutlimodality in language and speech systems. Kluwer Academic Publishers, Dordrecht, 2002.

GREIMAS, Algirdas Julien; COURTÉS, Joseph. Dicionário de semiótica. São Paulo: Contexto, 2008.

HAMER, Naomi. The Design and Development of the Picture Book for Mobile and Interactive Platforms: ‘You get to BE Harold’s Purple Crayon’. In: HAMER, Naomi; NODELMAN, Perry; REIMER, Mavis (eds.) More Words about Pictures: Current Research on Picture Books and Visual/Verbal Texts for Young People. London & New York: Routledge, 2017, p. 63-80.

HAPPONEN, Kaisa; VASKO, Anne. L’orsetta Mur. Milão: D’Agostini, 2018. App desenvolvido pela STEP IN BOOKS, 2017.

HERMAN, David. Word-Image/Utterance-Gesture Case Studies in Multimodal Storytelling. In: PAGE, Ruth. New Perspectives on Narrative and Multimodality. London & New York, Routledge: 2010, p. 78-98

HERMAN, David; PAGE, Ruth. Coda/Prelude - Eighteen Questions for the Study of Narrative and Multimodality. In: PAGE, Ruth. New Perspectives on Narrative and Multimodality. London & New York: Routledge, 2010, p. 217-220.

JEWITT, Carey; BEZEMER, Jeff; O’HALLORAN, Kay. Introducing multimodality. Routledge: London & New York, 2016.

KALANTZIS, Mary; COPE, Bill; CHAN, Eveline; DALLEY-TRIM, Leanne. Literacies. Cambridge: University Press, 2016.

KOIE, Aksel. A Bear Called Mur: An Interview with Aksel Koie. Jun, 13, 2017. Disponível em: https://newbooksnetwork.com/interview-with-aksel-koie-ceo-of-step-in-books/. Último acesso em 05/05/2021.

KRESS, Gunther. Multimodality: a social semiotic approach to contemporary communication. London & New York: Routledge, 2010.

KRESS, Gunther. What is mode? In: JEWITT, Carey (ed.) The Routledge Handbook of multimodal analysis’. 2º ed. London & New York: Routledge, 2014, p. 60–75.

KRESS, Gunther; van LEEUWEN, Theo. Reading Images: The Grammar of Visual Design. (2º ed.) London & New York: Routledge, 2006.

KRESS, Gunther; van LEEUWEN, Theo. Multimodal Discourse: The Modes and Media of Contemporary Communication. London: Bloomsbury, 2001.

NIKOLAJEVA, Maria. The Verbal and Visual: The Picturebook as a Medium. In: SELL, Roger D. (ed.) Children’s Literature as Communication: The ChiLPA Project. Amsterdam: John Benjamins, 2002, p. 85–108.

TOOLAN, Michael. Electronic Multimodal Narratives and Literary Form. In: PAGE, Ruth (Ed.). New Perspectives on Narrative and Multimodality. London & New York, Routledge, 2010, p. 127-141.

WIGNELL, Peter. Picture Books for Young Children of Different Ages: The Changing Relationships between Images and Words. In: O’HALLORAN, Kay L.; SMITH, Bradley A.. Multimodal Studies: Exploring Issues and Domains. London & New York, 2011, p. 202-219

VIEIRA, Josenia; SILVESTRE, Carminda. Introdução à Multimodalidade: Contribuições da Gramática Sistêmico-Funcional, Análise de Discurso Crítica, Semiótica Social. Brasília, DF: J. Antunes Vieira, 2015.

Downloads

Published

2021-08-04

Issue

Vol. 17 No. 1 (2021): Multimodalidade em textos digitais - Parte I

Section

Dossiê

License

Copyright (c) 2021 Edgar Roberto Kirchof

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Authors who have their works published in Texto Digital agree that:

  1. Copyrights remain with the authors, who grant the journal the right of first publishing their submitted manuscripts. All materials published by the journalare under an Attribution 4.0 International - Creative Commons License, which allows them to be shared since authorship and first publication credits are mentioned.

  2. The Attribution 4.0 International - Creative Commons allows the copy and redistribution of the material in any medium or format, as well as its adaptation for any purpose, even commercially.

  3. Authors can take additional contracts for non-exclusive distribution of the version of their works published by our journal separately (e.g. to publish it in an institutional repository or as a book chapter) with both expressed authorship acknowledgment and Texto Digital’s first publication credit.