The invention of the “chromosound”: challenges for the development of a possible language

Desafíos para el desarrollo de un posible lenguaje

Authors

  • Sergio Basbaum Programa de Pós Graduação em tecnologias da inteligência e design digital TIDD Pontifícia Universidade Católica de São Paulo Depto de Computação

DOI:

https://doi.org/10.5007/1807-9288.2023.e96079

Keywords:

Chromossound, Chromossonia, Chromossonium, synesthesia, Dome aesthetics, Cognition, Neural Networks

Abstract

This article describes the Chromossonium, the project of an interactive environment conceived to host performances of Chromossonia, a synesthetic language for performance and composition. Chromossonia is based in a theoretical particle named Chromossound, which unites sound and color in a non-hierarchical way, and with which we expect creators to be able to compose scored pieces to be staged and performed in the Chromossonium. Following a description of how these elements are conceived, and which are its theoretical demands, the paper discusses the many potential directions of research the project offers, and the promises it brings. Then we discuss the technical challenges we found in the process of its implementation, choosing to generate the Chromossound as the output of a neural network, according to some technical choices and a defined architecture.

References

ABBADO, Adriano: Visual music Masters - abstract explorations: history and contemporary research. Milano: Skira, 2017.

ANTUNES, Jorge: A correspondência entre os sons e as cores. Brasilia: Thesaurus. 1982.

BASBAUM, Sérgio: Fundamentos da Cromossonia: sinestesia, arte e tecnologia. (Master's dissertation). Pontifícia Universidade Católica de São Paulo, 1999.

BASBAUM, Sérgio: Sinestesia, arte e tecnologia: fundamentos da cromossonia. São Paulo: Annablume/FAPESP, 2002.

BASBAUM, Sérgio: O primado da percepção e suas consequências no ambiente midiático. São Paulo: Intermeios/FAPESP, 2016.

BASTOS, Marcus; MORAN, Patricia: Audiovisual ao vivo: tendências e Conceitos. São Paulo, Intermeios, 2021.

BOSSEUR, Jean-Yes: Sound and the visual arts. Paris: Dix Voir, 1993.

CYTOWIC, Richard: Synesthesia: phenomenology and neuropsychology. in BARON-COHEN, Simon e HARRISON, John: Synesthesia: classic and contemporary readings. Oxford: Blackwell Publishers, 1997.

EISENSTEIN, Sergei: Film Form – essays in film theory. New York: Harvest Books, 1949.

FERREIRA MH, RENOVATO Tobo P, BARRICHELLO Cr, GUALTIERI M: Olfactory interference on the emotional processing speed of visual stimuli: The influence of facial expressions intensities. PLoS ONE 17(5): e0264261. https://doi.org/10.1371/journal.pone.0264261, 2022.

FULKERSON, Matthew: Rethinking the senses and their interactions: the case for sensory pluralism. Frontiers in Psychology | Consciousness Research - December 2014 | Volume 5 | Article 1426, 2014.

LUNDBORG, Goran (2014): The Hand and the Brain - From Lucy's Thumb to the Thought- Controlled Robotic Hand. London: Spinger-Verlag.

PEACOCK, Kenneth: Instruments to perform Color Music. LEONARDO, Vol. 21, No. 4, pp. 397-406, 1988.

MCDONNELL, Maura: Visual music – a composition of the ‘things themselves’. https://www.academia.edu/525221/Visual_Music_A_Composition_Of_The_Things_Themselves (undated).

MARKS, Lawrence E.: On Colored-Hearing Synesthesia: Cross-Modal Translations of Sensory Dimensions. Psychological Bulletin. VOL. 82, No. 3 MAY 1975.

MERLEAU-PONTY, Maurice: Phenomenology of Perception. New York: Routledge, 2012.

POUND, Ezra: ABC or Reading. London: Faber and Faber, 1961.

RAMACHANDRAN, V. S.; HUBBARD, Edward: Synaesthesia – a Window to Perception Thought and Language. Journal of Consciousness Studies, v. 8, n. 12, p. 3-34, 2001.

RUTHROF, Horst: Semantics and the Body - Meaning from Frege to the Post Modern. Toronto: Toronto University Press, 1997.

TOZZO DE DEUS, Alexandre; NESPOLI, Eduardo: Luteria digital experimental: singularização de instrumentos musicais em contexto pós-industrial. Revista Tecnologia e Sociedade. V. 16, No 45, 2020.

WHITNEY, John: To paint on water: the audiovisual duet of complementarity. Computer Music Journal 18, num 3, Cambridge: MIT Press, 1994.

WHITNEY, John: Digital Harmony. Peterborough: Byte Books, 1980.

Downloads

Published

2023-12-20

Issue

Vol. 19 No. 2 (2023)

Section

Articles

License

Copyright (c) 2023 Sergio Basbaum

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Authors who have their works published in Texto Digital agree that:

  1. Copyrights remain with the authors, who grant the journal the right of first publishing their submitted manuscripts. All materials published by the journalare under an Attribution 4.0 International - Creative Commons License, which allows them to be shared since authorship and first publication credits are mentioned.

  2. The Attribution 4.0 International - Creative Commons allows the copy and redistribution of the material in any medium or format, as well as its adaptation for any purpose, even commercially.

  3. Authors can take additional contracts for non-exclusive distribution of the version of their works published by our journal separately (e.g. to publish it in an institutional repository or as a book chapter) with both expressed authorship acknowledgment and Texto Digital’s first publication credit.