LIRA — Intermodal Language For Affective Recognition: a multimodal database for music emotion recognition

Authors

DOI:

https://doi.org/10.5007/1518-2924.2026.e107990

Keywords:

Database, Multimodal, Music emotion recognition, Music feature extraction, Information retrieval

Abstract

Objective: this paper presents LIRA — Intermodal Language for Affective Recognition, a multimodal dataset designed to advance research in music emotion recognition. It addresses limitations in existing databases by providing rich emotional annotations alongside diverse feature representations across five modalities.

Method: LIRA contains 1,412 thirty-second song segments, each labeled with one of four discrete emotions: joy, anger/fear, serenity, or sadness. The dataset encompasses five modalities: audio, chords, lyrics, symbolic features, and voice. Feature extraction was performed using tools including Librosa, Essentia, music21, and Spleeter.

Results: a total of 171 features were extracted: 67 from audio, 58 from voice, 25 from chords, 12 symbolic, and 9 from lyrics. Emotional and structural data are systematically organized in a reusable format. All data and processing scripts are publicly accessible via Mendeley Data and GitHub.

Conclusions: LIRA is a publicly available, multimodal, affectively annotated database that fosters robust and reproducible research in music emotion recognition. Its multimodality and standardized structure enable comprehensive exploration of emotional responses to music and support the development of more expressive computational models.

Downloads

Download data is not yet available.

Author Biographies

Paulo Sergio da Conceição Moreira, Federal University of Paraná

PhD in Information Management (2023) from the Graduate Program in Information Management at the Federal University of Paraná (PPGGI/UFPR). He holds a Master's (2019) and a Bachelor's (2017) degree in Information Management from the same institution. He conducts research in the areas of Data Analysis, Data Mining, Information Metrics, Emotion Recognition in Music, and Musical Information Retrieval. He is currently a professor and coordinator of the Software Engineering and Information Systems courses at UniSenai Paraná (São José dos Pinhais campus), as well as a permanent professor in the Graduate Program in Data Science at UFPR.

Denise Fukumi Tsunoda, Federal University of Paraná

Full Professor at the Federal University of Paraná, Department of Information Science and Management, working in the undergraduate program in Information Management and a permanent faculty member in the Graduate Program in Information Management (PPGGI) and the Professional Master's Program in Economics (PPGEcon). She holds a degree in Computer Science from the Federal University of Paraná (1992), a Master's degree in Electrical Engineering and Industrial Informatics from the Federal University of Technology of Paraná (1996), and a PhD in Electrical Engineering and Industrial Informatics - Biomedical Engineering from the Federal University of Technology of Paraná (2004). Her work focuses on artificial intelligence, machine learning, deep learning, pattern discovery in databases, data mining, process mining, text mining, evolutionary computation, genetic algorithms, genetic programming, and data analysis.

Marília Nunes-Silva, Federal University of Minas Gerais

She holds a PhD in Neuroscience from the Federal University of Minas Gerais (UFMG), a Master's in Developmental Psychology from UFMG, a Specialist in Art Therapy from Faculdade Vicentina (FAVI), and a Music Therapist from Censupeg. She holds a degree in Music (recorder) from the State University of Minas Gerais (UEMG) and a degree in Psychology from UFMG. She has experience in Psychology, with an emphasis on Neuropsychology, Psychology and Education, and Psychometry, and in Music, with an emphasis on Baroque music performance (recorder and singing). 

References

BIGAND, E. et al. Multidimensional scaling of emotional responses to music: the effect of musical expertise and of the duration of the excerpts. Cognition and Emotion, [s. l.], v. 19, n. 8, p. 1113-1139, 2005. Available at: https://doi.org/10.1080/02699930500204250. Accessed on: 11 July 2025. DOI: https://doi.org/10.1080/02699930500204250

CHATURVEDI, V. et al. Music mood and human emotion recognition based on physiological signals: a systematic review. Multimedia Systems, [s. l.], v. 28, n. 21, p. 21-44, apr. 2021. Available at: https://doi.org/10.1007/s00530-021-00786-6. Accessed on: 11 July 2025. DOI: https://doi.org/10.1007/s00530-021-00786-6

FAN, J. et al. Ranking-Based emotion recognition for experimental music. In: INTERNATIONAL SOCIETY FOR MUSIC INFORMATION RETRIEVAL CONFERENCE, 18., 2017, Suzhou. Proceedings [...]. Suzhou: ISMIR, 2017. p. 368-375. Available at: http://archives.ismir.net/ismir2017/paper/000217.pdf. Accessed on: 11 July 2025.

FLEISS, J. L. Measuring nominal scale agreement among many raters. Psychological Bulletin, [s. l.], v. 76, n. 5, p. 378-382, Nov. 1971. Available at: https://psycnet.apa.org/doi/10.1037/h0031619. Accessed on: 11 July 2025. DOI: https://doi.org/10.1037/h0031619

GÓMEZ-CAÑÓN, J. S. et al. Music emotion recognition: toward new, robust standards in personalized and context-sensitive applications. IEEE Signal Processing Magazine, [s. l.], v. 38, n. 6, p. 106-114, Oct. 2021. Available at: https://doi.org/10.1109/MSP.2021.3106232. Accessed on: 11 July 2025. DOI: https://doi.org/10.1109/MSP.2021.3106232

JUSLIN, P. N.; LAUKKA, P. Expression, perception, and induction of musical emotions: a review and a questionnaire study of everyday listening. Journal of New Music Research, [s. l.], v. 33, n. 3, p. 217-238, 2004. Available at: https://doi.org/10.1080/0929821042000317813. Accessed on: 11 July 2025. DOI: https://doi.org/10.1080/0929821042000317813

MCKAY, C. Automatic music classification with jMIR. 2010. 600 f. Tese (Doutorado) - Curso de Music Technology, Music, McGill University, Montreal, 2010. Available at: http://jmir.sourceforge.net/publications/PhD_Dissertation_2010.pdf. Accessed on: 27 July 2025.

MAUCH, M. Automatic chord transcription from audio using computational models of musical context. 2010. 168 f. Tese (Doutorado) - Curso de Electronic Engineering and Computer Science, University of London, Londres, 2010.

NUNES-SILVA, M. et al. Avaliação de músicas compostas para indução de relaxamento e de seus efeitos psicológicos. Psicologia: Ciência e Profissão, [s. l.], v. 36, n. 3, p. 709-725, July/Sept. 2016. Available at: https://doi.org/10.1590/1982-3703001672014. Accessed on: 11 July 2025. DOI: https://doi.org/10.1590/1982-3703001672014

PANDA, R. E. S. Emotion-based analysis and classification of audio music. 2019. 373 f. Tese (Doutorado) - Curso de Engenharia Informática, Universidade de Coimbra, Coimbra, 2019. Available at: https://hdl.handle.net/10316/87618. Accessed on: 04 Oct. 2025.

RAJESH, S.; NALINI, N. J. Musical instrument emotion recognition using deep recurrent neural network. Procedia Computer Science, [s. l.], v. 167, p. 16-25, 2020. Available at: https://doi.org/10.1016/j.procs.2020.03.178. Accessed on: 11 July 2025. DOI: https://doi.org/10.1016/j.procs.2020.03.178

RUSSO, M. et al. Cochleogram-based approach for detecting perceived emotions in music. Information Processing & Management, v. 57, n. 5, p. 1-17, Sept. 2020. Available at: https://doi.org/10.1016/j.ipm.2020.102270. Accessed on: 11 July 2025. DOI: https://doi.org/10.1016/j.ipm.2020.102270

YANG, D.; LEE, W.S. Music emotion identification from lyrics. In: IEEE INTERNATIONAL SYMPOSIUM ON MULTIMEDIA, 11., 2009, San Diego. Proceedings [...]. San Diego: IEEE, 2009. p. 624-629. Available at: https://doi.org/10.1109/ISM.2009.123. Accessed on: 11 July 2025. DOI: https://doi.org/10.1109/ISM.2009.123

YANG, X.; DONG, Y.; LI, J. Review of data features-based music emotion recognition methods. Multimedia Systems, [s. l.], v. 24, n. 4, p. 365-389, Aug. 2017. Available at: https://link.springer.com/article/10.1007/s00530-017-0559-4. Accessed on: 11 July 2025. DOI: https://doi.org/10.1007/s00530-017-0559-4

Capa do periódico Académico Encontros Bibli, da Universidade Federal de Santa Catarina (UFSC). Volume 31, ano 2026 Nela se mostram as mãos de uma pessoa com a pele morena segurando um punhado de terra escura, na qual um pequeno broto verde com duas folhas está plantado. As mãos estão centralizadas na imagem e são enquadradas por um fundo que mostra um solo seco e rachado, sugerindo um ambiente de seca ou desértico. No topo da imagem, o título da publicação, ENCONTROS BIBLI, está em letras maiúsculas brancas. Abaixo, há uma citação entre aspas: "Ética e integridade informacional: a responsabilidade científica e socioambiental da Ciência da Informação diante da urgência climática". Na parte inferior da capa, há informações institucionais: No canto esquerdo, o logo da UFSC. Ao lado, o texto "Departamento de Ciência da Informação", "Programa de Pós-graduação em Ciência da Informação" e "Portal de Periódicos UFSC". Abaixo, a data "2026", o volume "V. 31" e o ISSN 1518-2924. No canto inferior direito, o logo da publicação, que consiste em um escudo roxo com as letras "EB" estilizadas em laranja. A capa utiliza as cores roxo, branco e laranja, com a maior parte da imagem ocupada pelo cenário do solo e as mãos. A arte visual e o tema do título da publicação sugerem um foco na interseção entre a ciência, a informação e as questões ambientais, como a crise climática.

Downloads

Additional Files

Published

2025-10-29

How to Cite

MOREIRA, Paulo Sergio da Conceição; TSUNODA, Denise Fukumi; NUNES-SILVA, Marília. LIRA — Intermodal Language For Affective Recognition: a multimodal database for music emotion recognition. Encontros Bibli: electronic journal of library science, archival science and information science, Florianópolis/SC, Brasil, v. 31, p. 1–16, 2025. DOI: 10.5007/1518-2924.2026.e107990. Disponível em: https://periodicos.ufsc.br/index.php/eb/article/view/107990. Acesso em: 27 dec. 2025.

Issue

Vol. 31 (2026): Ethics and informational integrity in the climate emergency: the socio-environmental responsibility of Information Science.

Section

Data Articles

License

Copyright (c) 2025 Paulo Sergio da Conceição Moreira, Denise Fukumi Tsunoda, Marília Nunes-Silva

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

The author must guarantee that:

  • there is full consensus among all the coauthors in approving the final version of the document and its submission for publication.
  • the work is original, and when the work and/or words from other people were used, they were properly acknowledged.

Plagiarism in all of its forms constitutes an unethical publication behavior and is unacceptable. Encontros Bibli has the right to use software or any other method of plagiarism detection.

All manuscripts submitted to Encontros Bibli go through plagiarism and self-plagiarism identification. Plagiarism identified during the evaluation process will result in the filing of the submission. In case plagiarism is identified in a manuscript published in the journal, the Editor-in-Chief will conduct a preliminary investigation and, if necessary, will make a retraction.

This journal, following the recommendations of the Open Source movement, provides full open access to its content. By doing this, the authors keep all of their rights allowing Encontros Bibli to publish and make its articles available to the whole community.

 

Encontros Bibli content is licensed under a Creative Commons Attribution 4.0 International License.

Any user has the right to:

  • Share - copy, download, print or redistribute the material in any medium or format.
  • Adapt - remix, transform and build upon the material for any purpose, even commercially.

According to the following terms:

  • Attribution - You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
  • No additional restrictions - You may not apply legal terms or technological measures that legally restrict others from doing anything that the license permits.
Crossref
Scopus
Google Scholar
Europe PMC