Python scripts for web scraping metadata from descriptions of the international scenario of research data repositories

Authors

DOI:

https://doi.org/10.5007/1518-2924.2023.e94877

Keywords:

Data Repository, Research Data, Geosciences, Re3data

Abstract

Objective: Research data repositories are an evolution of document repositories that aim to access and preserve all materials used before, during, and after scientific research. In this context, this study aims to conduct an exploratory and descriptive investigation of the international scenario of data repositories by monitoring the descriptive metadata of the international register of this type of repositories in the Registry of Research Data Repositories (re3data.org).

Methods: The process requires applying knowledge inherent to the techniques and technologies used for descriptive data analysis, information retrieval, manipulation, analysis, and data visualization. Consequently, three scripts in Python 3.11 are provided for collecting metadata from re3data and scripts and converting the metadata to enable visualization in software such as VOSviewer, a dataset with metadata descriptions of repositories and conversions for visualization of networks. The datasets produced in this study can be found in the ZENODO Data Repository (https://doi.org/10.5281/zenodo.7903109). In a collection on (05/05/2023), 3108 links to the repository descriptions were retrieved. Data and scripts were created for this methodological experiment and shared at (DOI: doi.org/10.5281/zenodo.7903109). The dataset contains a root directory with three subdirectories: (scripts) with (.py) Python codes, another directory called (data) with textual files containing tab-separated values (.TSV), and the file (Information Systems Research, RIS). The third directory (env) contains the Python libraries required to run the scripts. 

Potential for reuse: The research method applied to manipulate this dataset is based on automated re3data metadata extraction and network visualization; after the data collection and analysis process, it is possible to trigger a study based on the descriptions extracted from the Registry of Research Data Repositories (re3data), researchers can visualize the international scenario of research data repositories, verified by re3data, which allows ethical monitoring of the number of research data repositories that are registered in re3data, what are their areas, institutions, countries, the language of research data, the typology of repositories and deposited data, their themes, areas of knowledge, types of access, licenses and software used. In addition, other issues can be raised while interpreting the data. The community of Librarianship and Information Science professionals need to share data and the extraction technique  these research data. Finally, it can be concluded whether information about research data repositories allows us to state that they are heterogeneous data sources that enable access and preservation of a wide range of research data types

Downloads

Download data is not yet available.

Author Biographies

Alexandre Ribas Semeler, Federal University of Rio Grande do Sul

Alexandre Ribas Semeler is a Data Librarian at the Institute of Geosciences of the Federal University of Rio Grande do Sul (UFRGS), Porto Alegre, Brazil. He is a graduate in Librarianship from UFRGS (2003-2007), and holds a Master in Communication and Information from UFRGS (2008-2010). He is a Doctor in Information Science at the Federal University of Santa Catarina (UFSC) (2015-2017). He is a (2018-2019) postdoctor in Information Science at the Federal University of Santa Catarina, Florianópolis, Brazil. His areas of research interest include: interdisciplinary topics in information science, e-science, data science, and data librarianship.

Arthur Longoni Oliveira, Universidade Federal do Rio Grande do Sul

Graduando de Ciência da Computação no Instituto de Informática da Universidade Federal Do Rio Grande Do Sul (2013-atual). Cientista de Dados Junior, Desenvolvedor Python Senior, Atuando nas seguintes áreas: Ciência da Informação, Programação Orientada a Objetos, Computação Paralela Aplicada à Coleta, Análise e Visualização de Dados Científicos.

Fabiana Andrade Pereira, São Paulo Research Foundation

Sou mestra em Ciência da Informação pelo Programa de Pós-Graduação em Ciência da Informação - PPGCI - ECA - Universidade de São Paulo (2020), onde pesquisei sobre os impactos dos processos avaliativos gerados por mecanismos de buscas acadêmicos, nas áreas de Bibliometria e Cientometria. Fiz uma pós-graduação - especialização em Gestão da Comunicação em Mídias Digitais pelo SENAC-SP (2011). Possuo graduação em Biblioteconomia pela Fundação Escola de Sociologia e Política de São Paulo (2009). Tenho experiência profissional nas áreas de Biblioteconomia e Ciência da Informação, com ênfase em Bibliotecas Virtuais, Controle de Vocabulários e Buscadores. Esforço-me nos diálogos entre a Ciência da Informação e Informação Digital, onde atuo, estudo e algumas vezes ministro aulas em Search Engine Optimization (SEO), Big Data, algoritmos e dataficação, mídias sociais/digitais e novas tecnologias. Também possuo diversos cursos de extensão em áreas correlatas da comunicação e da informação digital.

Policarpo Matiquite, Universidade Eduardo Mondlane em Moçambique

Possui doutorado em Ciências de Informação pela UFSC Universidade Federal de Santa Catariana (2018) com Tese apresentada ao Programa de Pós-Graduação em Ciência da Informação - área de concentração Gestão da Informação, linha de pesquisa Organização, Representação e Mediação da Informação e do Conhecimento - no Centro de Ciências da Educação da Universidade Federal de Santa Catarina (2018). Possui também mestrado em Library and information science - University of the Western Cape (2011). Tem experiência em docência nas áreas de Biblioteconomia, Comunicação e Gestão de conhecimento. Possui experiência em gestão de projectos com enfoque para projectos para biblioteca universitárias. Participou da construção da Biblioteca Central da Universidade Eduardo Mondlane em Moçambique, Actualmente é Professor de varias disciplinas na Escola de Comunicação e Artes da Universidade Eduardo Mondlane.

References

RE3DATA. Disponível em: <http://www.re3data.org/about>. Acesso em: maio 2023.

Revista Encontros Bibli, Volume 28, ano 2023 com o tema "Inovação, Tecnologia e Sustentabilidade"

Downloads

Published

2023-08-04

How to Cite

SEMELER, Alexandre Ribas; LONGONI OLIVEIRA, Arthur; ANDRADE PEREIRA, Fabiana; MATIQUITE, Policarpo. Python scripts for web scraping metadata from descriptions of the international scenario of research data repositories. Encontros Bibli: revista eletrônica de biblioteconomia e ciência da informação, [S. l.], v. 28, p. 1–8, 2023. DOI: 10.5007/1518-2924.2023.e94877. Disponível em: https://periodicos.ufsc.br/index.php/eb/article/view/94877. Acesso em: 15 jan. 2025.

Issue

Vol. 28 (2023): Innovation, Technology and Sustainability

Section

Data Articles

License

Copyright (c) 2023 Alexandre Ribas Semeler, Arthur Longoni Oliveira, Fabiana Andrade Pereira, Policarpo Matiquite

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

The author must guarantee that:

  • there is full consensus among all the coauthors in approving the final version of the document and its submission for publication.
  • the work is original, and when the work and/or words from other people were used, they were properly acknowledged.

Plagiarism in all of its forms constitutes an unethical publication behavior and is unacceptable. Encontros Bibli has the right to use software or any other method of plagiarism detection.

All manuscripts submitted to Encontros Bibli go through plagiarism and self-plagiarism identification. Plagiarism identified during the evaluation process will result in the filing of the submission. In case plagiarism is identified in a manuscript published in the journal, the Editor-in-Chief will conduct a preliminary investigation and, if necessary, will make a retraction.

This journal, following the recommendations of the Open Source movement, provides full open access to its content. By doing this, the authors keep all of their rights allowing Encontros Bibli to publish and make its articles available to the whole community.

 

Encontros Bibli content is licensed under a Creative Commons Attribution 4.0 International License.

Any user has the right to:

  • Share - copy, download, print or redistribute the material in any medium or format.
  • Adapt - remix, transform and build upon the material for any purpose, even commercially.

According to the following terms:

  • Attribution - You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
  • No additional restrictions - You may not apply legal terms or technological measures that legally restrict others from doing anything that the license permits.
Crossref
Scopus
Google Scholar
Europe PMC