Reproducibility in e-Science: An overview of the related concepts and the most cited support tools

Authors

DOI:

https://doi.org/10.5007/1518-2924.2025.e103385

Keywords:

Research reproducibility, Data reuse, Research data management

Abstract

Objective: To present concepts related to research reproducibility, such as repeatability and replicability, as well as reuse, and explore how the most cited research support tools in the literature can be used to promote the reproducibility of research in e-Science.

Methods: A bibliographic search was conducted in the Web of Science, Scopus, and Google Scholar databases, along with a document analysis of the manuals of the investigated tools.

Results: Concepts of reproducibility, repeatability, and replicability were presented, with a discussion of their origins, and an exploration of how reuse differs from data use. The literature review also revealed tools that support the development of research in e-Science and promote the reproducibility of scientific research and data reuse, with the four most cited tools were, in order: the source code versioning tool Git Hub, the container tool Docker, the research project management tool Open Science Framework (OSF), and the Jupyter Notebook tool for interactive data processing program descriptions.

Conclusions: Science must go beyond sharing its results through literature; it is necessary to share the artifacts used in research to achieve reproducibility and reuse, concepts that require clear and consensual definitions. These artifacts must be shared with complete and error-free documentation, which is essential for understanding their meaning. For e-Science fields that require computational infrastructure to process large amounts of data, tools are available to facilitate this documentation. The current research context requires that information management teams and data repository managers understand such concepts and tools used by researchers.

Downloads

Download data is not yet available.

Author Biographies

Manuela Klanovicz Ferreira, Federal University of Rio Grande do Sul

She holds a PhD in Communication in the area of Scientific Communication, a bachelor's degree in Computer Science (2008) and a master's degree in Computing (2012), all from the Federal University of Rio Grande do Sul. Since 2009, she has been a system analyst at the Federal University of Rio Grande do Sul, and is a member of the team responsible for Lume - UFRGS's Digital Repository, which uses the open source software DSpace, and SABi - Library Automation System, which uses the proprietary software Aleph. She has taught courses on installing and customizing the DSpace tool and internal UFRGS courses on using the Redmine tool and software versioning. He has knowledge of LINUX, C, C++, C#, Java, XSLT, SQL, PHP, Javascript, Angular, CSS, SVN, Git, DSpace, Dataverse, AWS, Jupyter and Docker containers, with an interest in the following areas: research data repositories, digital repositories, scientific communication, virtualization and containers, research reproducibility, parallel and distributed programming and problem complexity (study of better algorithms for solving problems).

Samile Andrea de Souza Vanz, Federal University of Rio Grande do Sul

Associate professor at the Department of Information Sciences, the Postgraduate Program in Communication (PPGCOM UFRGS) and the Postgraduate Program in Information Science at the Federal University of Rio Grande do Sul (PPGCIN UFRGS). Graduated in Library Science from the Federal University of Rio Grande do Sul (1999), master and doctorate in Communication and Information from PPGCOM UFRGS (2004 and 2009), with a sandwich internship at Dalian University of Technology (China, 2007-2008). Post-doctorate from Universidad Carlos III de Madrid (Spain, 2016). Among the activities developed in university management, she was coordinator of the Library Science course at UFRGS (2012-2014), head of the Department of Information Sciences at UFRGS (2019-2020), deputy head of the Department (2021-2022), deputy coordinator of the PPGCOM UFRGS (2021-2022), member of the UFRGS Postgraduate Chamber (2022 - ). She was editor-in-chief of the magazine Em Questão (2014 - 2023) and currently holds the position of Coordinator of the Advisory Committee to Support the Editing of Scientific Periodicals at UFRGS (2023 - ). He is deputy leader of the Scientific Communication Research Group at UFRGS (2013 - ). Organized the Brazilian Meeting of Bibiometrics and Scientometrics (2012) and ENANCIB (2022). Develops research in the area of ​​Scientific Communication, with an emphasis on the production of scientific indicators, bibliometrics, scientific collaboration, citation analysis, co-citation analysis and university rankings. He has academic and professional experience in the area of ​​library planning, management and architecture.

References

ACM. Artifact Review and Badging. [S. l.: s. n.], 2020. Disponível em: https://www.acm.org/publications/policies/artifact-review-and-badging-current. Acesso em: 18 out. 2023.

BAKER, Monya. 1,500 scientists lift the lid on reproducibility. Nature, [s. l.], v. 533, p. 452-454, maio 2016. Disponível em: https://doi.org/10.1038/533452a. Acesso em: 29 abr. 2022. DOI: https://doi.org/10.1038/533452a

BARBA, Lorena A. Terminologies for reproducible research. arXiv. [S. l.: s. n.], 2018. arXiv:1802.03311. Disponível em: https://doi.org/10.48550/arXiv.1802.03311. Acesso em: 28 set. 2023.

BOETTIGER, Carl. An introduction to Docker for reproducible research. ACM SIGOPS Operating Systems Review, [s. l.], v. 49, n. 1, p. 71-79, Jan. 2015. Disponível em: http://doi.acm.org/10.1145/2723872.2723882. Acesso em: 21 fev. 2022. DOI: https://doi.org/10.1145/2723872.2723882

BRINCKMAN, Adam; CHARD, Kyle; GAFFNEY, Niall; HATEGAN, Mihael; JONES, Matthew B.; KOWALIK, Kacper; KULASEKARAN, Sivakumar; LUDÄSCHER, Bertram; MECUM, Bryce D.; NABRZYSKI, Jarek; STODDEN, Victoria; TAYLOR, Ian J.; TURK, Matthew J.; TURNER, Kandace. Computing environments for reproducibility: capturing the “Whole Tale”. Future Generation Computer Systems, [s. l.], v. 94, p. 854-867, 2019. Disponível em: https://doi.org/10.1016/j.future.2017.12.029. Acesso em: 13 jan. 2023. DOI: https://doi.org/10.1016/j.future.2017.12.029

CAREGNATO, Sônia Elisa; ROCHA, Rafael Port da; GABRIEL JUNIOR, Rene Faustino. Reúso de dados: princípios FAIR e o ecossistema de pesquisa. In: SALES, Luana Farias; VEIGA, Viviane dos Santos; HENNING, Patrícia; SAYÃO, Luís Fernando (org.). Princípios FAIR aplicados à gestão de dados de pesquisa. Rio de Janeiro: Ibict, 2021. p. 187-200. Disponível em: https://ridi.ibict.br/bitstream/123456789/1182/2/IBICT_Principios%20FAIR%20aplicados%20a%20gest%C3%A3o%20de%20dados%20de%20pesquisa_2021.pdf. Acesso em: 15 abr. 2023.

CHEIFET, Barbara. Promoting reproducibility with code ocean. Genome Biology, [s. l.], v. 22, n. 1, p. 1-2, 2021. Disponível em: https://doi.org/10.1186/s13059-021-02299-x. Acesso em: 13 jan. 2023. DOI: https://doi.org/10.1186/s13059-021-02299-x

CHEN, Xiaoli; DALLMEIER-TIESSEN, Sünje; DASLER, Robin; FEGER, Sebastian; FOKIANOS, Pamfilos; GONZALEZ, Jose Benito; HIRVONSALO, Harri; KOUSIDIS, Dinos; LAVASA, Artemis; MELE, Salvatore; RODRIGUEZ, Diego Rodriguez; ŠIMKO, Tibor; SMITH, Tim; TRISOVIC, Ana; TRZCINSKA, Anna; TSANAKTSIDIS, Ioannis; ZIMMERMANN, Markus; CRANMER, Kyle; HEINRICH, Lukas; WATTS, Gordon; HILDRETH, Michael; IGLESIAS, Lara Lloret; LASSILA-PERINI, Kati; NEUBERT, Sebastian. Open is not enough. Nature Physics, [s. l.], v. 15, p. 113-119, Feb. 2019. Disponível em: https://doi.org/10.1038/s41567-018-0342-2. Acesso em: 24 maio 2022. DOI: https://doi.org/10.1038/s41567-018-0342-2

CITO, Jürgen; GALL, Harald C. Using Docker Containers to improve reproducibility in software engineering research. In: IEEE/ACM INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING COMPANION (ICSE-C), 38., 2016, Austin, TX, USA. Proceedings […]. Austin: IEEE, 2016. p. 906-907. Disponível em: http://dx.doi.org/10.1145/2889160.2891057. Acesso em: 12 maio 2025. DOI: https://doi.org/10.1145/2889160.2891057

CLAERBOUT, Jon F.; KARRENBACH, Martin. Electronic documents give reproducible research a new meaning. In: SEG TECHNICAL PROGRAM EXPANDED, 1992, [s. l.]. Abstracts […]. [S. l.]: Society of Exploration Geophysicists, 1992. p. 601-604. Disponível em: https://doi.org/10.1190/1.1822162. Acesso em: 28 nov. 2023. DOI: https://doi.org/10.1190/1.1822162

FASEB. Enhancing research reproducibility. Bethesda: FASEB, 2016. Disponível em: https://www.faseb.org/FASEB/media/PDF/News/Washington%20Update/FASEB_Enhancing-Research-Reproducibility_1.pdf. Acesso em: 23 nov. 2023.

FEGER, Sebastian Stefan; WOŹNIAK, Paweł W.; NIESS, Jasmin; SCHMIDT, Albrecht. Tailored science badges: enabling new forms of research interaction. In: ACM DESIGNING INTERACTIVE SYSTEMS CONFERENCE, 21., 2021, New York. Proceedings […]. New York: Association for Computing Machinery, 2021. p. 576-588. Disponível em: https://doi.org/10.1145/3461778.3462067. Acesso em: 03 jan. 2022. DOI: https://doi.org/10.1145/3461778.3462067

FIGUEIREDO FILHO, Dalson; LINS, Rodrigo; DOMINGOS, Amanda; JANZ, Nicole; SILVA, Lucas. Seven reasons why: a user’s guide to transparency and reproducibility. Brazilian Political Science Review, [s. n.], v. 13, n. 2, e0001, 2019. Disponível em: https://doi.org/10.1590/1981-3821201900020001. Acesso em: 20 nov. 2023. DOI: https://doi.org/10.1590/1981-3821201900020001

GOODMAN, Steven N.; FANELLI, Daniele; IOANNIDIS, John P. A. What does research reproducibility mean? Science translational medicine, [s. n.], v. 8, n. 341, p. 341ps12-341ps12, 2016. Disponível em: https://doi.org/10.1126/scitranslmed.aaf5027. Acesso em: 12 set. 2023. DOI: https://doi.org/10.1126/scitranslmed.aaf5027

HARZING, Anne-Wil. Publish or perish. Versão 8.8.4384, 2021. Software. Disponível em: https://harzing.com/resources/publish-or-perish/command-line. Acesso em: 23 fev. 2023.

HERNÁNDEZ, José Armando; COLOM, Miguel. Repeatability, Reproducibility, replicability, reusability (4R) in Journals' Policies and Software/Data Management in Scientific Publications: a survey, discussion, and perspectives. arXiv. [S. l.: s. n.], 2023. arXiv:2312.11028. Disponível em: https://doi.org/10.48550/arXiv.2312.11028. Acesso em: 13 mar. 2024.

MALIK, Tanu. Reproducible eScience: the data containerization challenge. In: IEEE INTERNATIONAL CONFERENCE ON E-SCIENCE (E-SCIENCE), 19., 2023, Limassol. Proceeedings [...]. Limassol: IEEE, 2023. p. 1-5. Disponível em: https://doi.org/10.1109/e-Science58273.2023.10254837. Acesso em: 9 out. 2023. DOI: https://doi.org/10.1109/e-Science58273.2023.10254837

MOREAU, David; WIEBELS, Kristina; BOETTIGER, Carl. Containers for computational reproducibility. Nature Reviews Methods Primers, [s. n.], v. 3, n. 1, p. 1-16, 2023. Disponível em: https://doi.org/10.1038/s43586-023-00236-9. Acesso em: 9 set. 2023. DOI: https://doi.org/10.1038/s43586-023-00236-9

MOUAT, Adrian. Using docker: developing and deploying software with containers. Boston: O’Reilly, 2016. Disponível em: https://books.google.com.br/books?hl=pt-BR&lr=&id=wpYpCwAAQBAJ&oi=fnd&pg=PP1&dq=container+history+docker&ots=QhL5yLnT9Q&sig=7PVD3sSBTX9ab0gIq5lrMIqkVps#v=onepage&q&f=false. Acesso em: 20 nov. 2022.

MUNAFÒ, Marcus R.; NOSEK, Brian A.; BISHOP, Dorothy V. M.; BUTTON, Katherine S.; CHAMBERS, Christopher D.; PERCIE DU SERT, Nathalie; SIMONSOHN, Uri; WAGENMAKERS, Eric-Jan; WARE, Jennifer J.; IOANNIDIS, John P. A. A manifesto for reproducible science. Nature human behaviour, [s. n.], v. 1, n. 1, p. 1-9, 2017. Disponível em: http://dx.doi.org/10.1038/s41562-016-0021. Acesso em: 29 fev. 2024. DOI: https://doi.org/10.1038/s41562-016-0021

NATIONAL ACADEMIES OF SCIENCES, ENGINEERING, AND MEDICINE. Reproducibility and replicability in science. Washington: The National Academies Press, 2019. Disponível em: https://doi.org/10.17226/25303. Acesso em: 23 nov. 2023. DOI: https://doi.org/10.17226/25303

PASQUETTO, Irene V.; RANDLES, Bernadette M.; BORGMAN, Christine L. On the reuse of scientific data. Data Science Journal, [s. n.], v. 16, n. 8, p. 1-9, 2017. Disponível em: https://doi.org/10.5334/dsj-2017-008. Acesso em: 15 abr. 2023. DOI: https://doi.org/10.5334/dsj-2017-008

PENG, Roger D.; DOMINICI, Francesca; ZEGER, Scott L. Reproducible epidemiologic research. American Journal of Epidemiology, [s. n.], v. 163, n. 9, p. 783-789, 2006. Disponível em: https://doi.org/10.1093/aje/kwj093. Acesso em: 20 nov. 2023. DOI: https://doi.org/10.1093/aje/kwj093

PENG, Roger D.; HICKS, Stephanie. C. Reproducible research: a retrospective. Annual Review of Public Health, [s. n.], v. 42. p. 79-93, 2021. Disponível em: https://doi.org/10.1146/annurev-publhealth-012420-105110. Acesso em: 24 abr. 2022. DOI: https://doi.org/10.1146/annurev-publhealth-012420-105110

PEREZ-RIVEROL, Yasset; GATTO, Laurent; WANG, Rui; SACHSENBERG, Timo; USZKOREIT, Julian; LEPREVOST, Felipe da Veiga; FUFEZAN, Christian; TERNENT, Tobias; EGLEN, Stephen J.; KATZ, Daniel S.; POLLARD, Tom J.; KONOVALOV, Alexander; FLIGHT, Robert M.; BLIN, Kai; VIZCAÍNO, Juan Antonio. Ten simple rules for taking advantage of Git and GitHub. PLoS Computational Biology, [s. n.], v. 12, n. 7, e1004947, 2016. Disponível em: https://doi.org/10.1371/journal.pcbi.1007142. Acesso em: 15 nov. 2023. DOI: https://doi.org/10.1371/journal.pcbi.1004947

POTTERBUSCH, Megan; LOTRECCHIANO, G. R. Shifting paradigms in information flow: an open science framework (OSF) for knowledge sharing teams. Informing Science, [s. n.], v. 21, p. 179, 2018. Disponível em: https://doi.org/10.28945/4031. Acesso em: 14 jan. 2023. DOI: https://doi.org/10.28945/4031

ROUGIER, Nicolas P.; HINSEN, Konrad; ALEXANDRE, Frédéric; ARILDSEN, Thomas; BARBA, Lorena A.; BENUREAU, Fabien C.Y.; BROWN, C. Titus; DE BUYL, Pierre; CAGLAYAN, Ozan; DAVISON, Andrew P.; DELSUC, Marc-André; DETORAKIS, Georgios; DIEM, Alexandra K.; DRIX, Damien; ENEL, Pierre; GIRARD, Benoît; GUEST, Olivia; HALL, Matt G.; HENRIQUES, Rafael N.; HINAUT, Xavier; JARON, Kamil S.; KHAMASSI, Mehdi; KLEIN, Almar; MANNINEN, Tiina; MARCHESI, Pietro; MCGLINN, Daniel; METZNER, Christoph; PETCHEY, Owen; PLESSER, Hans Ekkehard; POISOT, Timothée; RAM, Karthik; RAM, Yoav; ROESCH, Etienne; ROSSANT, Cyrille; ROSTAMI, Vahid; SHIFMAN, Aaron; STACHELEK, Jemma; STIMBERG, Marcel; STOLLMEIER, Frank; VAGGI, Federico; VIEJO, Guillaume; VITAY, Julien; VOSTINAR, Anya E.; YURCHAK, Roman; ZITO, Tiziano. Sustainable computational science: the ReScience initiative. PeerJ Computer Science, [s. n.], v. 3, e142, 2017. Disponível em: https://doi.org/10.7717/peerj-cs.142. Acesso em 23 nov. 2023. DOI: https://doi.org/10.7717/peerj-cs.142

SAYÃO, Luis F.; SALES, Luana F. Dados abertos de pesquisa: ampliando o conceito de acesso livre. RECIIS, Rio de Janeiro, v. 8, n. 2, p. 76-92, jun. 2014. Disponível em: https://www.arca.fiocruz.br/handle/icict/17102. Acesso em: 21 fev. 2022.

SCHAEFER, B.; CAMPOS, L. A.; CANDIDO, M. R. Revista DADOS cria editoria especializada em replicabilidade. In: SCIELO. SciELO em Perspectiva, 20 out. 2023. Disponível em: https://blog.scielo.org/blog/2023/10/20/revista-dados-cria-editoria-especializada-em-replicabilidade/. Acesso em: 05 fev. 2025.

TRISOVIC, Ana; DURBIN, Philip; SCHLATTER, Tania; DURAND, Gustavo; BARBOSA, Sonia; BROOKE, Danny; CROSAS, Mercê. Advancing computational reproducibility in the dataverse data repository platform. In: INTERNATIONAL WORKSHOP ON PRACTICAL REPRODUCIBLE EVALUATION OF COMPUTER SYSTEMS, 3., 2020, Stockholm. Proceedings […]. New York: Association for Computing Machinery, 2020. p. 15–20. Disponível em: https://doi.org/10.1145/3391800.3398173. Acesso em: 8 jun. 2022. DOI: https://doi.org/10.1145/3391800.3398173

UNESCO. Recommendation on open science. [S.l.: s.n.], 2021. Disponível em: https://unesdoc.unesco.org/ark:/48223/pf0000379949.locale=en. Acesso em: 27 maio 2024.

VAN DE SANDT, Stephanie; DALLMEIER-TIESSEN, Sünje; LAVASA, Artemis; PETRAS, Vivien. The definition of reuse. Data Science Journal, [s. l.], v. 18, n. 1, p. 1–19, 2019. Disponível em: http://dx.doi.org/10.5334/dsj-2019-022. Acesso em: 28 nov. 2022. DOI: https://doi.org/10.5334/dsj-2019-022

VANZ, Samile Andrea de Souza; PAVÃO, Caterina Marta Groposo; CAREGNATO, Sônia Elisa; PASSOS, Paula Caroline Schifino Jardim; MOURA, Ana Maria Mielniczuk de; BORGES, Eduardo Nunes; GABRIEL JUNIOR, Rene Faustino; ROCHA, Rafael Port da. Diretrizes para o estabelecimento de um checklist para curadoria de dados de pesquisa. Informação em Pauta, [s. l.], v. 6, n. 00, p. 1-18, 26 out. 2021. Disponível em: http://www.periodicos.ufc.br/informacaoempauta/article/view/68088/197501. Acesso em: 22 out. 2022

WILKINSON, Mark D.; DUMONTIER, Michel; AALBERSBERG, IJsbrand Jan; APPLETON, Gabrielle; AXTON, Myles; BAAK, Arie; BLOMBERG, Niklas; BOITEN, Jan-Willem; BONINO DA SILVA SANTOS, Luiz; BOURNE, Philip E.; BOUWMAN, Jildau; BROOKES, Anthony J.; CLARK, Tim; CROSAS, Mercè; DILLO, Ingrid; DUMON, Olivier; EDMUNDS, Scott; EVELO, Chris T.; FINKERS, Richard; GONZALEZ-BELTRAN, Alejandra; GRAY, Alasdair J.G.; GROTH, Paul; GOBLE, Carole; GRETHE, Jeffrey S.; HERINGA, Jaap; ’T HOEN, Peter A.C.; HOOFT, Rob; KUHN, Tobias; KOK, Ruben; KOK, Joost; LUSHER, Scott J.; MARTONE, Maryann E.; MONS, Albert; PACKER, Abel L.; PERSSON, Bengt; ROCCA-SERRA, Philippe; ROOS, Marco; VAN SCHAIK, Rene; SANSONE, Susanna-Assunta; SCHULTES, Erik; SENGSTAG, Thierry; SLATER, Ted; STRAWN, George; SWERTZ, Morris A.; THOMPSON, Mark; VAN DER LEI, Johan; VAN MULLIGEN, Erik; VELTEROP, Jan; WAAGMEESTER, Andra; WITTENBURG, Peter; WOLSTENCROFT, Katherine; ZHAO, Jun; MONS, Barend. Comments: The FAIR Guiding Principles for scientific data management and stewardshi. Scientific Data, [s. l.], v. 3, n. 160018, Mar. 2016. Disponível em: https://doi.org/10.1038/sdata.2016.18. Acesso em: 21 fev. 2022. DOI: https://doi.org/10.1038/sdata.2016.18

A capa da Revista ENCONTROS BIBLI apresenta como título do volume 30 do ano 2025 "Por uma ciência diversa, equitativa, inclusiva e acessível." Visualmente, a imagem destaca várias silhuetas de perfis de rostos humanos, representando diversidade, com diferentes tons de pele. À direita, há uma representação do globo terrestre. Na parte inferior da capa, são exibidas informações institucionais: "Departamento de Ciência da Informação, Programa de Pós-graduação em Ciência da Informação, Portal de Periódicos UFSC." A publicação é datada de 2025, Volume 30, ISSN 1518-2924.

Downloads

Published

2025-06-16

How to Cite

FERREIRA, Manuela Klanovicz; VANZ, Samile Andrea de Souza. Reproducibility in e-Science: An overview of the related concepts and the most cited support tools. Encontros Bibli: electronic journal of library science, archival science and information science, Florianópolis/SC, Brasil, v. 30, p. 1–25, 2025. DOI: 10.5007/1518-2924.2025.e103385. Disponível em: https://periodicos.ufsc.br/index.php/eb/article/view/103385. Acesso em: 3 feb. 2026.

Issue

Vol. 30 (2025): For diverse, equitable, inclusive, and accessible science

Section

Articles

License

Copyright (c) 2025 Manuela Klanovicz Ferreira, Samile Andrea de Souza Vanz

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

The author must guarantee that:

  • there is full consensus among all the coauthors in approving the final version of the document and its submission for publication.
  • the work is original, and when the work and/or words from other people were used, they were properly acknowledged.

Plagiarism in all of its forms constitutes an unethical publication behavior and is unacceptable. Encontros Bibli has the right to use software or any other method of plagiarism detection.

All manuscripts submitted to Encontros Bibli go through plagiarism and self-plagiarism identification. Plagiarism identified during the evaluation process will result in the filing of the submission. In case plagiarism is identified in a manuscript published in the journal, the Editor-in-Chief will conduct a preliminary investigation and, if necessary, will make a retraction.

This journal, following the recommendations of the Open Source movement, provides full open access to its content. By doing this, the authors keep all of their rights allowing Encontros Bibli to publish and make its articles available to the whole community.

 

Encontros Bibli content is licensed under a Creative Commons Attribution 4.0 International License.

Any user has the right to:

  • Share - copy, download, print or redistribute the material in any medium or format.
  • Adapt - remix, transform and build upon the material for any purpose, even commercially.

According to the following terms:

  • Attribution - You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
  • No additional restrictions - You may not apply legal terms or technological measures that legally restrict others from doing anything that the license permits.
Crossref
Scopus
Google Scholar
Europe PMC