Replicable prompt engineering model for scientific literature retrieval with ChatGPT

Authors

DOI:

https://doi.org/10.5007/1518-2924.2026.e109140

Keywords:

Generative artificial intelligence, prompt engineering, information retrieval, university libraries, open science

Abstract

The expansion of generative artificial intelligence has transformed scientific literature retrieval processes, creating opportunities for greater efficiency and accessibility while raising concerns regarding verifiability, rigor, and the generation of fabricated references in academic contexts. In Library and Information Science, a methodological gap persists concerning replicable protocols capable of evaluating the operational performance of these tools in bibliographic searches.

Objective: This study aimed to assess the efficacy of ChatGPT, GPT-5.2 model, in the ChatGPT Plus environment, through a structured prompt engineering protocol applied to nine controlled searches conducted in February 2026 across three geographic levels: global, Latin American, and Chilean.

Method: An exploratory–descriptive documentary design was adopted, predominantly quantitative with a complementary interpretative component, operationalizing dichotomous indicators of precision, traceability, conceptual coherence, and temporal efficiency, alongside manual validation of DOI or URL.

Results: The final corpus consolidated 130 unique verified references, with no hallucinated records after protocol implementation and with 100 percent conceptual coherence and traceability across all zones. Average response times ranged from 14 to 22 minutes per search.

Conclusions: The findings demonstrate that prompt engineering under explicit methodological control can mitigate risks documented in the literature, provide a replicable model for university libraries, and offer a substantive contribution to systematic literature mapping and the grounded identification of knowledge gaps and research voids at multiple geographic levels.

Downloads

Download data is not yet available.

Author Biography

Carlos Andrés Castillo Badilla, University of the Americas

Bachelor of Library and Information Science and Licentiate in Documentary Information from the Universidad Católica de la Santísima Concepción (Chile), with more than eight years of experience in managing academic libraries and supporting teaching services. I am currently pursuing a Master’s degree in Social Research and Development at the University of Concepción, strengthening my research profile with the aim of publishing in indexed journals. I also hold a Diploma in Action Research Applied to University Teaching (Office of the Vice-Rector for Academic Affairs – Faculty of Education), which enhances my skills in applied research and information literacy.

My areas of specialization include the management of academic databases, the standardization of bibliographic references (APA, Vancouver, ISO 690), the development of information literacy skills, user training, collection analysis, and the evaluation of electronic resources. I have advanced experience with bibliographic reference managers (EndNote, Zotero, Mendeley), the development of academic communities, information literacy instruction, and advisory services for scientific publishing. In addition, I work in the digitization of collections, the implementation of library management software, the assessment of performance indicators in university libraries, and research metrics. My professional practice also encompasses team leadership, coordination of accreditation processes, infrastructure optimization, and projects aimed at the continuous improvement of information services.

I have led library digitization projects and actively participated in institutional accreditation processes, including the coordination of peer review visits by the National Accreditation Commission (CNA). I am recognized for my leadership, team management, and process optimization skills, performing strategic functions at the Universidad de Las Américas, where I supervise the updating of curricular bibliographies and the maintenance of faculty records. My experience in information technologies applied to libraries, combined with advanced management of specialized systems and software, has enabled me to promote innovative improvements in infrastructure and management.

I am a member of professional networks related to library and information science and research in the social sciences, promoting the exchange of knowledge and best practices. My current research interests focus on bibliometrics, knowledge management, innovation in academic libraries, and information sustainability.

My goal is to contribute to solutions and produce applied research that strengthens the quality, relevance, and innovation of university library services.

References

ALANIZ, L.; VU, C.; PFAFF, M. J. The utility of artificial intelligence for systematic reviews and Boolean query formulation and translation. Plastic and Reconstructive Surgery Global Open, v. 11, n. 10, e5339, 2023. DOI: 10.1097/GOX.0000000000005339. Disponible en: https://doi.org/10.1097/GOX.0000000000005339. Consultado el: 4 abr. 2026.

ALSHAMI, A.; ELSAYED, M.; ALI, E.; ELTOUKHY, A. E.; ZAYED, T. Harnessing the power of ChatGPT for automating systematic review process: methodology, case study, limitations, and future directions. Systems, Basel, v. 11, n. 7, p. 351, 2023. DOI: 10.3390/systems11070351. Disponible en: https://doi.org/10.3390/systems11070351. Consultado el: 4 abr. 2026.

ARIZA RUIZ, E. D. ChatGPT: una mirada desde la investigación. Revista Investigaciones Andina, Bogotá, v. 25, n. 46, p. 24–39, 2023. DOI: 10.33132/01248146.2256. Disponible en: https://doi.org/10.33132/01248146.2256. Consultado el: 4 abr. 2026.

BALDRICH DOMÍNGUEZ-OLLER, S. La inteligencia artificial y su impacto en la alfabetización académica: una revisión sistemática. Educatio Siglo XXI, Murcia, v. 42, n. 1, p. 55–74, 2024. DOI: 10.6018/educatio.609591. Disponible en: https://doi.org/10.6018/educatio.609591. Consultado el: 4 abr. 2026.

BLUM, M. ChatGPT produces fabricated references and falsehoods when used for scientific literature search. Journal of Cardiac Failure, v. 29, n. 9, p. 1332–1334, 2023. DOI: 10.1016/j.cardfail.2023.06.015. Disponible en: https://doi.org/10.1016/j.cardfail.2023.06.015. Consultado el: 4 abr. 2026.

BRASIL. Ministério da Saúde. Conselho Nacional de Saúde. Resolução n.º 510, de 7 de abril de 2016. Establece las normas aplicables a investigaciones en Ciencias Humanas y Sociales. Diário Oficial da União: sección 1, Brasilia, DF, 24 de mayo de 2016. Disponible en: https://www.gov.br/conselho-nacional-de-saude/pt-br/atos-normativos/resolucoes/2016/resolucao-no-510.pdf/view. Consultado el: 19 sep. 2025.

CAMACHO TAMBASCO, J. El impacto de la inteligencia artificial en la educación: riesgos y potencialidades de la IA en el aula. Revista Interuniversitaria de Investigación en Tecnología Educativa, Murcia, v. 16, n. 1, p. 1–15, 2023. DOI: 10.6018/riite.584501. Disponible en: https://doi.org/10.6018/riite.584501. Consultado el: 4 abr. 2026.

CHELLI, M.; KHALFI, L.; RODRÍGUEZ, L.; TEJADA-LLACSA, P. J.; AUGEREAU, O. Hallucination rates and reference accuracy of ChatGPT and Bard for systematic reviews: comparative analysis. Journal of Medical Internet Research, Toronto, v. 26, e53164, 2024. DOI: 10.2196/53164. Disponible en: https://doi.org/10.2196/53164. Consultado el: 4 abr. 2026.

CHILE. Ministerio de Ciencia, Tecnología, Conocimiento e Innovación. Política Nacional de Inteligencia Artificial. Santiago de Chile: MINCIENCIA, 2024. Disponible en: https://minciencia.gob.cl/areas/inteligencia-artificial/politica-nacional-de-inteligencia-artificial/. Consultado el: 19 sep. 2025.

JI, Ziwei; LEE, Nayeon; FRIESKE, Rita; YU, Tiezheng; SU, Dan; XU, Yan; ISHII, Etsuko; BANG, Yejin; CHEN, Delong; DAI, Wenliang; CHAN, Ho Shu; MADOTTO, Andrea; FUNG, Pascale. Survey of hallucination in natural language generation. ACM Computing Surveys, New York, v. 55, n. 12, p. 1–38, 2023. DOI: 10.1145/3571730. Disponible en: https://doi.org/10.1145/3571730. Consultado el: 1 abr. 2026.

LEE, Y.; OH, J. H.; LEE, D.; KANG, M.; LEE, S. Prompt engineering in ChatGPT for literature review: practical guide exemplified with studies on white phosphors. Scientific Reports, v. 15, n. 1, p. 15310, 2025. DOI: 10.1038/s41598-025-99423-9. Disponible en: https://doi.org/10.1038/s41598-025-99423-9. Consultado el: 4 abr. 2026.

MAHRISHI, M.; ABBAS, A.; RADOVANOVIĆ, D. Emerging dynamics of ChatGPT in academia: a scoping review. Journal of University Teaching and Learning Practice, Wollongong, v. 21, n. 3, p. 1–19, 2024. DOI: 10.53761/b182ws13. Disponible en: https://doi.org/10.53761/b182ws13. Consultado el: 4 abr. 2026.

MOSTAFAHPOUR, M.; FORTIER, J. H.; PACHECO, K.; MURRAY, H.; GARBER, G. Evaluating literature reviews conducted by humans versus ChatGPT: comparative study. JMIR AI, v. 3, e56537, 2024. DOI: 10.2196/56537. Disponible en: https://doi.org/10.2196/56537. Consultado el: 4 abr. 2026.

NAVAS-MARTÍN, M. A.; CUERDO-VILCHES, T. Discurso grupal basado en narrativas generadas por inteligencia artificial como metodología activa en la enseñanza en arquitectura. Advances in Building Education, Madrid, v. 8, n. 1, p. 59–74, 2024. DOI: 10.20868/abe.2024.1.5234. Disponible en: https://doi.org/10.20868/abe.2024.1.5234. Consultado el: 4 abr. 2026.

OPENAI. Introducing deep research. 2025. Disponible en: https://openai.com/index/introducing-deep-research/. Consultado el: 13 feb. 2026.

PARISI, V.; SUTTON, A. The role of ChatGPT in developing systematic literature searches: an evidence summary. Journal of EAHIL, Roma, v. 20, n. 2, p. 30–34, 2024. DOI: 10.32384/jeahil20623. Disponible en: https://doi.org/10.32384/jeahil20623. Consultado el: 4 abr. 2026.

RUIZ MENDOZA, K. K.; PEDROZA ZÚÑIGA, L. H. Uso de ChatGPT como ayudante en una RSL con el método PRISMA. Sciencevolution, México, v. 2, n. 10, p. 9–17, 2024. DOI: 10.61325/ser.v2i10.81. Disponible en: https://doi.org/10.61325/ser.v2i10.81. Consultado el: 4 abr. 2026.

SAHOO, P.; SINGH, A. K.; SAHA, S.; JAIN, V.; MONDAL, S.; CHADHA, A. A systematic survey of prompt engineering in large language models: techniques and applications. arXiv, 2024. DOI: 10.48550/arXiv.2402.07927. Disponible en: https://doi.org/10.48550/arXiv.2402.07927. Consultado el: 4 abr. 2026.

SERCOMBE, J.; BRYANT, Z.; WILSON, J. Evaluating a customized version of ChatGPT for systematic review data extraction in health research: development and usability study. JMIR Formative Research, v. 9, e68666, 2025. DOI: 10.2196/68666. Disponible en: https://doi.org/10.2196/68666. Consultado el: 02 abr. 2026.

UNIVERSIDAD DE LAS AMÉRICAS (Chile). Marco para el uso de la inteligencia artificial en UDLA: docencia, investigación y vinculación con el medio. Santiago de Chile: Universidad de Las Américas, 2025. Serie: Documentos para la formación del cuerpo académico en IA. Disponible en: https://www.udla.cl/descargas/normativas/marco-para-uso-ia-en-udla.pdf. Consultado el: 1 oct. 2025.

WALTERS, William H.; WILDER, Esther Isabelle. Fabrication and errors in the bibliographic citations generated by ChatGPT. Scientific Reports, [S. l.], v. 13, n. 1, art. 14045, 2023. DOI: 10.1038/s41598-023-41032-5. Disponible en: https://doi.org/10.1038/s41598-023-41032-5. Consultado el: 5 abr. 2026.

WANG, S.; SCELLS, H.; KOOPMAN, B.; ZUCCON, G. Can ChatGPT write a good Boolean query for systematic review literature search? En: INTERNATIONAL ACM SIGIR CONFERENCE ON RESEARCH AND DEVELOPMENT IN INFORMATION RETRIEVAL, 46., 2023, Taipei. Proceedings [...]. New York: ACM, 2023. p. 1426–1436. DOI: 10.1145/3539618.3591703. Disponible en: https://doi.org/10.1145/3539618.3591703. Consultado el: 4 abr. 2026.

WILKINSON, C.; OPPERT, M.; OWEN, M. Investigating academics’ attitudes towards ChatGPT: a qualitative study. Australasian Journal of Educational Technology, Sydney, v. 40, n. 1, p. 142–156, 2024. DOI: 10.14742/ajet.9456. Disponible en: https://doi.org/10.14742/ajet.9456. Consultado el: 4 abr. 2026.

ZHOU, D.; ZHANG, Y. Political biases and inconsistencies in bilingual GPT models: the cases of the U.S. and China. Scientific Reports, London, v. 14, n. 13489, 2024. DOI: 10.1038/s41598-024-76395-w. Disponible en: https://doi.org/10.1038/s41598-024-76395-w. Consultado el: 4 abr. 2026.

Capa do periódico Académico Encontros Bibli, da Universidade Federal de Santa Catarina (UFSC). Volume 31, ano 2026 Nela se mostram as mãos de uma pessoa com a pele morena segurando um punhado de terra escura, na qual um pequeno broto verde com duas folhas está plantado. As mãos estão centralizadas na imagem e são enquadradas por um fundo que mostra um solo seco e rachado, sugerindo um ambiente de seca ou desértico. No topo da imagem, o título da publicação, ENCONTROS BIBLI, está em letras maiúsculas brancas. Abaixo, há uma citação entre aspas: "Ética e integridade informacional: a responsabilidade científica e socioambiental da Ciência da Informação diante da urgência climática". Na parte inferior da capa, há informações institucionais: No canto esquerdo, o logo da UFSC. Ao lado, o texto "Departamento de Ciência da Informação", "Programa de Pós-graduação em Ciência da Informação" e "Portal de Periódicos UFSC". Abaixo, a data "2026", o volume "V. 31" e o ISSN 1518-2924. No canto inferior direito, o logo da publicação, que consiste em um escudo roxo com as letras "EB" estilizadas em laranja. A capa utiliza as cores roxo, branco e laranja, com a maior parte da imagem ocupada pelo cenário do solo e as mãos. A arte visual e o tema do título da publicação sugerem um foco na interseção entre a ciência, a informação e as questões ambientais, como a crise climática.

Downloads

Additional Files

Published

2026-08-06

How to Cite

CASTILLO BADILLA, Carlos Andrés. Replicable prompt engineering model for scientific literature retrieval with ChatGPT. Encontros Bibli: electronic journal of library science, archival science and information science, Florianópolis/SC, Brasil, v. 31, p. 1–30, 2026. DOI: 10.5007/1518-2924.2026.e109140. Disponível em: https://periodicos.ufsc.br/index.php/eb/article/view/109140. Acesso em: 29 aug. 2026.

Issue

Vol. 31 (2026): Ethics and informational integrity in the climate emergency: the socio-environmental responsibility of Information Science.

Section

Articles

License

Copyright (c) 2026 Carlos Andrés Castillo Badilla

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

The author must guarantee that:

  • there is full consensus among all the coauthors in approving the final version of the document and its submission for publication.
  • the work is original, and when the work and/or words from other people were used, they were properly acknowledged.

Plagiarism in all of its forms constitutes an unethical publication behavior and is unacceptable. Encontros Bibli has the right to use software or any other method of plagiarism detection.

All manuscripts submitted to Encontros Bibli go through plagiarism and self-plagiarism identification. Plagiarism identified during the evaluation process will result in the filing of the submission. In case plagiarism is identified in a manuscript published in the journal, the Editor-in-Chief will conduct a preliminary investigation and, if necessary, will make a retraction.

This journal, following the recommendations of the Open Source movement, provides full open access to its content. By doing this, the authors keep all of their rights allowing Encontros Bibli to publish and make its articles available to the whole community.

 

Encontros Bibli content is licensed under a Creative Commons Attribution 4.0 International License.

Any user has the right to:

  • Share - copy, download, print or redistribute the material in any medium or format.
  • Adapt - remix, transform and build upon the material for any purpose, even commercially.

According to the following terms:

  • Attribution - You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use.
  • No additional restrictions - You may not apply legal terms or technological measures that legally restrict others from doing anything that the license permits.
Crossref
Scopus
Google Scholar
Europe PMC