Sull’affidabilità delle bibliografie generate dai chatbot
Alcune considerazioni
Keywords:
Artificial Intelligence, Chatbot, Fake References, Hallucinations, Confabulations.Abstract
This paper explores the use of chatbots in the academic field, focusing on the reliability of bibliographies generated by artificial intelligence. By analyzing the language models of OpenAI and Google, the study evaluates the capabilities and limitations of these technologies in providing accurate citations. Through the interrogation of ChatGPT 3.5 and Bard 2.0, the results obtained are examined and compared with similar studies. Criticisms related to the accuracy of citations and the correct classification of sources emerge. Furthermore, the phenomenon of hallucinations and confabulations in Artificial Intelligence is explored, highlighting how these can affect the accuracy of information provided by chatbots.
References
Agrawal, Ayush, Lester Mackey, and Adam Tauman Kalai. 2023. “Do Language Models Know When They’re Hallucinating References?” arXiv, January 2023. https://doi.org/10.48550/arxiv.2305.18248.
Athaluri, Sai Anirudh, Sandeep Varma Manthena, V.S.R. Krishna Manoj Kesapragada, Vineel Yarlagadda, Tirth Dave, and Rama Tulasi Siri Duddumpudi. 2023. “Exploring the Boundaries of Reality: Investigating the
Phenomenon of Artificial Intelligence Hallucination in Scientific Writing Through ChatGPT References.” Curēus 15 (4). https://doi.org/10.7759/cureus.37432.
Brown, Tom B., Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, et al. 2020. “Language Models are Few-Shot Learners.” arXiv, May 2020. https://arxiv.org/abs/2005.14165.
Buruk, Oğuz Oz. 2023. “Academic Writing with GPT-3.5 (ChatGPT): Reflections on Practices, Efficacy and Transparency”. In 26th International Academic Mindtrek Conference, 144-53. Tampere, Finland: ACM. https://doi.org/10.1145/3616961.3616992.
Capaccioni, Andrea. 2021. “Origins and Developments of the Open Access Books.” In Digital Libraries. Advancing Open Science. https://doi.org/10.5772/intechopen.95357.
Capaccioni, Andrea, e Elena Ranfa. 2023. “Gli editori e le monografie scientifiche ad accesso aperto: un rapporto in evoluzione.” Sistema Editoria.Rivista internazionale di studi sulla contemporaneità 1 (1): 79-94. https://doi.org/10.14672/se.v1i1.2124.
Common Crawl. n.d. Consultato il 19 maggio 2024. https://commoncrawl.org/.
DataReportal; W3Techs. 2024. “Languages most frequently used for web content as of January 2024, by share of websites.” Statista, January 2024.https://www.statista.com/statistics/262946/most-common-languages-on-the-internet/.
Day, Terence. 2023. “A Preliminary Investigation of Fake Peer-Reviewed Citations and References Generated by ChatGPT.” The Professional Geographer75 (6): 1024-27. https://doi.org/10.1080/00330124.2023.2190373.
Emsley, Robin 2023. “ChatGPT: these are not hallucinations – they’re fabrications and falsifications.” Schizophrenia 9 (1). https://doi.org/10.1038/s41537-023-00379-4.
Estevez, Almenzar Marina, David Fernandez Llorca, Emilia Gomez Gutierrez, and Fernando Martinez Plumed. 2022. “Glossary of Human-centric Artificial Intelligence.” JRC Publications Repository. https://doi.org/10.2760/86066.
Google. n.d. “Gemini.” Consultato il 19 maggio 2024. https://gemini.google.com.
Ji, Ziwei, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Andrea Madotto, and Pascale Fung. 2023. “Survey of Hallucination in Natural Language Generation.” ACM Computing Surveys
(12): 1-38. https://doi.org/10.1145/3571730.
Knight, Will. 2023. “Meet Bard, Google’s Answer to ChatGPT.” WIRED, February 6, 2023. https://www.wired.com/story/meet-bard-googles-answer-to-chatgpt/.
Macdonald, Calum, Davies Adeloye, Aziz Sheikh, and Igor Rudan. 2023. “Can ChatGPT draft a research article? An example of population-level vaccine effectiveness analysis.” Journal of Global Health 13 (February). https://doi.org/10.7189/jogh.13.01003.
Mahyoob, Mohammad, Jeehaan Algaraady, and Abdulaziz Alblwi. 2023. “Proposed Framework for Human-like Language Processing of ChatGPT in Academic Writing.” International Journal of Emerging Technologies in Learning 18 (14): 282-93. https://doi.org/10.3991/ijet.v18i14.41725.
McGowan, Alessia, Yunlai Gui, Matthew Dobbs, Sophia Shuster, Matthew Cotter, Alexandria Selloni, Marianne Goodman, Agrima Srivastava, Guillermo A. Cecchi, and Cheryl M. Corcoran. 2023. “ChatGPT and Bard Exhibit Spontaneous Citation Fabrication during Psychiatry Literature Search.” Psychiatry Research 326 (August): 115334. https://doi.org/10.1016/j.psychres.2023.115334.
Open AI. 2022. “Introducing ChatGPT.” November 30. https://openai.com/blog/chatgpt/.
Pichai, Sundar, and Demis Hassabis. 2023. “Introducing Gemini: Our largest and most capable AI model.” December 06. https://blog.google/technology/ai/google-gemini-ai/#sundar-note.
Roncaglia, Gino. 2023. L’architetto e l’oracolo: Forme digitali del sapere da Wikipedia a ChatGPT. Bari; Roma: GLF editori Laterza. EPUB.
Sabzalieva, Emma, and Arianna Valentini. 2023. ChatGPT and artificial intelligence in higher education: quick start guide. UNESCO International Institute for Higher Education in Latin America and the Caribbean. https://unesdoc.unesco.org/ark:/48223/pf0000385146.
Science Direct. n.d. “Information Economics and Policy.” Consultato il 19 maggio 2024. https://ww.sciencedirect.com/journal/information-economics-and-policy.
Smith, Craig S. 2023. “Hallucinations could blunt ChatGPT’s success.” IEEE Spectrum, March 29. https://spectrum.ieee.org/ai-hallucination.
Srivastava, Mashrin. 2023. “A day in the life of ChatGPT as an academic reviewer: Investigating the potential of large language model for scientific literature review.” Preprint. Open Science Framework, February 16. https://doi.org/10.31219/osf.io/wydct.
Stokel-Walker, Chris. 2022. “AI Bot ChatGPT Writes Smart Essays-Should Professors Worry?” Nature, December 9. https://doi.org/10.1038/d41586-022-04397-7.
The University of Waikato. n.d. “Open Research.” Consultato il 19 maggio 2024. www.waikato.ac.nz/library/guidance/guides/open-access-information.
Thoppilan, Romal, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, et al. 2022. “LAMDA: Language Models for Dialog Applications.” arXiv, January 2022. https://doi.
org/10.48550/arxiv.2201.08239.
Downloads
Published
Issue
Section
License
Copyright (c) 2024 Aracne Editrice
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
