Since the release of ChatGPT last year, there has been a lot of excitement over the speed and power of artificial intelligence. One potential benefit is that AI language models seemingly allow scientists and journalists to quickly search for scholarly articles on different subjects. However, when asking ChatGPT to retrieve content from academic journals, there are often limitations. Duke University and Macquarie University have already published warnings of the inaccurate scientific journal results that AI language models such as ChatGPT may produce.
To test ChatGPT’s capabilities, Hoag Levins of Penn’s Leonard Davis Institute of Health Economics and Marcella Barnhart of Wharton’s Lippincott Library asked ChatGPT to provide a list of citations for top scientific journal articles regarding racial disparities and birth outcomes. The results: Every citation was wrong. ChatGPT provided inaccurate information for each of its recommended articles, most of which did not even exist. When asked about the mistakes, ChatGPT stated: “I apologize if any of the scientific articles I cited in response to your previous queries contained errors.”
See the full article from the Leonard Davis Institute here.
