AI and Large Language Models in Academic Psychology

Artificial intelligence, including large language models such as ChatGPT, has gained significant attention in academic circles. There have been criticisms and excitement about the opportunities for increased productivity. The primary questions are: What is it? What are the potential dangers? And how can these tools be used safely and effectively?

What Is It?

Artificial intelligence refers to computer systems designed to perform tasks typically expected of humans, including decision-making, inference, reasoning, information processing, and visual perception. A subset of artificial intelligence is large language models (LLMs), which produce text based on previously developed and published information found on the internet. ChatGPT is the most well-known LLM. Although LLMs have been called "stochastic parrots," they are capable of answering questions and creating large amounts of text (Bender et al., 2021). It should also be noted that LLMs are capable of producing computer code using the same logic.

What Are the Potential Dangers?

Despite enthusiasm for the use of LLMs in science, there are four primary concerns that provide guide rails concerning the widespread implementation in science.

The first concern is that eventually, artificial intelligence will replace scientists and academics. If Robert Heinlein's quote, "Most scientists are bottle washers and button sorters," is accurate, then AI will replace this type of scientist. Mediocre scientists whose work is primarily a rehash of the work of others may have a reason to worry. At this point, there is no risk of AI replacing creative, innovative, useful, and insightful scholarship.

Plagiarism is a real concern, and LLMs have been labeled as “automated plagiarizers.” Experts in a particular field who request text generated by LLMs may find their exact words used in the LLM-generated response, as the output is based on existing work available on the internet. The challenge with LLMs is that they do not provide attribution to the original author, and it is often difficult for the user to locate the original source. Additionally, LLMs may autofill knowledge gaps with fictional scenarios, referred to as “hallucinations,” making it challenging to determine if the text is accurate. Due to the possibility of undetected plagiarism and accuracy concerns, scholars are strongly advised against using AI-generated text for any essay or paper (Rooij, 2022).

Although many student assignments can be completed entirely by LLMs, detecting their use is possible, but challenging, with existing software (Younis, 2023). Therefore, instructors making assignments need to be mindful of the purpose of the task. Assigning students to summarize a paper, for example, will likely result in an LLM-generated response (Mucharraz y Cano et al., 2023). Instructors are advised to raise the bar by requiring assignments that focus on how ideas are implemented, what the next steps are in research, how studies could be improved, how new technology could enhance the measurement and quality of research, and other creative, value-added thought. Such assignments will go beyond LLM use to be more engaging, useful, and practical, and require deeper knowledge and learning.

Because LLMs use existing scientific literature and information available on the internet, they are likely to expand and reinforce biases present in the extant literature (Weidinger et al., 2021). For instance, research in psychology has a history of excluding diverse samples with linguistic, cultural, racial, and ethnic factors minimally considered. As a result, the text generated by LLMs will share these weaknesses and amplify them. Moreover, evidence shows that research conducted by minority and female scholars is not cited at the same level as that of white male scholars, perpetuating and reinforcing this inequity in the scientific literature (Birhane, 2021).

How to Use Tools Effectively and Ethically

The advantage of LLMs is that they have the potential to free academics from tedious tasks, such as creating boilerplate, form letters, and conducting simple administrative work. If used widely, such models will allow academics to think, innovate, and create. At best, LLMs have the potential for academics to be more human and reach greater heights of creativity and productivity (Winchester, 2023).

My thinking on the use of artificial intelligence tools has evolved. Only six months ago, I insisted on solely handmade artisanal manuscripts. However, I have found utility for several purposes that avoid many of the potential problems, add efficiency, and allow for a higher quality of academic work. I am sure my thinking will continue to evolve with technology improvements and careful consideration of the downsides of using these techniques.

Conducting literature searches is among the most valuable and time-saving uses of AI. There are a variety of programs and websites available. Research rabbit (researchrabbitapp.com/) is helpful for finding new papers related to the topic of interest. Elicit (elicit.org/) is a solid tool for evaluating similarities and differences in published studies (e.g., sample size, research methods, conclusions). In addition, SearchSmart (www.searchsmart.org) is an excellent tool for determining which databases are most effective when searching for a specific topic. All these systems work well and are compatible with manuscript management systems (e.g., Zotero).

Artificial Intelligence Essential Reads

Is OpenAI's Latest LLM Advancement Also a Step Back?

How We Talk About AI Is Important: A Case Study

Reading papers is always a challenge. Reading 50 to 100 new papers for every manuscript written is common. Most scholars review the title and abstract before determining if a detailed and time-intensive read is required. ChatDoc (https://chatdoc.com/) summarizes papers and provides a more detailed overview than can be achieved in the abstract. An old-fashioned close read of the methodology and analysis sections for empirical manuscripts is still recommended. Yet, Chat Doc is a strong tool for quickly understanding complex theoretical papers.

Outsourcing manuscript writing to LLMs is a poor idea. However, using AI for revising text is an excellent approach. Writing poor sentences and struggling to improve them is a time-consuming activity. Using Compose AI, a Google extension, to rephrase sentences is simple and helpful. It offers six to eight options for paraphrasing a sentence or paragraph. Some scholars who have English as a second language use LLMs to enhance the quality of their English language communication. Revising and rephrasing are strong and appreciated uses of LLMs.

Formatting for grant or journal submissions can be a time-consuming and tedious effort. AI is useful for reformatting tables and text to meet journal specifications. For instance, a paper may be written with Chicago-style references, but APA-style referencing is required for a journal. AI tools (e.g., chat.openai.com/) are excellent at reformatting references or citations.

Conclusion

Like all valuable tools, using AI in research productivity for academics has risks. Simply because we can use AI and LLMs does not mean we should use them without limitations. Considering the strengths and drawbacks of these tools can guide how to use AI in a positive way while avoiding common problems with research in psychology.