Seaver College Student Seeks Medical Applications for Large Language Models

Plagiarism. Cheating. And the future of healthcare. 

With ChatGPT, Claude 2, and many other large language models (LLMs) readily accessible to anyone with a computer today, academics have bristled at how students could employ AI to engage in the first two activities. Yet, emerging from the increasingly murky world of academic integrity comes an unexpected result:

Students using these new innovations to improve the latter.

Sean Wu, a  junior computer science major at Seaver College, invested his summer vacation in studying how LLMs might be used in the medical field. Specifically, Wu led a team of six other researchers as they tested LLMs’ ability to process detailed medical terminology. 

In the midst of this endeavor, the cohort sought to understand how open-source AI programs (whose code can be accessed and manipulated by its users), and closed-source AI programs (like GPT-4, whose code cannot be accessed or manipulated),varied from one another.

With this goal in mind, Wu and his team put a set of AI programs through a series of 858 multiple-choice questions related to nephrology. The insights their study yielded represent a step toward a more integrated and efficient medical training practice. 

Wu and his cohort found that open-source LLMs struggled to answer questions correctly. Most of these more accessible programs scored between 17.1 percent and 25.5 percent correct; whereas closed-source programs such as Claude 2 and GPT-4 performed much better, at 54.4 percent and 73.3 percent, respectively. These findings could go on to benefit the medical industry, as practitioners seek new ways to incorporate this burgeoning technology. Two potential uses for AI may come in the form of refined training techniques and enhanced medical copilot applications. 

Wu explained that the project was started as a collaboration between the department of nephrology at the University of California, Los Angeles and Fabien Scalzo, associate professor of computer science. Working alongside his professor, Wu was allowed to develop and embark upon a series of research projects in the pursuit of creating a generalized AI medical practitioner. 

Now, while his summer research project awaits publication, the computer science major has set his sights on PhD programs —where he hopes to continue integrating AI with medicine.

“I would like to continue conducting research in computer vision and its possible applications to medicine,” Wu says. “Previously, I have worked on such projects in ophthalmology and neuroradiology, so I would love to continue this research direction.”

Thus, while plagiarism, cheating, and academic dishonesty all surround the concept of AI’s integration into higher education, Sean Wu has found a way for these technological developments to encourage personal learning as opposed to diminishing it. And the effects of his curiosity could well go on to affect the future of healthcare.