Leading AI models struggle to identify genetic conditions from patient-written descriptions
NIH researchers find that large language models rely on concise, textbook-like language to evaluate medical questions
National Institutes of Health (NIH) researchers discover that while artificial intelligence (AI) tools can make accurate diagnoses from textbook-like descriptions of genetic diseases, the tools are significantly less accurate when analyzing summaries written by patients about their own health. These findings, reported in the American Journal of Human Genetics, demonstrate the need to improve these AI tools before they can be applied in health care settings to help make diagnoses and answer patient questions.
The researchers studied a type of AI known as a large language model, which is trained on massive amounts of text-based data. These models have the potential to be very helpful in medicine due to their ability to analyze and respond to questions and their often user-friendly interfaces.
“We may not always think of it this way, but so much of medicine is words-based,” said Ben Solomon, M.D., senior author of the study and clinical director at the NIH’s National Human Genome Research Institute (NHGRI). “For example, electronic health records and the conversations between doctors and patients all consist of words. Large language models have been a huge leap forward for AI, and being able to analyze words in a clinically useful way could be incredibly transformational.”
This page was last updated on Wednesday, August 14, 2024