By Gaurav Singal and Anupam B. Jena

The history of medical diagnosis is a march through painstaking observation. Ancient Egyptian physicians first diagnosed urinary tract infections by observing patterns in patients’ urine. To diagnose diseases of the heart and lungs, medieval doctors added core elements of the physical examination: pulse, palpation and percussion. The 20th century saw the addition of laboratory studies, and the 21st century of sophisticated imaging and genetics.

Despite advances, however, diagnosis has largely remained a human endeavor, with doctors relying on so-called illness scripts — clusters of signs, symptoms and diagnostic findings that are hallmarks of a disease. Medical students spend years memorizing such scripts, training themselves to, for example, identify the sub-millimeter variations in electrocardiogram wave measurements that might alert them to a heart attack.

But human beings, of course, err. Sometimes, misdiagnosis occurs because a doctor overlooks something — when the patterns of illness fit the script, but the script is misread. This happens in an estimated 15% to 20% of medical encounters. Other times, misdiagnosis occurs because the illness has features that do not match known patterns — they do not fit the script, such as when a heart attack occurs without telltale symptoms or EKG findings.

Artificial intelligence can help solve these two fundamental problems — if it’s given enough financial support and deployed correctly.

First, AI is less susceptible to common factors that lead doctors to make diagnostic errors: fatigue, lack of time and cognitive bandwidth when treating many patients, gaps of knowledge and reliance on mental shortcuts. Even when illnesses conform to scripts, computers will sometimes be better than humans at identifying details buried within voluminous health care data.

Using AI to improve the accuracy and timeliness with which doctors recognize illness can mean the difference between life and death. Ischemic stroke, for example, is a life-threatening emergency where a blocked artery impedes blood flow to the brain. Brain imaging clinches the diagnosis, but that imaging must be performed and interpreted by a radiologist quickly and accurately. Studies show that AI, through superhuman pattern matching abilities, can identify strokes seconds after imaging is performed — tens of minutes sooner than by often-busy radiologists. Similar capabilities have been demonstrated in diagnosing sepsis, pneumonia, blood clot in the lungs (pulmonary embolism), acute kidney injury and other conditions.

Second, computers can be useful for illnesses for which we haven’t developed the right scripts. AI can, in fact, diagnose disease using new patterns too subtle for humans to identify. Consider, for example, hypertrophic cardiomyopathy, a rare genetic condition in which the heart’s muscle has grown more than it should, leading to eventual heart failure and sometimes death. Experts estimate that only 20% of those affected are diagnosed, a process that requires consultation with a cardiologist, a heart ultrasound and often genetic testing. What, then, of the remaining 80%?

Researchers across the country, including at the Mayo Clinic and UC San Francisco, have demonstrated that AI can detect complex, previously unrecognized patterns to identify patients likely to have hypertrophic cardiomyopathy, meaning AI-driven algorithms will be able to screen for the condition in routine EKGs.

AI was able to recognize these patterns after examining the EKGs of many people with and without the disease. The rapid growth in health care data — including detailed electronic health records, imaging, genomic data, biometrics and behavioral data — combined with advancements in artificial intelligence technology has created a major opportunity. Because of its unique ability to identify patterns from the data, AI has helped radiologists to find hidden cancers, pathologists to characterize liver fibrosis and ophthalmologists to detect retinal disease.

One challenge is that AI is expensive, requiring large-scale data to train computer algorithms and the technology to do so. As these resources become more ubiquitous, that can make the associated intellectual property difficult to protect, discouraging private investment in these products. More generally, diagnostics have long been considered unattractive investments. Unlike their therapeutic counterparts, which see around $300 billion in research and development investment a year, diagnostics receive a modest $10 billion in private funding.

Then there’s the question of who pays for the use of AI-based tools in medicine specifically. Some applications, such as detecting strokes, save insurers money (by preventing costly ICU stays and subsequent rehabilitation). These technologies tend to get reimbursed more quickly. But other AI solutions, such as detecting hypertrophic cardiomyopathy, may lead to increased spending on costly downstream therapies to treat newly identified chronic illness. Although the use of AI may improve quality of care and long-term outcomes in such cases, without financial incentives for insurers, reimbursement and thus adoption may be slow.

Life sciences companies have on rare occasion agreed to subsidize development or reimbursement of AI-based diagnostics. This will help bridge the gap, but the federal government may need to play a greater role. Federal support for COVID diagnostics during the pandemic drove rapid development of critical tests, and the cancer moonshot project has helped drive R&D in screening and new treatments.

It is usually tough to marshal funding at the scale needed for new medical frontiers. But the National Academies of Medicine has estimated that tens of billions of dollars and countless lives could be saved from improving diagnosis in medicine.

Artificial intelligence offers a path toward that. It should complement, rather than replace, the human expertise that already saves so many lives. The future of medical diagnosis doesn’t mean handing over the keys to AI but, rather, making use of what it can do that we can’t. This could be a special moment for diagnosis, if we invest enough and do it right.

Gaurav Singal is a computer scientist and physician at Harvard Medical School and was previously the chief data officer of Foundation Medicine, a cancer diagnostics company. Anupam B. Jena is an economist, physician and professor at Harvard Medical School and co-author of ” Random Acts of Medicine: The Hidden Forces That Sway Doctors, Impact Patients, and Shape Our Health” and the Random Acts of Medicine Substack. They wrote this column for the Los Angeles Times.