Why a Hybrid Approach is Key for Bridging the Mental Health Data Gap

By David Talby, CEO, John Snow Labs

As health systems navigate rising costs, workforce strain, and an increasingly complex patient population, another crisis continues to deepen behind the scenes and it might surprise you. Despite an explosion of wellness platforms and increasing awareness around mental health, data from healthcare organizations reveal a troubling reality: many mental health disorders remain hidden in plain sight.

For health systems, this blind spot in mental health diagnosis carries significant clinical and operational risks. It’s not all bad news, though. With the right tools and processes, providers can finally access a more complete picture of the state of mental health. This can lead to improved care quality, outcomes, and safety. And it starts with the data. 

Where Data Falls Short

Recent research led by Oracle Health, John Snow Labs, and the Children’s Hospital of Orange County analyzed over 109,000 patients using Oracle’s Real-World Data platform. The goal was to measure how well neuropsychiatric symptoms were captured using structured EHR data alone versus structured data combined with unstructured clinical notes.

The gap was stark: suicidality and self-harm events were twice as common when unstructured data was included. Symptoms such as agitation, memory issues, and irritability often appeared only in clinicians’ narrative notes—missing entirely from structured fields. This underreporting is not just a data quality issue. It represents missed diagnoses, delayed interventions, and potential safety risks.

The High Cost of Structured-Only Approaches

Structured data—from diagnosis codes and prescription records, for exacmple—serves as a foundational element in modern health systems. However, these fields often exclude the nuances of behavioral health. Clinicians may intentionally avoid formal diagnoses due to stigma, uncertainty, or deferral to specialists. And what remains is formal codes without any clinical context.

By contrast, unstructured notes reflect the real-world dynamics of patient encounters. They can capture mood shifts, behavioral cues, cognitive changes, and patient-reported concerns. Without these narratives, care teams may miss early signs of certain conditions or overlook non-obvious symptoms that could shape treatment decisions.

This not only impacts patient outcomes, but can be a serious burden on hospitals and other providers. For healthcare leaders focused on quality metrics, readmissions, or value-based care, this information gap can lead to poor care planning and missed quality benchmarks.

NLP and LLMs are Forging the Path 

Artificial intelligence (AI), specifically natural language processing (NLP) and large language models (LLMs) can enable health systems to bridge the data divide. By extracting insights from unstructured clinical text, paired with the foundational knowledge we have of structured sources, the AI can surface hidden symptoms, detect trends across populations, and flag patients at risk earlier and more accurately. 

According to the aforementioned study, the inclusion of unstructured data via NLP led to a 20% increase in detected outcome events. This is a significant improvement that has a direct impact on clinical decision support, care coordination, and risk stratification programs. Although the value is clear, implementation isn’t as cut and dry.

AI is powerful, but it’s not turnkey. To run properly, providers need annotated data, in some cases, a data scientist, clinician validation, and robust governance frameworks to operationalize AI models. Many organizations also face resource limitations, from staffing and budget to IT expertise.

Hybrid Models are the New Gold Standard 

To meet the demands of real-world clinical environments, hybrid AI pipelines, which combine structured rule-based systems with flexible, reasoning-capable LLMs, are emerging as the gold standard. These models not only extract clearly defined symptoms but also infer subtler signs of distress that would be missed by traditional methods.

Unlike general-purpose AI, healthcare-specific LLMs can adapt to clinical language, learn from evolving documentation styles, and operate across diverse systems. For hospitals, this means scalable, high-fidelity extraction of behavioral health data from notes, regardless of EHR platform or documentation variance.

This unlocks a powerful capability: automated, comprehensive mental health surveillance that supports earlier diagnosis, improved drug safety research, and more precise population health management.

A Look Ahead

Millions of people are affected by mental illness each year—and these are just the documented cases. In fact, nearly 60 million adults (23.08%) experienced a mental illness in the past year, according to research from Mental Health America. Among other worrisome findings, nearly 13 million adults reported serious thoughts of suicide. The number of individuals who died by suicide in 2022 was the highest number ever recorded in the US. 

The same research found that more than half (54.7%) of adults with a mental illness do not receive treatment, totaling over 28 million individuals. Even in states that ranked high in care, such as Montana (ranked #1), over 4 in 10 adults with a mental illness did not receive care. Again, this is only inclusive of the people who have reported experiencing symptoms. 

To summarize, for now, mental health is getting worse, not better. In response, we need tools that capture the full clinical narrative, and not just what’s codified in structured data. Integrating AI-powered analysis of unstructured notes into hospital workflows is no longer a forward-looking experiment, but an essential piece of the puzzle for delivering equitable, informed, and proactive care. After all, our mental health depends on it.