LLMs in Healthcare at Scale – the promise, the reality, the cost, and the path forward

By Nassib Chamoun

Background

As the inevitable transition to an “intelligent health system” accelerates, the development, implementation, and deployment of AI-powered solutions is moving at speeds uncharacteristic of healthcare. Leveraging intelligence in all aspects of healthcare management and operations is becoming essential in helping to address many of the industry’s most pressing challenges, including clinician burnout, increased complexity of care, strained resources, operational inefficiencies, and EHR-inflicted productivity drag.

The Promise

Large Language Models (LLMs) provide powerful new capabilities that can uniquely support many transformational solutions in healthcare. With LLMs, the ability to unlock the value of unstructured data is at our fingertips like never before. Proponents of Generative AI promise unprecedented improvements in productivity, personalization, and reductions in clinician burnout. The belief is that as data rapidly changes, spanning large corpuses of tokenized text, images, and audio, the LLMs will adapt, leveraging their immensely deep and complex weights to provide thoughtful, ethical, and accurate responses across a variety of use cases in near-real time.

Reality Check

However, beyond meeting performance and responsible healthcare AI requirements, little consideration has been given to the energy consumption and related carbon emissions associated with the computational burden of the training, response generation, and fine-tuning of these AI models to support healthcare use cases. Perhaps the most overlooked factor in delivering AI at scale in healthcare is the “inference rate” which is driven by frequency at which these inputs are changing and the rate at which the model outputs must be updated. For AI to enable enterprise-wide healthcare use cases, including generation of clinical documentation, extraction of structured data from medical records, risk stratification, diagnostics, and utilization, cost, and operational predictions, it must efficiently thrive in a high inference rate environment.

The Cost

Recently, The New York Times reported, “by 2027 AI servers could use between 85 to 134 terawatt hours annually.  That’s similar to the annual energy use of Argentina or Sweden¹.”  A major contributing factor to this projection is the widespread adoption of energy-intensive technologies such as LLMs and Generative AI models, including GPT models. With healthcare generating more than 30% of all data globally, the deployment of AI at scale across the sector is likely to drive a disproportionate share of the forecasted energy use by AI servers in the next 5 years. A recent eye-opening study by researchers at AI startup Hugging Face and Carnegie Mellon University found that using large generative models to create outputs was far more energy intensive than using smaller AI models tailored for specific tasks². This highlights the importance of selecting for each use case an AI architecture optimized for the appropriate tradeoffs between performance and cost.

Fit to Purpose

The resulting costs from broad adoption of LLMs in healthcare could be far greater than anticipated and are likely to further strain extremely challenged healthcare budgets. Of course, such expenses would be justified if the impact on productivity or patient outcomes far exceeded the investment. STAT News summarized several papers that highlighted significant shortcoming of notes or assessments generated by LLMs including more time spent by clinicians reading AI-generated responses³. In addition, a study evaluating an experimental institution-wide deployment of GPT-4 for research and operational exploration at Dana-Farber Cancer Institute raised many questions including significant cost concerns even within the limited scope of their deployment⁴.

Roberta Schwartz, Executive Vice President and Chief Innovation Officer at Houston Methodist, made it clear at a recent conference that AI in healthcare must always rely on “a human in the loop” and that it cannot distract or further burden the user. In my view today, the best “human in the loop” for LLMs are experienced healthcare AI developers supported by clinical experts who can leverage the power of LLMs to rapidly develop, train and optimize targeted algorithms using more efficient and understandable AI tools that meet the performance, cost, and transparency requirements necessary for healthcare.

HDAI's Experience

Beyond published data, many who have used LLMs extensively in healthcare know that while some of the output is remarkable, much of it is also very concerning. This aligns with Health Data Analytics Institute (HDAI)‘s experience where in collaboration with Houston Methodist, we analyzed millions of physician notes with GPT-3.5/4.0 to extract diagnostic codes. In parallel, we processed the same data with HDAI’s deterministic NLP algorithm based on methodologies we have been using for many years. The results, which also included human review of thousands of notes, led us to discard more than 90% of the codes generated by GPT due to redundancies within ICD families or total hallucinations that were completely off base. Furthermore, the cost and time required to process the notes was nearly 1,000 times greater for GPT when compared to our targeted NLP solution. On the other very positive side, GPT was able to identify some complex patterns that were missed by traditional models. Ultimately, we adopted a hybrid approach that leverages the strengths of each technology to deliver a performant, cost-effective and high-throughput solution that can be deployed at-scale for our intended use case.

Path Forward

Many of the recently published studies and our direct experience in using LLMs make it difficult to see LLMs as a primary solution for use cases with high inference frequency anytime soon due to a wide range of challenges⁵⁶⁷.  Beyond the cost and performance issues, we also need to be cognizant of responsible healthcare AI requirements, including consistency and reproducibility of results, transparency, and explainability. Nonetheless, today we can leverage LLMs and GPTs for training cheaper, faster, and task-specific AI models or run them in parallel in a more cost-effective hybrid framework that can deliver urgently needed capabilities. Several collaborative communities are forming to guide others along this path. VALID AI out of University of California, Davis and Ashish Atreja, MD, MPH are already deeply focused on these issues including the development of targeted Small Language Models (SLMs).

The "Intelligent Health System"

As Harry P. Pappas and Paul Frisch, PhD state in their new book, The Rise of the Intelligent Health System, “…an Intelligent Health System is an entity that leverages data and AI to create strategic advantages through the efficient provision of health and medical services across all touchpoints, experiences, and channels⁸.”

Looking ahead, the major challenge for US healthcare is the need to focus on the paradigm shift required to successfully transition to an environment where embedded intelligence becomes an integral part of every operational and clinical workflow. Market leaders will take advantage of the rapid cycles of innovation by tech companies to transform the way they deliver care. Health systems must invest in the AI literacy of their workforce by exposing them to solutions and technologies that enable frontline workers to experiment, learn, innovate, and reinvent their work environment. I look forward to the day in the not-so-distant future when widespread, responsible use of AI will significantly reduce the burden on clinicians, enhance care-team productivity, and deliver better, cost-effective outcomes for patients.

#HealthcareonLinkedIn #healthcareAI #generativeAI Beth Kutscher Haley Deming Coalition for Health AI (CHAI)OpenAI Joint Commission The New York Times STAT JAMA, Journal of the American Medical Association NEJM Group

Contact us!

If you would like to discuss your work in the healthcare space and learn more about HDAI, please reach out to [email protected]. We would love to speak with you.  

References

 “A.I. Could Soon Need as Much Electricity as an Entire Country” https://www.nytimes.com/2023/10/10/climate/ai-could-soon-need-as-much-electricity-as-an-entire-country.html

“Power Hungry Processing: Watts Driving the Cost of AI Deployment?” https://arxiv.org/pdf/2311.16863

“Generative AI is supposed to save doctors from burnout. New data show it needs more training.” https://www.statnews.com/2024/04/25/health-ai-large-language-models-clinical-documentation

“GPT-4 in a Cancer Center: Institute-Wide Deployment Challenges and Lessons Learned” https://ai.nejm.org/doi/full/10.1056/AIcs2300191

“AI-Generated Clinical Summaries Require More than Accuracy” https://jamanetwork.com/journals/jama/article-abstract/2814609

“AI-Generated Draft Replies Integrated Into Health Records and Physicians’ Electronic Communication” https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2817615?resultClick=1

“Large Language Models are Poor Medical Coders – Benchmarking of Medical Code Querying” https://ai.nejm.org/doi/full/10.1056/AIdbp2300040

“The Rise of the Intelligent Health System” by Paul H. Frisch and Harry P. Pappas https://www.routledge.com/The-Rise-of-the-Intelligent-Health-System/Pappas-Frisch/p/book/9780367769345

Your privacy matters

We believe that an individual’s meaningful control over their own data is as valuable as the data itself. That’s why we are committed to protecting privacy at every turn.
We always:
  • Provide transparency about the information we collect and why we collect it
  • Comply with all applicable privacy rules and regulations
  • Delete personal data from our servers if an individual opts out
We never:
  • Sell data to third parties
  • Share data with third parties besides the primary, explicitly authorized institution
  • Collect data besides what is disclosed
  • Keep personal data for individuals who have opted out

We speak legalese only when we have to, but we do our best to keep it human. Provided below are the full versions of our Privacy Policy and Terms of Service. If you have any questions, don’t hesitate to contact us for clarification.

George Pace

VP Business Development, Payer Markets

George has over two decades of consulting, business development and client success experience in health care, working with both providers and payers to address their most pressing challenges. The bulk of his healthcare career was with The Advisory Board, Optum, Premier and Verisk Health, with a focus on risk mitigation, value-based care and population health management. He is deeply passionate about helping healthcare organizations do their best for their patients and beneficiaries. 

When he’s not working, you will find him playing guitar, riding his bike or spending time with family, friends, and his dog Luna.  

Summer Kramer
VP Clinical Programs

Summer is a clinical pharmacist with over 15 years of experience in the healthcare industry, including hospitals, payors, and start-ups. Summer has focused her career on clinical program development and healthcare analytics with a constant drive to improve patient outcomes.

Residing in the pacific northwest, Summer enjoys hiking, camping, sailing, and traveling. She received her PharmD from the University of New Mexico, College of Pharmacy.  

Karan (Kay) Mansukhani
VP Business Development

Kay has a decade of experience in population health analytics.  Starting his career in Maryland’s state-designated HIE – CRISP,  he was entrusted to set up the infrastructure and data elements needed by health system to achieve CMMIs Triple Aim. Kay is a trained epidemiologist who has a passion for preventive medicine, psychology, and anthropology. Kay enjoys spending lazy days with his wife Sonal, and son Avtar on the beach.  Kay also enjoys listening to podcasts, playing the guitar and badminton. He received his joint MPH MBA from Johns Hopkins University.

Kevin Sampson
VP Finance

Kevin is a CPA with 25 years of executive and managerial experience in accounting, corporate finance, and operations in a variety of industries including medical device, biotechnology, and pharmaceuticals.

By listening and connecting with people, Kevin understands that accurate timely information is vital for everyone in the healthcare system. He received a BS in Accounting and an MBA in Finance and Entrepreneurial Studies from Babson College.

Paul Manberg
VP Clinical and Regulatory Affairs

Paul is an innovative, pragmatic leader with over 40 years’ experience in the medical device and pharmaceutical industries. He has directed clinical trial programs and regulatory activities supporting the successful market introduction of numerous impactful products. He spends his leisure time on the Outer Banks of North Carolina, body surfing and fishing. He received a PhD in pharmacology from The University of North Carolina School of Medicine.

Scott Greenwald
Chief Scientist

With over 25 years’ experience in healthcare research and innovation, Scott is an unprecedented leader in his field, focusing on improved care for everyone. He is an author of over 10 US Patents and over 100 peer-reviewed abstracts and manuscripts. While he’s not immersed in research, you’ll find him playing a mean round of mini golf. He received a PhD in Medical Engineering from the Harvard-MIT Division of Health Sciences and Technology (HST).

Josh Gray VP Analytic Services

Josh has 30 years’ experience in healthcare, including The Advisory Board Company, athenahealth, and Boston Consulting Group. Across his career, he has used a wide range of conventional and unconventional data sources to understand and teach the keys to top performance among the nation’s most innovative health systems and physician groups. Josh enjoys music, hiking, traveling, and watching his daughter take part in synchronized skating competitions. He received an MBA from The Wharton School at the University of Pennsylvania and an undergraduate degree from Brown University.

Carola Endicott
VP Customer Engagement, Marketing

Carola has 30 years’ experience in healthcare, including HMOs, hospitals, and start-ups. Her knowledge and optimistic outlook in operations and leadership, and her passion for making healthcare workers’ lives easier is what drives her devotion to customer engagement. She received an EdM from Harvard University Graduate School of Education.

David Clain Chief Product Officer

David has vast experience with healthcare customers from his time at The Advisory Board Company and athenahealth. His expertise and passion for statistics, strategy, and innovative analytic insights is unrivaled, as is his devotion to the Duke Blue Devils. Looking for that “aha moment” of positive change is what drives him. He received a BA from Duke University.

Margie Ahearn
VP HR

Margie has 25 years of experience in HR leadership. An avid boater, Margie has received numerous awards for excellence in HR and is always looking for challenges to grow professionally and personally. She graduated with an MBA from Southern New Hampshire University.

Nassib Chamoun
Founder and CEO

Nassib is a data scientist, lover of logistical regression and a serial entrepreneur. Previously, he was Founder and CEO of Aspect Medical Systems, a Nasdaq traded company now owned by Medtronic. Outside of work, he enjoys hosting and honing his cooking skills – specializing in Middle Eastern and outdoor feasts. He received an MS from Boston University.

Phil Devlin
COO

Phil has 30 years of experience in executive leadership. Along his journey he’s made effective and profound change in science. Little known facts about him — he loves traveling with his family, he was in the MD Engineering program at Harvard Medical School and owns 15 patents. He received an MS from Northeastern University.

What guides our culture

1

We are driven to make a profound impact on US healthcare

2

We operate in the best interests of patients and those who serve them

3

We understand and protect patient privacy and health information security

4

We use data science to enhance, but never replace clinical expertise and judgment

5

We are committed to continuous learning and improvement

6

We bring rigor, discipline, and reliability to our work

7

We start with kindness, assuming good intent in others

8

We are committed to building a diverse, inclusive team where all employees feel welcomed and appreciated

9

We are committed to the well-being of our employees and their families

Our vision

We envision a healthcare system where decisions are routinely informed by personalized care profiles and predictive analytics, which will improve the health of individuals and populations.

Our mission

Deploy a versatile analytic platform that creates a shared understanding of quantified health risks to inform actions with the greatest potential to benefit patients.

Who we are

We collaborate with leading health systems and integrated delivery networks, value-based care and accountable care organizations, physician groups, and commercial payers to improve care delivery and population health while reducing costs and making efficient use of scarce clinical resources.

Phil Devlin
COO

Phil has 30 years of experience in executive leadership. Along his journey he’s made effective and profound change in science. Little known facts about him — he loves traveling with his family, he was in the MD Engineering program at Harvard Medical School and owns 15 patents. He received an MS from Northeastern University.