New service leverages speech recognition and generative AI to automatically create preliminary clinical documentation from patient-clinician conversations

3M Health Information Systems, Babylon Health, and ScribeEMR among customers and partners using AWS HealthScribe

With AWS HealthScribe, healthcare software providers can use a single API to automatically create robust transcripts, extract key details (e.g., medical terms and medications), and create summaries from doctor-patient discussions that can then be entered into an electronic health record (EHR) system. Powered by Amazon Bedrock, AWS HealthScribe makes it faster and easier for healthcare software providers to integrate generative AI capabilities into their application starting with two popular specialties (i.e., general medicine and orthopedics), without needing to manage the underlying machine learning (ML) infrastructure or train their own healthcare-specific large language models (LLMs). AWS HealthScribe enables responsible deployment of AI systems by citing the source of every line of generated text from within the original conversation transcript, making it easier for physicians to review clinical notes before entering them into the EHR. Built with security and privacy in mind, AWS HealthScribe gives customers control over where their data is stored, encrypts data in transit and at rest, and does not use inputs or outputs generated through the service to train its models. To learn more about AWS HealthScribe, visit https://aws.amazon.com/healthscribe.

Generative AI is quickly transforming many industries, including healthcare and life sciences. As interest in generative AI continues to grow, healthcare software vendors are looking to leverage this technology in their clinical applications to solve common pain points for clinicians in the healthcare industry. One of the most common issues is compiling clinical documentation after every patient-clinician discussion. This is important for compliance, quality measures, and reimbursement, but it is also a complex, multi-step process that takes time away from seeing patients. While many of these healthcare software providers use text to speech and natural language processing (NLP) to streamline this process today, generative AI has been the missing piece to help these applications go from recorded discussions to concise clinical documentation that can be entered into an EHR. However, working with generative AI is complex, and integrating multiple AI systems into a cohesive solution requires significant engineering resources. To build these generative AI capabilities, a provider must train or fine-tune their own LLM to generate accurate clinical documentation, which requires access to in-demand AI experts, massive amounts of carefully annotated healthcare data, and significant compute capacity. Even then, an LLM for healthcare needs to be specially trained to understand complex medical terminology across different specialties (e.g., general medicine, pediatrics, or orthopedics), to be capable of understanding, analyzing, and summarizing free-flowing discussions, as well as recognizing prescription names and dosages. To ensure these solutions are working properly, software providers must also build with responsible AI in mind, including designing the solution so that clinicians can trace the origin of any generated text to mitigate the risk of errors or hallucinations. Healthcare software providers must also dedicate engineering time and resources to ensuring these systems meet the stringent security and privacy requirements of the healthcare industry. Because of these barriers, it is challenging for healthcare software providers to bring AI-powered solutions to market quickly, despite the potential benefits to both clinicians and patients.

AWS HealthScribe is an AI-powered, HIPAA-eligible health service that enables healthcare software providers to build clinical applications that save clinicians time by automatically creating transcripts, generating notes, and analyzing patient-clinician conversations. With a single, easy-to-use API, healthcare software providers can create these clinical solutions quickly and focus on building a differentiated experience for their end users, reducing the need to integrate and optimize multiple separate AI services into their application. By integrating AWS HealthScribe into a clinical application, healthcare providers can leverage built-in text-to-speech capabilities to create robust conversation transcripts that identify speaker roles and segment transcripts into categories (e.g., small talk, subjective comments, or objective comments) based on clinical relevance. The application can then use AWS HealthScribe’s NLP and generative AI capabilities to extract structured medical terms, such as medical conditions and medications, and generate discussion-based notes that include relevant details (e.g., key takeaways, reason for visit, and history of the present illness) that a clinician can review and finalize in their EHR. With generative AI capabilities powered by Amazon Bedrock, AWS HealthScribe is designed to create clinical notes for two medical specialties (i.e., general medicine and orthopedics) allowing physicians to focus on their discussions with patients rather than capturing details to enter into the EHR. Every sentence used in the AI-generated clinical notes comes with references to the original doctor-patient conversation transcripts, allowing clinicians to easily view the historical context of notes for greater accuracy and transparency. Data security and privacy are also built into the service–the service does not retain any customer data after processing the customer request and encrypts customer data in transit and at rest. Healthcare software providers have control over where they want to store transcriptions and preliminary clinical notes, maintaining ownership of their content. Additionally, the inputs and outputs generated through the service will not be used to train AWS HealthScribe.

“Our healthcare customers and partners tell us they want to spend more time creating innovative clinical care and research solutions for their patients while spending less time building, maintaining, and operating foundational health data capabilities,” said Bratin Saha, vice president of Machine Learning and Artificial Intelligence Services at AWS. “That is why AWS has invested in building a portfolio of AI-powered, high-performance, and population-scale health applications so that clinicians can spend more time with the patients during the face-to-face or telehealth visits. Documentation is a particularly time-consuming effort for healthcare professionals, which is why we are excited to leverage the power of generative AI in AWS HealthScribe and reduce that burden. Today’s announcement builds on AWS’s commitment to the healthcare and life sciences industry and our responsible approach to technologies like generative AI to help reduce the burden of clinical documentation and improve the consultation experience.”

AWS HealthScribe is part of a broad set of purpose-built health services that help thousands of healthcare and life sciences customers reinvent how they collaborate, make data-driven clinical and operational decisions, advance precision medicine, and decrease the cost of care. Continuing its innovation in the healthcare field, AWS today also announced the general availability of AWS HealthImaging, a service that makes it easier to store, transform, and analyze medical imaging data at a petabyte scale—delivering performance while reducing the burden of provisioning underlying infrastructure. To learn more about AWS HealthImaging, visit https://aws.amazon.com/blogs/industries/introducing-aws-healthimaging/.

3M Health Information Systems (HIS) is an industry leader whose various M*Modal speech understanding, conversational, and ambient AI solutions are currently used by more than 300,000 clinicians. “Machine learning on AWS enables 3M HIS to transform clinician workflows and laborious processes to help healthcare organizations streamline clinical documentation and billing,” said Garri Garrison, president, 3M HIS. “3M HIS is collaborating with AWS to bring conversational and generative AI directly into clinical documentation workflows. AWS HealthScribe will be a core component of our clinician applications to help expedite, refine, and scale the delivery of 3M's ambient clinical documentation and virtual assistant solutions.”

Babylon is an integrated digital-first primary care service that manages population health at scale. “Integrating AI with human medical expertise can make quality healthcare more affordable and accessible, and alleviates burdens on providers,” said Saurabh Johri, chief science officer, Babylon. “Innovating in areas like clinical summarization is one example with the potential to improve healthcare outcomes. As a leader in AI innovation, Babylon looks forward to continuing our collaboration with AWS and exploring integrating AWS HealthScribe’s generative AI capabilities with our natural language processing solutions.”

ScribeEMR is a leading provider of virtual medical scribing, virtual medical coding, and virtual medical office services for hundreds of medical practices, hospitals, and health systems. “ScribeEMR’s goal is to help increase practice efficiency, maximize revenue, and reduce clinician burnout in the healthcare industry,” said Daya Shankar, co-founder and general manager at ScribeEMR, Inc. “By harnessing the power of AWS HealthScribe, we can transform the process of healthcare documentation using generative AI. With AWS HealthScribe, our advanced processes can now capture and interpret patient visits more effectively and optimize EMR workflows, coding, and reimbursement processes.