Microsoft rolls out new tools to build healthcare AI agents

Oct 29, 2024

Microsoft recently unveiled several new artificial intelligence capabilities for healthcare organizations designed to help analyze diverse medical data, streamline nursing documentation and enable companies to build their own healthcare AI agents.

As the tech giant moves to take a leading position in healthcare AI, the company announced, through its Microsoft Cloud for Healthcare, new healthcare AI models in Azure AI Studio, capabilities for healthcare data solutions in Microsoft Fabric, a healthcare agent service in Copilot Studio and an AI-driven nursing workflow solution.

We are at an inflection point where AI breakthroughs are fundamentally changing the way we work and live,” said Joe Petro, corporate vice president, Healthcare and Life Sciences Solutions and Platforms at Microsoft in a statement. “Across the broader healthcare and life sciences industry, these advancements are dramatically enhancing patient care and also rekindling the joy of practicing medicine for clinicians. Microsoft’s AI-powered solutions are helping lead these efforts by streamlining workflows, improving data integration, and utilizing AI to deliver better outcomes for healthcare professionals, researchers and scientists, payors, providers, medtech developers, and ultimately the patients they all serve.”

The tech company developed foundation models for medical imaging available in its Azure AI model catalog. Developed in collaboration with partners like Providence and Paige.ai, these models enable healthcare organizations to integrate and analyze diverse data types—ranging from medical imaging to genomics and clinical records, according to Microsoft in a press release.

By using these advanced models as a foundation, healthcare organizations can build, fine-tune and deploy AI solutions tailored to their specific needs, all while minimizing the extensive compute and data requirements typically associated with building multimodal models from scratch, Microsoft executives said.

“The development of foundational AI models in pathology and medical imaging is expected to drive significant advancements in cancer research and diagnostics,” said Carlo Bifulco, MD, chief medical officer of Providence Genomics and a co-author of the Prov-GigaPath study in a statement. “These models can complement human expertise by providing insights beyond traditional visual interpretation and, as we move toward a more integrated, multimodal approach, will reshape the future of medicine.”

The tech giant also sees the potential for generative AI to help address healthcare workforce shortages and meet patient demands by automating administrative tasks, analyzing vast amounts of data for actionable insights and assisting healthcare professionals in decision-making.

Microsoft developed a healthcare agent service in Copilot Studio to enable healthcare organizations to create AI tools with pre-built templates and data sources to tackle tasks like appointment scheduling, clinical trial matching and patient triaging. The service is currently in public preview.

Cleveland Clinic is one early adopter and provided feedback to help optimize the solution for a healthcare setting, Microsoft executives said.

The company also is deepening its partnership with electronic health record (EHR) giant Epic to use AI to ease the workload of paperwork for nurses.

Microsoft and Epic are collaborating with several hospitals and health systems, including Advocate Health, Baptist Health of Northeast Florida, Duke Health, Intermountain Health Saint Joseph Hospital, Mercy, Northwestern Medicine, Stanford Health Care, and Tampa General Hospital, to build an AI solution using ambient technology to streamline nursing documentation. The AI tool drafts flowsheets for review.

“AI is transforming nursing workflows by streamlining administrative tasks, allowing nurses to focus more on patient care,” said Corey Miller, vice president of R&D at Epic, in a statement. “Together with Microsoft, we’re using AI-powered ambient voice technology to populate patient assessments. Nurses using the tool are already sharing positive feedback on how it enhances personalized patient interactions.”

“For nurses, the integration of AI-driven solutions into our workflows is a game changer,” said Terry McDonnell, senior vice president and chief nurse executive, Duke University Health System, vice dean for Clinical Affairs, Duke University School of Nursing, Duke Health. “It allows us to focus more on patient care rather than the administrative burden of documentation. By automating tedious tasks, Microsoft’s ambient AI solution helps alleviate burnout and gives us more time to connect with our patients at the bedside, where we truly make a difference."

Microsoft is also launching new capabilities in public preview within healthcare data solutions in Microsoft Fabric. These new tools can tackle conversational data integration, ingest and harmonize social determinants of health datasets and perform care management analytics.

Healthcare organizations can send conversational data, such as patient conversations, from DAX Copilot to the Fabric platform. By sending DAX Copilot audio files, transcripts and draft clinical notes to Fabric, customers and partners can leverage various native tools in Azure and Fabric to analyze this data and combine it with other data to generate comprehensive insights.

Another capability enables healthcare organizations to streamline the ingestion of Centers for Medicare and Medicaid Services (CMS) claims data and harmonize it with clinical, imaging and SDOH data.