Description
The National Heart, Lung, and Blood Institute (NHLBI) convened a workshop of diverse experts in the field of “Artificial Intelligence in Cardiovascular Imaging: Translating Science to Patient Care.” The workshop was conducted virtually on June 27th and 28th, 2022 to examine the opportunities and gaps that relate to research and clinical implementation of artificial intelligence (AI) in cardiovascular imaging and in cardiovascular practice. The workshop focused on how the National Institutes of Health (NIH) and other government stakeholders can support research and development of AI with cardiovascular imaging; how the field can translate promising proofs of concept to robust, generalizable, equitable, scalable, and clinically implementable AI algorithms; and how best to support researchers and their needs.
The workshop included two keynote addresses highlighting current challenges to AI advancement, as well as current success stories of AI in the research and clinical settings. Overviews of AI use in different cardiovascular applications were presented in lightning talk format, and additional conversations were incorporated into multiple break-out sessions and panel discussions, providing the opportunity for participants to exchange ideas across the multi-faceted expertise that was represented within the workshop.
The workshop is responsive to NHLBI Strategic Vision Objectives 5-8.
OVERARCHING GOALS OF THE WORKSHOP:
- Build consensus around, and prioritize, researchers’ challenges and needs in six key areas:
- Data
- Algorithms
- Infrastructure
- Regulatory considerations
- Implementation
- Human Capital
- Generate conversation across the diverse expertise represented in order to brainstorm solutions, share lessons learned, and discuss opportunities to improve forward progress
- Establish/strengthen connections across invested groups and create new opportunities for collaborations
BACKGROUND
AI can improve efficiency, reduce variability, and improve diagnoses associated with cardiovascular imaging. However, in spite of considerable technological innovations and developments, clinical use of AI algorithms is limited.
Many challenges, including technical difficulties, impede clinical implementation. Imaging data sets used to train AI models often do not include data from some underrepresented patient groups. Also, development and implementation of AI for cardiovascular imaging requires a multidisciplinary team of researchers, clinicians, engineers, programs, U.S. Food and Drug Administration (FDA) regulators, medical insurance experts, and others. NHLBI’s research portfolio currently supports a small selection of research projects on AI and/or machine learning (ML) with cardiovascular imaging, and trends indicate a surge of research on this topic over the past 2-3 years. NHLBI’s HeartShare and other programs will help provide access to data, promote data sharing, and identify subtypes and endotypes across cardiovascular conditions. Input from conference attendees will inform investigator-initiated, peer-reviewed projects and, potentially, a white paper as well.
DISCUSSIONS
Presentations and discussion focused upon an overview of AI use in an array of cardiovascular diagnostic imaging studies and electrocardiography; and data-related, algorithm-related, and technical infrastructure-related challenges. A diverse representation of available data platforms were reviewed with examples of successful use of AI applications in registries. FDA partners presented regulatory perspectives in AI development, and translational and implementation challenges related to user education of AI, payor issues, AI expertise in the review process, and societal support of AI initiatives were discussed in a large panel format.
GAPS and OPPORTUNITIES: Workshop participants identified gaps and opportunities in the following areas to guide future work.
- Data
- Access/Sharing
- Improving broad available data access
- Balancing data access with security by considering different levels of permissions required depending on intended use (e.g., academic versus commercial)
- Enforcing responsible data use
- Determining what research and development is required in this area (e.g., through community and societal consensus)
- Developing standardization of data sharing policies for all hospitals and institutions to use
- Curation
- Defining what constitutes curation: inclusion of labeling and metadata
- Identifying and standardizing the curation level that should accompany datasets with considerations for harmonization
- Creating a universal format for metadata collection
- Prospectively planning for missing data and managing missing data that might compromise data integrity
- Quality and Diversity of Data
- Creating standards for training and testing data: determining what constitutes “quality” for datasets and using test sets reflective of the “real world”
- Confirming repeatability is another key aspect of quality
- Identifying appropriate measures of “diversity” for datasets and ensuring directed collection of under-represented communities
- Selecting who should be responsible for vetting quality
- Access/Sharing
- Algorithms
- Development, Explainability, and Transparency (allowing non-algorithm developers to understand the algorithms at a basic level)
- Creating minimum universal requirements for explainable AI
- Breaking down classification into a set of anatomic landmark detection and segmentation tasks
- Interpreting AI performance
- Updating models when they fail on new data
- Creating challenges to stimulate and support algorithm development
- Validation and Testing
- Comparing algorithms and results by external validation with consideration for multi-center validation
- Considering different methods of learning, e.g., federated learning versus pool-data learning
- Appropriately designing prospective, randomized trials of AI
- Acquisition and Reconstruction
- Optimally managing image artifacts
- Critically evaluating the trade-offs of improving signal-to-noise versus losing spatial/temporal resolution
- Evaluating performance in the absence of the reference (can this be done)
- Using AI-enabled acquisition in commercial products and considering what defines clinical viability
- Development, Explainability, and Transparency (allowing non-algorithm developers to understand the algorithms at a basic level)
- Infrastructure/Available Support Platforms
- Managing the rapidly evolving nature of technology with the changing requirements of new services, software, standards, and security; and the immaturity of the systems, many of which are still being designed and tested
- Dealing with interoperability with different medical devices and machine learning platforms using different formats and managing compatibility issues
- Maintaining constant vigilance and adaptation with close oversight of data management
- Regulatory
- Fostering early communication between AI developers and regulatory bodies
- Using regulatory approaches to streamline and standardize AI algorithm development
- Considering alternative approaches to regulatory engagement including pre-certification of a given manufacturer’s process rather than the product
- Clearly understanding the required regulatory steps in order to gain FDA approval and understanding that regulatory approval does not define “clinical success” of a given application
- Monitoring AI applications after approval
- Managing “off-label” use of AI products and concerns for mis-interpretation of AI abilities
- Translation and Implementation
- Payor concerns
- De-mystifying the “black box” process of AI development for regulatory and payor bodies to more clearly understand the steps between development and implementation
- Fostering better communication between AI developers, AI users, and payors
- Creating equitable means of reimbursement for the use of AI
- Defining transparent, justifiable criteria by which AI methodologies may be financially covered by more than a hospital system
- Identifying and creating standards by which AI applications might undergo approval at the hospital level for use in patient care
- Educational and academic barriers
- Offering more broad opportunities for all medical professionals to become familiar and comfortable with AI technology
- Ensuring appropriate expertise is available for review of AI-related publications and research applications
- Avoiding diversion of AI research publications into lower tier niche journals
- Societal roles and responsibilities
- Linking individual societal data in a complementary fashion into a larger registry in order to amplify the significance of the data available within each society
- Removing the barriers to site contribution to registry (streamlining the process and minimizing the administrative burden)
- Developing standards for data mining capabilities
- Improving data access to the public
- Mitigating adverse and mistrustful attitudes toward AI
- Payor concerns
PUBLICATION PLANS:
Post-workshop conversations continue, and a white paper summarizing the gaps and opportunities that were identified at the workshop, as well as action items for progress, is in preparation.
WORKSHOP PARTICIPANTS
Co-Chairs
Rima Arnaout, M.D., University of California San Francisco (UCSF)
Damini Dey, Ph.D., Cedars Sinai Medical Center
NHLBI Co-Leads
W. Patricia Bandettini, M.D.
Vandana Sachdev, M.D.
Additional Program Planning Contacts
John Haller, Ph.D., NHLBI
Lucy Hsu, Ph.D., NHLBI
Erin Iturriaga, D.N.P., M.S.N., R.N., NHLBI
Krishna Kandarpa, M.D., Ph.D., National Institute of Biomedical Imaging and Bioengineering (NIBIB)
Sweta Ladwa, Ph.D., NHLBI
Huiqing Li, Ph.D., NHLBI
James Luo, Ph.D., NHLBI
Wendy Nilsen, Ph.D., National Science Foundation
Sujata M. Shanbhag, M.D., M.P.H., NHLBI
Speakers and Moderators
Anas Abidin, Ph.D., NVIDIA
Theodore Abraham, M.D., UCSF American Society of Echocardiography
Sameer Antani, Ph.D., National Library of Medicine,
Charles Apgar, American College of Radiology (ACR)
Rima Arnaout, M.D., UCSF
Aldo Badano, Ph.D., Food and Drug Administration (FDA)
W. Patricia Bandettini, M.D., NHLBI
Lew Berman, Ph.D., M.S., NIH/Office of the Director
Ami Bhatt, M.D., American College of Cardiology (ACC)
Davis Bitton, M.S., Ambra Health
Stephen Browning, FDA
Anthony Chang, M.D., American Board of Artificial Intelligence in Medicine (ABAIM)
Melissa Chen, M.D., Anderson Cancer Center
Andy Crowne, Google Cloud Platform
Damini Dey, Ph.D., Cedars Sinai
Dennis Dean, Ph.D., Seven Bridges
Aneesh Deoras, M.S.E., FDA
Anne Docimo, M.D., M.B.A., UnitedHealth Group
Shawn Forrest, M.S., FDA
Steve Fu, AWS
Maryellen Giger, Ph.D., University of Chicago
David C. Goff, Jr., M.D., Ph.D., NHLBI
Li-Yueh Hsu, D.Sc., NIH Clinical Center Radiology and Imaging Sciences
Louis Jacques, M.D., ADVI, formerly of Centers for Medicare & Medicaid Services (CMS)
Jessica Lamb, Ph.D., FDA
Tim Leiner, M.D., Ph.D., Mayo Clinic
Huiqing Li, M.D., Ph.D., NHLBI
Joao Lima, M.D., Johns Hopkins University
Michael Lu, M.D., M.P.H., Massachusetts General Hospital
James Luo, Ph.D., NHLBI
Edward Margerrison, Ph.D., FDA
Manish Motwani, M.B., Ch.B., Ph.D., Manchester University, National Health Service
Peter Noseworthy, M.D., Mayo Clinic
Elaine Nsoesie, Ph.D., NIH Office of Data Science Strategy
David Ouyang, M.D., Cedars Sinai Medical Center
Steffen Petersen, M.D., Ph.D., Queen Mary University of London, Centre for Advanced Cardiovascular Imaging
Allison Pompey, CMS
Pranav Rajpurkar, Ph.D., Harvard Medical School
Rajesh Ranganath, Ph.D., New York University
R. Todd Reilly, Ph.D., NIH/Center for Information Technology, NIH STRIDES Initiative
Michael Rice, Microsoft
Travis Richardson, Flywheel
Christine Rutan, American Heart Association (AHA)
Vandana Sachdev, M.D., NHLBI
Ravi Samala, Ph.D., FDA
Partho Sengupta, M.D., Rutgers Robert Wood Johnson School of Medicine
Sanjiv Shah, M.D., Northwestern University
Sujata M. Shanbhag, M.D., M.P.H., NHLBI
Orlando Simonetti, Ph.D., Society for Cardiovascular Magnetic Resonance (SCMR)
Piotr Slomka, Ph.D., Cedars Sinai
Charles Stemple, M.D., M.B.A., formerly of Humana
Alastair Thomson, NHLBI
Natalia Trayanova, Ph.D., M.S., Johns Hopkins University
Wes Wiggins, M.S., AHA
Michelle Williams, M.B.Ch.B., Ph.D., B.Sc., University of Edinburgh
Pamela Woodard, M.D., ACR
Hui Xue, Ph.D., NHLBI