04/05/2024

Human-AI Interaction Models for Clinical Decision Support

exploring human-machine collaboration pathways for clinical AI systems

This piece is inspired by a part of my MPhil dissertation at University of Cambridge, in collaboration with Cyted. I worked primarily with the wonderful Charlene Tang who helped shape these ideas and models.

“Can machines think?” are the opening words of Alan Turing’s 1950 paper, ‘Computing Machinery and Intelligence’, in which he introduces The Turing Test, originally called the imitation game, to differentiate humans from machines. AI now allows for algorithm-based techniques and applications that can simulate human mental processes, enabling machines to solve problems with knowledge. 

AI is taking healthcare by storm and its use can be broadly divided into two strands (1): 

  1. Data-facing techniques – using machine learning to derive new knowledge from large datasets, improving diagnostic accuracy from images and scans 

  2. User-facing applications – intelligent agents using probabilities to interact with people in real-time, using inferences to provide advice, chatbot substituting, or complementing a healthcare consultation

As the supply of doctors diminishes (2) and diagnostics continues to get a bad rep from healthcare investors (3), clinical decision support via AI for prognostics/diagnostics remains a whitespace (4,5). It won’t be long before we will need to contemplate the integration of AI into human decision-making processes within clinical settings. 

The focus in this piece is on clinical AI systems and models for human-machine collaboration in this context. 

Everything below will be in the context of computational pathology.

Why Pathology?
Pathology is a great sandbox to illustrate human-machine collaboration for clinical AI systems because

  1. The digital to computational pipeline in pathology is arguably at a stage where the groundwork (digital infrastructure) required for computational is largely in place (6)

  2. Pathology can learn from the mistakes of radiology (7)

  3. There is growing interest in responsible and explainable AI on both the technical and policy fronts (8)

Background
In the year 2000, the scientific community agreed on the term digital pathology to denote the digitization efforts in pathology and its applications –– capturing, sharing, managing, interpreting, and analyzing the digital information from histopathology, immunohistochemistry, or cytology glass slides (9). The microscope is replaced by a slide scanner linked to a reading station where the image data sets are assessed. Thus, the pathologist no longer needs to “read” the physical slide under a microscope, but the digital images on a computer monitor. 

In digital pathology, data from these slides can then be stored in central cloud-based spaces, allowing for remote access to information for three types of application depending on digital infrastructure

  1. Manual review by a pathologist on a computer screen (from slide to monitor)

  2. Digitally analyzed using image analysis techniques (annotate/analyze on monitor in an image management system)

  3. Automated review by a data algorithm (computational pathology for clinical decision support) – this piece will be covering this one

These digitization efforts are allowing for the application of computational science to pathology by using machine learning algorithms to assist tasks by primarily utilizing data-facing techniques of AI to improve diagnostic accuracy via clinical decision support. Computational pathology includes extracting information from digitized pathology images in combination with their associated metadata, then using mathematical models to produce diagnostic inferences, presenting clinically actionable knowledge for pathologists, clinicians, or consultants (10). 

Figure 1 demonstrates the process of developing an algorithm, from the initial phase of collecting applicable samples and clinical information to creating annotated WSIs to image analysis and data training. (10)

Data: Whole Slide Images
The evaluation of pathology slides manually has not changed in over 150 years – from sample preparation to viewing slides under an optical microscope. The hematoxylin and eosin (H&E) stain combination is the gold standard and has not changed since it was first introduced in 1876 (11). The protocols in place today are well-established and trusted by pathologists, and continue to serve as the first diagnosis point for most cancers and other diseases. 

For the past 30 years, digital pathology has promised to change pathologists' workflow. The major enablers of digital pathology are WSI technologies that address many of the aforementioned issues such as features that we currently take for granted when processing digital data – annotating photographs with text descriptions or sending photos to remote specialists for a second opinion. Image acquisition scanners, display and administration software, as well as accompanying communication and storage systems, are all included in today's WSI systems (11). 

In 2015, the first research using deep learning to analyze WSI data started appearing. The FDA approved the first ever WSI system produced by Philips for primary diagnosis in 2017, with a second approval for Leica in 2019. While more are expected, it is evident that WSI systems are still in their infancy, especially because the FDA is formulating its own evaluation criteria. It also is apparent that, in order to support an AI ecosystem, digitization and WSI system adoption rates must eventually reach critical mass.

Figure 2 (timeline) illustrates a timeline of significant events in the field of digital pathology, as well as related AI applications. (11).

AI solutions in computational pathology
Commercial AI solutions for computational pathology work are broadly categorized in three focus areas – laboratory operations, clinical decision support, and research & development (11). 

Lab operations: A diagnostic digital pathology infrastructure offers workflow improvements in laboratory operations, potentially reducing costs through enhanced efficiency, increased throughput, shorter case turnaround times, and improved collaboration. A study comparing manual and digital workflow models found that digital pathology saved approximately one hour of a pathologist's time per day, particularly in routine diagnosis (12). However, despite these savings in logistics, the study suggests that additional factors may be needed for a financially neutral business case for diagnostic pathology.

Figure 3 shows the focus of this piece.

R&D: AI applications in R&D focus on developing imaging biomarkers and identifying crucial image aspects for outcomes in pathology, such as clinical trial recruitment and precision medicine (11). These R&D efforts are crucial for the growth of AI-based pathology companies and inform implementation models for computational pathology. While many startups currently focus on clinical decision support and laboratory operations, transitioning them towards R&D through partnerships with pharmaceutical companies can accelerate drug development and clinical trial designs. Diversifying R&D verticals among startups generates new value propositions for pharmaceutical firms, accelerating the adoption of translational AI in pathology. For instance, PathAI's partnership with Labcorp announced in March 2021 broadens their collaboration to deploy algorithms in Labcorp-managed clinical trials (13).

Table 1: List of research and application areas for AI and image analysis in pathology (14).

Clinical decision support: The pathologist's primary clinical responsibilities is to diagnose and characterize clinical samples. Clinical decision support applications could include classification models for identifying malignant cells, predicting prognostic features, and grading tumors depending on how distinct they are from healthy tissue around them. Early applications will likely begin by giving pathologists a second opinion, and it will be some time before they become fully autonomous. Implementation models of semi-automated triage and fully-automated review for computational pathology are explored below.

Computational pathology’s clinical deployment has been an ongoing effort in different subspecialties. By reviewing the practice of AI across diverse application areas, the majority of research is revealed to currently focus on clinical decision support, which is summarized in Table 1. The wide range of application areas demonstrated in table 1 utilize CNNs and strengthen the clinical value proposition for computational AI for clinical settings. Computational pathology has been shown to aid with diagnostic decision making (ex benign vs cancer cell), and is projected to expand to disease severity assessment and prognosis prediction (14), demonstrating how data-facing techniques of ML can revolutionize healthcare.

Human-machine collaboration in pathology
Based on the level of involvement of the pathologist, as well as, the stage at which computational pathology currently is coupled with implementation and adoption barriers that present – four models of human-machine collaboration in the digital to computational pathology pipeline for clinical decision support make sense: 

Digital Pathology

  1. Pre-screen by a non-expert – a non-clinical staff member, for example, a biomedical scientist or cytoscreener

  2. Semi-automated review – overlay of multiple sample stains and saliency map

Computational Pathology 

  1. Semi-automated triage – where preliminary interpretation/diagnosis is given by computational AI software or ML algorithm, then the result is reviewed by the pathologist 

  2. Fully-automated review – where annotation is done by the computational AI software or ML algorithm

Figure 4 demonstrates an overview of the digital pathology to computational pathology pipeline from electronic health records (EHR) and laboratory information systems (LIS) to computational pathology, allowing us to visualize different combinations of human-machine collaboration models. 

Proposed Human-AI Interaction Models
The implementation models of semi-automated triage and fully-automated review for clinical decision support also provide for a combination of human-AI interaction models, where four models for the use of AI stand out: 

  1. As a quality assurance step prior to pathologist review (H-AI-1)

  2. As a diagnostic tool for pathologists (H-AI-2)

  3. As a second opinion for clinicians (H-AI-3)

  4. As a test result for clinicians (H-AI-4)

Figure 5 demonstrates the stage at which each of the Human-AI interaction models impact the proposed implementation models of semi-automated triage and fully-automated review for clinical decision support. 

As a quality assurance step prior to pathologist review (H-AI-1)
The AI system in this model is used as a pre-screening tool to analyze medical images or data before they are reviewed by a pathologist. The AI system can identify potential abnormalities, flag areas of concern, or highlight specific features that may require further examination. This can help streamline the pathologist's workflow by prioritizing cases and directing their attention to the most critical areas, potentially improving efficiency and reducing the risk of overlooking important findings.

As a diagnostic tool for pathologists (H-AI-2)
The AI system can analyze medical images or data and provide diagnostic suggestions or probability scores for various conditions or diseases. Pathologists can then use these AI-generated insights as a reference or additional input when making their own diagnostic assessments. This collaborative approach can potentially enhance diagnostic accuracy and reduce the risk of misdiagnosis.

As a second opinion for clinicians (H-AI-3)
The AI system here acts as a virtual second opinion for clinicians. Clinicians can input patient data, medical images, or other relevant information into the AI system, which then provides its analysis and diagnostic recommendations. This can be particularly useful in cases where the clinician seeks additional insights or wants to confirm their initial assessment. The AI system's analysis can serve as a valuable reference point and potentially help identify alternative diagnostic considerations or treatment options.

As a test result for clinicians (H-AI-4)
In this model, the AI system's output is treated as a standalone test result, similar to how clinicians would interpret the results of a laboratory test or imaging study. The clinician would input the relevant patient data into the AI system, and the system would provide its analysis and diagnostic recommendations, which the clinician would then interpret and use to guide their clinical decision-making process. This approach treats the AI system as a diagnostic tool in its own right, rather than as a supplementary aid or second opinion.

It's important to note that in all of these models, the AI system's role is to assist and augment human decision-making, rather than replace it entirely. The ultimate responsibility for diagnostic and treatment decisions still lies with the clinicians and pathologists, who must critically evaluate the AI system's output in the context of their professional expertise and the patient's specific circumstances.

Each of these human-AI interaction models play a different role depending on the “human” (stakeholder) they are interacting with that cause a variety of legal, ethical, regulatory, technical, and clinical implications. 

Humans (stakeholders)
From an implementation standpoint, computational pathology requires multiple stakeholders and therefore, a team surrounding the pathologist is required from discovery and diagnosis of disease into the treatment pathway, especially for clinical decision support. The role of the pathologist can vary depending on the model adopted, as a computational pathology team includes pathologists, data scientists, engineers, and clinicians. The pathologists establish clinically relevant issues, data scientists develop and train algorithms, engineers support the operating environment, and clinicians develop the treatment plan with the patient. 

Figure 6 shows the various stakeholders in a computational pathology developer team demonstrating pathologists playing a pivotal role in applying and monitoring the algorithm in real clinical deployment for constant iterative optimization. Stakeholders in blue represent “front-end” with direct patient contact. Stakeholders in red represent “back-end” without direct patient contact, however they still play a major role in the pipeline. 

Figure 7; Perderson et al. show the types of prognostic and diagnostic clinical questions in neurology can be answered, especially with multimodal data (16).

Multimodal data
Research suggests that integrating multiple data modalities into a single AI model can enhance performance and predictive accuracy (15). While the challenge of integrating data with differing dimensions, time scales, and scopes persists, progress is evident. Ensemble methods, leveraging collections of separately learned models, consistently outperform single monolithic models, highlighting ongoing advancements in this field.

The more data we have, the more complicated clinical questions we can answer. In the following figure, Perderson et al. show the types of prognostic and diagnostic clinical questions in neurology can be answered, especially with multimodal data (16). Oncotype DX is a great example of deep learning-based pathology image analysis for breast recurrence score prediction (17).

Perspective
The future of clinical AI systems in the context of human-machine collaboration in healthcare depends on fostering a symbiotic relationship between human domain-specific expertise and machine intelligence fine-tuned also by human experts. While AI algorithms can extract patterns and make predictions from large datasets, their outputs must be carefully interpreted and translated into clinical practice by human domain-experts (18). AI, for now, should be viewed as an assistive technology that augments and enhances human decision-making capabilities, not as an autonomous system operating independently. 

By combining the respective strengths of human practitioners and AI models, we can unlock the full transformative potential of this technology in improving patient outcomes and delivering higher quality, more efficient healthcare services. However, this will require a mindset shift towards embracing augmented intelligence, where humans and machines collaborate in symbiosis, with ultimate authority over key decisions residing with human experts guided by professional ethics and a deeper understanding of real-world contexts.

I write white papers, primers, think-pieces, reflective essays + send personal updates in my newsletter. Feel free to check out other technical pieces and essays.
Please subscribe!