Ryan B. Appleby
DVM, DACVR
Dr. Appleby is a boarded radiologist and faculty member at the Ontario Veterinary College. His research focuses on the use of artificial intelligence in veterinary imaging. He is the 2023 chair of the ACVR/ECVDI Artificial Intelligence Education and Development Committee. Dr. Appleby is passionate about education and technology and is a cofounder of Obi Veterinary Education, a website dedicated to delivering education in short, manageable segments.
Read Articles Written by Ryan B. ApplebyOver the past year, I have given several talks on artificial intelligence (AI) in veterinary medicine. At the beginning of each lecture, I begin with a slide outlining my own views on AI so that the audience can understand where I stand and the lens I bring to the presentation. Numerous times, I have stated loudly and honestly to rooms of veterinary professionals that AI is a transformative technology that will reshape our profession, and that the current iteration of AI is at its best an underwhelming collection of products with vast unmet potential and at its worst is the unethical endpoint of unbridled capitalism.
This line in my presentation is meant to draw the attention of listeners, and now you, the reader, but it is also not hyperbole. Current manifestations of AI in veterinary medicine are poor examples of what this technology can be and seem to be misguided by the desire to make a quick buck (or millions of bucks) rather than what is best for the profession and the people and patients we serve.
There are 2 main challenges with the current dialogue around AI in our profession. The first is a lack of nuance around discussing AI and the distinct types of AI solutions on the market. AI is not a monolith, and different use cases have different underlying technologies. Each technology has different risks and should be approached in different ways. The second is the lack of transparency around AI tools. Without transparency, it is impossible for veterinarians to make informed decisions and for clients to give informed consent on the use of AI tools.
Classification of AI
The concept of classification of AI systems in health care is used within the regulatory frameworks for AI in human health. AI is considered software as a medical device (SaMD) and is categorized based on its impact on (or risk to) patients. The International Medical Devices Regulators Forum classifies SaMD into 4 categories (I through IV),1 depending on the significance of the information it provides and the healthcare situation (TABLE 1).
- Category I encompasses software that informs clinical management without urgent implications.
- Category II includes tools assisting with nonserious conditions.
- Category III covers software aiding in significant, though not immediately life-threatening, conditions.
- Category IV includes software that delivers critical information for treating or diagnosing severe conditions.
This framework has been adopted by national regulatory bodies such as the U.S. Food and Drug Administration (FDA), Health Canada, and the U.K. Medicines and Healthcare Products Regulatory Agency (MHRA). While we do not have the same regulatory frameworks in veterinary medicine, we can borrow from this model to understand how distinct types of AI systems might impact patients and clients and stratify them by risk.
As the FDA uses 3 categories rather than 4, I suggest considering AI in low, medium, and high impact.
Low-Impact AI Systems
Low-impact (Class I) AI systems include those that improve efficiency and workflow, such as scheduling, billing, and inventory management. These systems do not directly impact patient outcomes and can be easily adopted by the veterinary team without much oversight or the need for guidance from institutions.
They are not entirely benign as any downstream effects of over- or under-scheduling or inappropriate inventory management or billing can have impacts on patient care. However, with appropriate human oversight by practice managers, client services teams, or technical staff, such systems can reduce workload when implemented effectively. Low-impact systems may also include those that aggregate data for monitoring of low-impact conditions, such as weight loss or exercise monitoring apps.
Medium-Impact AI Systems
Medium-impact (Class II) AI systems are those that provide medical record documentation or assist veterinarians in decision making, such as clinical calculators, symptom checkers, or triage tools. These systems provide recommendations or suggestions based on data and algorithms, but they do not replace human judgment or responsibility.
High-Impact AI Systems
High-impact (Class III) AI systems are those that perform diagnostics, such as pathology detection, radiography interpretation, and blood test analysis. These systems provide diagnoses or predictions based on data and algorithms and may have a significant impact on patient outcomes and treatment plans. Veterinarians should use these systems with caution and only after validating their performance and quality against established standards and benchmarks. To date, there is limited information on the performance of the currently available systems and no clinical validation.2 By using these products, veterinarians are making decisions to skip over the validation phase of these technologies in favor of their early implementation. This poses a risk to patient safety.
Regardless of the classification, all AI systems require more research and clinical validation. Internal performance metrics do not equate to clinical relevance. This is especially pertinent in high-impact systems where improvements in clinical outcome should drive adoption.
Transparency
Alongside clinical validation of AI systems, veterinarians require more transparency to be able to make informed decisions on when to use AI systems. The information required to be able to understand an AI system and where it can be applied to practice has been described by the U.S. FDA, Health Canada, and the U.K. MHRA through the guiding principles for Good Machine Learning Practice (GMLP) for medical devices3 and Transparency for Machine Learning–Enabled Medical Devices (MLMDs).4 The more recently published guiding principles for transparency in MLMDs emphasize several key aspects of the GMLP to ensure patient-centered care and device safety. Transparency should target relevant audiences, including healthcare professionals, patients, caregivers, support staff, administrators, and governing bodies. The motivation behind this transparency is to facilitate the understanding of complex device information and in turn help users make informed decisions, control risks, and detect errors or performance issues. The goal is to foster trust and confidence in the technology.
The information shared should detail the device’s medical purpose, conditions addressed, intended users, and integration into healthcare workflows. It should also cover device performance, benefits, risks, and bias management strategies. Providing insights into the logic of the model and details about product development and life cycle management is essential. Transparency includes addressing known biases, limitations, and gaps in data to support critical assessment and safe usage.
Information should be easily accessible through the user interface, which includes training materials, controls, displays, packaging, labeling, and alarms. The software user interface should be designed to offer personalized and adaptive information through various modalities, such as audio, video, text, and alerts. Transparency must be maintained throughout the product life cycle, with timely notifications about updates or new information and targeted guidance during high-risk steps or specific triggers.
None of the current commercially available AI solutions in any classification offer this degree of transparency. This is often shrouded behind a claim of proprietary information. However, if medical devices in human health, which carry a far greater monetary value and earning potential given the available market, can be required to offer this degree of transparency, there is no reason a veterinary product cannot do the same while still protecting the company’s business interest.
Without such transparency, it is impossible for a veterinarian to make an informed decision on when to use a product, and some have argued that it would be impossible to have informed consent to use such a product.5
I want to see AI succeed in veterinary medicine and support practitioners in improving patient health outcomes, as well as allow a better quality of life for veterinarians. For this to happen, we need better transparency from companies providing AI solutions.
References
- IMDRF Software as a Medical Device (SaMD) Working Group. “Software as a medical device”: possible framework for risk categorization and corresponding considerations. International Medical Device Regulators Forum. September 18, 2014. Accessed August 23, 2024. https://www.imdrf.org/documents/software-medical-device-possible-framework-risk-categorization-and-corresponding-considerations
- Appleby RB, Basran PS. Artificial intelligence in diagnostic imaging. Adv Small Anim Care. Published online July 19, 2024. https://doi.org/10.1016/j.yasa.2024.06.005
- United States Food and Drug Administration. Good machine learning practice for medical device development: guiding principles. October 27, 2021. Accessed August 23, 2024. https://www.fda.gov/medical-devices/software-medical-device-samd/good-machine-learning-practice-medical-device-development-guiding-principles
- United States Food and Drug Administration. Transparency for machine learning-enabled medical devices: guiding principles. June 13, 2024. Accessed August 23, 2024. https://www.fda.gov/medical-devices/software-medical-device-samd/transparency-machine-learning-enabled-medical-devices-guiding-principles
- Kiseleva A, Kotzinos D, De Hert P. Transparency of AI in healthcare as a multilayered system of accountabilities: between legal requirements and technical limitations. Front Artif Intell. 2022;5:879603. doi:10.3389/frai.2022.879603