Good Machine Learning Practices for Medical Devices

In 2021 Health Canada, the U.S Food and Drug Administration (FDA), and the United Kingdom’s Medicines and Healthcare products Regulatory Agency (MHRA) established 10 guiding principles for good machine learning practice (GMLP).

The Guiding Principles for Good Machine Learning Practices Included:

• • Multi-Disciplinary Expertise Is Leveraged Throughout the Total Product Life Cycle
  • Good Software Engineering and Security Practices Are Implemented
  • Clinical Study Participants and Data Sets Are Representative of the Intended Patient Population
  • Training Data Sets Are Independent of Test Sets
  • Selected Reference Datasets Are Based Upon Best Available Methods
  • Model Design Is Tailored to the Available Data and Reflects the Intended Use of the Device
  • Focus Is Placed on the Performance of the Human-AI Team
  • Testing Demonstrates Device Performance during Clinically Relevant Conditions
  • Users Are Provided Clear, Essential Information
  • Deployed Models Are Monitored for Performance and Re-training Risks are Managed

These principles aim to support the development of safe, effective, and high-quality artificial intelligence/machine learning technologies that can learn from real-world use and enhance device performance. The FDA, Health Canada, and MHRA have identified guiding principles for transparency for Machine Learning-enabled Medical Devices (MLMDs). These principles build upon the GMLP principles, especially Principle 7 and Principle 9. Transparency is crucial for all medical devices and involves communicating appropriate information about an MLMD clearly to relevant audiences.

Importance Of Transparency in Medical Devices

Transparency is essential for patient-centered care and ensuring the safety and effectiveness of medical devices, especially machine learning-enabled ones.
Transparent information helps identify and evaluate risks and benefits, detect errors or performance declines, promote health equity, and support the continued safety of devices over time.

Transparency involves clearly communicating information about the device to relevant audiences. The guiding principles for transparency of MLMDs consider the following:

• • Who (Relevant audiences)
  • Why (Motivation)
  • What (Relevant information)
  • Where (Placement of information)
  • When (Timing)
  • How (Methods used to support transparency)

These constructs are discussed further in Table 1. They could also be discussed under concepts like explainability, human-centered design, and logic play a key role in promoting transparency and ensuring users have access to necessary information for safe and effective device use.

Human-Centered Design and Transparency

Human-centered design is an iterative process that considers the entire user experience and involves relevant parties in the design and development of medical devices. This approach helps in developing MLMDs with high transparency, validating transparency, and ensuring users have all necessary device-related information.

Communication of Device Information

Providing clear and accurate descriptions of medical devices, including their purpose, function, target users, environments, and intended impacts on healthcare decisions, is essential. Sharing information about device performance, benefits, risks, and risk management activities helps users make informed decisions and ensures safe and effective device use.

Placement And Timing of Information

Device information should be easily accessible through the user interface, including training, controls, display elements, and packaging. Timely communication of device updates and targeted information during specific workflow stages or triggers is important for successful transparency. Providing personalized and adaptive information through the software user interface can enhance user experience.

Optimizing User Interface for Transparency

Optimizing the software user interface of medical devices to convey personalized, adaptive, and reciprocal information can enhance transparency. Addressing user needs through various modalities such as audio, video, text, alerts, and diagrams supports successful transparency throughout the total product lifecycle.

Table 1: Summary of Transparency Guiding Principles

_____________________________________________

If you would like to discuss more with our regulatory team about the guiding principles for Good Machine Learning Practices, contact us so we can assist you. 

Learn how MedEnvoy can assist you in understanding the guiding principles for Good Machine Learning Practices:

• • US FDA Agent
  • US FDA Consulting
  • FDA Premarket Approval PMA Compilation Consulting
  • FDA 510(k) Compilation and Consulting