Learn about MDR/IVDR risk assessments including challenges, trends, and overall risk management for medical devices.

Medical Device Risk Assessment Under ISO 14971:2019

August 1, 2023

9 mins

EU MDR EU IVDR General

Home » Resources & News » Medical Device Risk Assessment Under ISO 14971:2019

At the time of this article, it has been approximately four years since the publication of ISO 14971:2019. However, while several years have passed since its publication, most manufacturers have held off transitioning to this version of the standard until they transition their EU Technical Documentation Files to conformity with the MDR/IVDR. As the compliance deadlines for up-classified devices were extended, it is only recently that we have had an opportunity to see how manufacturers have adapted to ISO 14971:2019 regarding medical device risk assessments.

MedEnvoy provides EU Authorized Representative and EU Regulatory Importer solutions to help you navigate European regulations.

Assessing Individual and Overall Risks

We have observed some potential issues in medical device risk management in the assessment of individual and overall risk, particularly within the context of the MDR/IVDR and their respective total product life cycle regulatory frameworks.

Individual Medical Device Risk Assessment

EN ISO 14971:2019 defines ‘risk’ as the “combination of the probability of occurrence of harm and the severity of that harm” which inherently is concerned with a particular potential source of harm (i.e. hazard). For example, if we take a patient-contacting (intact skin) device that is intended for short-term use the risk that a patient experiences harm in the form of skin irritation would be determined by the combination of the degree of irritation (e.g. burning sensation (no visible sign of skin irritation), skin rash, first-degree burn, etc.) and the probability of skin irritation occurring. For this example, the manufacturer has determined the respective probabilities of occurrence based on clinical data identified and assessed within the scope of its clinical evaluation (e.g. peer-reviewed literature and/or publicly available vigilance data on equivalent/similar devices) and has also performed biocompatibility testing for the device. Applying EN ISO 14971:2019 risk analysis concepts, we would have the following three individual residual risks (RR1, RR2, and RR3) in this scenario going through the process of risk estimation:

Medical device risk assessments by type.

These residual risks subsequently undergo risk evaluation where the estimated risk (i.e. R1, R2, and R3) is compared against given risk criteria to determine acceptability. Where manufacturers apply a Failure Modes Effect Analysis, the calculated level of risk is also commonly referred to as the Risk Probability Number (RPN).

Under ISO 14971:2007/EN ISO 14971:2012, it was already a requirement that the risk management plan (RMP) include the criteria for risk acceptability based on the manufacturers policy for determining acceptable risk. This included criteria for accepting risks when the probability of occurrence of harm could not be estimated, which is not unusual in certain circumstances such as the development of novel devices. While this requirement did not distinguish between risk acceptability criteria for individual and overall risk, residual risk evaluation (Clause 6.4) and evaluation of overall residual risk acceptability (Clause 7) did both establish that these evaluations needed to be performed using the risk acceptability criteria established in the RMP.

Taking the scenario above, the manufacturer could establish a matrix utilizing the parameters of severity and probability of occurrence of harm and distinguish between different risk categories such as the following.

In the above example matrix, severity is represented by the lowest level of severity of harm (Smin) on the left up to the highest level of severity of harm (Smax) on the right, while probability of occurrence is represented by the lowest probability at the bottom of the matrix (Pmin) up to the highest probability at to top (Pmax). Manufacturers will typically break up severity and probability of occurrence of harm into several different categories or ranges and ISO/TR 24971:2019 guides the types of qualitative and quantitative categories that may be used for such purposes.

While the severity of harm is relatively straightforward to break up into different categories (e.g. scaling from ‘negligible’ up to ‘catastrophic’ levels of harm), manufacturers should invest time into determining appropriate categories of the probability of occurrence of harm. For example, in the case of reusable medical devices, the probability could be based on the probability of the harm occurring over the lifetime of an individual device, while for single-use devices the probability could be based upon the probability of the harm occurring with an individual device compared to the number of devices sold over a defined period (e.g. per annum). The manufacturer should determine the most appropriate means to estimate risk based on the characteristics of the device.

In the above example matrix, colors are used to subsequently indicate where there is an unacceptable level of risk (red), where risk is acceptable only when the benefits of the device outweigh the individual risk and the implementation of additional risk controls does not lower risk to a broadly acceptable level (yellow), where there is a broadly acceptable level of risk (green). In this instance, the manufacturer’s individual risk acceptability policy is that no individual residual risk can be unacceptable (red) while an individual risk can be in either of the categories indicated by yellow or green.

Trend Reporting and Benefit-Risk Analysis

This brings us to another reason why it is a good investment of manufacturer time in determining harm severity and probability of occurrence: MDR/IVDR post-marketing surveillance (PMS) requirements, specifically trend reporting. Under the MDR/IVDR, manufacturers shall report any statistically significant increase in the frequency or severity of incidents that are not serious incidents or that are unexpected undesirable side-effects that could have a significant impact on the benefit-risk analysis. The significance of the increase shall be established in comparison to the foreseeable frequency or severity of such incidents in respect of the device, or category or group of devices, in question during a specific period as specified in the technical documentation and product information.

In order to determine the reportability of such trends, it is therefore critical that manufacturers establish the:

- Methodology for determining the statistical significance increase in the frequency or severity of such incidents, including the methodology utilized for determining the significance of the impact on the overall benefit-risk analysis.
- Observation period during which relevant data will be collated and analyzed in accordance with the established methodology.

These details must be established in the device’s PMS plan.

The best approach to be applied for the methodology in determining the statistical significance of any increase in the frequency or severity of such incidents is to utilize the identical categories of severity and probability of occurrence used in the risk management process. This would ensure consistency in the approach applied both during design and development and PMS, meaning that risk is managed in the same manner in both phases of the device’s life cycle. Secondly, the risk categories utilized (e.g. green, yellow, and red zones in the above matrix), can be used to determine the significance of the impact on the overall benefit-risk analysis. For example, where there has been an increase in severity or probability of occurrence of harm (in the vast majority of cases, the increase will be in regards to the latter) for a specific type of non-serious incident that was identified as a broadly acceptable risk (green) previously, but the individual risk continues to be broadly acceptable after risk re-calculation, the manufacturer can determine that there is no impact on the overall benefit-risk analysis and subsequently there is no need for trend reporting. However, where risk-recalculation results in an increase to an unacceptable level of risk (red), the impact on the overall benefit-risk analysis would be sufficiently significant to warrant trend reporting (and highly likely other actions to be taken by the manufacturer).

Overall Medical Device Risk Assessment

Having covered individual medical device risk acceptability, and touching on overall benefit-risk analysis, this brings us to overall medical device risk assessment. As previously noted, under Clause 7 of ISO 14971:2007 / EN ISO 14971:2012 it was a requirement that overall residual risk also be assessed against risk acceptability criteria established in the RMP, however in our experience, this was not a requirement that was heavily enforced by Notified Bodies and it was typically sufficient for manufacturers to establish in their risk management files that where no individual residual risks were unacceptable, the overall residual risk was also deemed to be acceptable. However, such an approach is likely to undergo further Notified Body scrutiny under the MDR / IVDR, particularly as it does not take into consideration the accumulative effect that multiple “intermediate” individual residual risks (e.g. yellow risk category in the above example matrix) may have on the overall residual risk and subsequently overall benefit-risk analysis for the device.

Therefore, it’s recommended that manufacturers give more thought to their risk acceptability policy when it comes to the acceptability of overall residual risk. The following chart represents a simple way in which manufacturers may map out the respective individual residual risks in order to get an understanding of the residual risk distribution for their devices.

In the above map, the first quadrant (Q1) could represent unacceptable overall residual risk where any individual residual risk is located inside the quadrant. Both the second (Q2) and third (Q3) could represent where overall residual risk is deemed acceptable only where the number (or percentage) of individual residual risks is below an established threshold/cut-off and the benefits of the device outweigh the cumulative individual residual risks in these quadrants. The fourth quadrant (Q4) could represent where overall residual risk is deemed acceptable where any individual residual risks are located in this quadrant.

While the utilization of such an individual residual risk distribution map combined with the criteria described above represents a simplified approach, it does address the requirement to establish an overall risk acceptability policy which is essential before addressing the overall benefit-risk analysis of a device.