Trend Reporting Under the IVDR: Device Performance Issues

One of several changes introduced in the transition from the In Vitro Diagnostic Medical Device Directive (IVDD) to the In Vitro Diagnostic Regulation (IVDR) that impacts both legacy devices and those CE marked under the IVDR is trend reporting. As established under Article 83 of the IVDR, manufacturers are obligated to report any statistically significant increase in the frequency or severity of incidents that are not serious incidents, but that could have a significant impact on the benefit-risk analysis.

As device performance plays a key role in the benefit-risk analysis of IVDs, manufacturers should consider the types of device performance issues that fall within the scope of trend reporting under the IVDR requirements to ensure ongoing compliance with post-market surveillance (PMS) and, as applicable, post-market performance follow-up (PMPF) requirements. In this article we cover such considerations, using an example IVD device scenario.

Serious vs. non-serious incidents

Given that IVDs are solely or primarily intended to provide information concerning one or more of the purposes established under Article 2(2) and (in the case of professional use IVDs) have minimal direct patient interaction, the overall tendency is that incidents are more likely to be non-serious rather than serious. However, to appropriately distinguish between non-serious and serious incidents, manufacturers must have a strong understanding of the types of incidents considered to be serious under the IVDR. Under Article 2(67), any one of the following must have occurred for an event to be considered an ‘incident’: 

• • Device malfunction 
  • Deterioration in device characteristics or performance 
  • Use-error due to ergonomic features 
  • Inadequacy in the information supplied by the manufacturer 
  • Harm as a consequence of a medical decision, action taken or not taken based on information or result(s) provided by the device 

Once fulfilling any one of the above-listed pre-conditions, incidents are considered a ‘serious incident’ per Article 2(68) when they directly or indirectly led, might have led or might lead to any of the following: 

• • Death of a patient, user, or other person 
  • Temporary or permanent serious deterioration of a patient’s, users, or other person’s state of health 
  • Serious public health threat 

Incorrect in vitro examination results are the most critical hazards for manufacturers to identify and control their devices. In the case of semi-quantitative and quantitative devices, results are incorrect if the difference from a correct value exceeds the error limit required for the clinical application. Where medical decisions may be influenced by the magnitude of the result, the clinical significance of an incorrect result can depend on the magnitude of the difference between the measured value and the true value. For qualitative devices, examination results are either correct (true positive/negative), incorrect (false positive/negative), or indeterminate. 

However, in the case of qualitative devices, particularly those used for disease screening, as we discuss below, the occurrence of false positive or false negative results may not necessarily constitute a serious incident and therefore the monitoring of such events should be flagged by manufacturers for trend analysis under the IVDR (and reporting, as necessary). 

As discussed in EN ISO 14971, hazards and hazardous situations may exist for devices in both normal and fault conditions, and false positive/negative results for qualitative screening devices are no exception. For this article, we’ll use a fecal immunochemical test (FIT) as an example. 

FITs are IVDs that detect hemoglobin in blood on feces (or in toilet bowl water) and are typically used for colorectal cancer screening. In this example, we’ll consider a FIT with clinical/diagnostic sensitivity of 87% and clinical/diagnostic specificity of 99%. This means that, for this FIT, 87% of patients with colorectal cancer obtain a true positive result, 13% of patients with colorectal cancer obtain a false negative result, while 99% of patients without colorectal cancer obtain a true negative result, and 1% of patients without colorectal cancer obtain a false positive result (and are typically therefore flagged for follow-up colonoscopy). 

In this example, false positive/negative results are therefore possible in a normal condition (i.e. when the device performs as intended, false positive/negative results are expected) and may be attributed to root causes such as: 

• • Colorectal bleeding from non-cancerous source 
  • Polyp/Tumor intermittent/non-continuous bleeding 
  • Non-bleeding polyp/tumor 

However, false positive/negative results are also possible in fault conditions, particularly where manufacturing errors have occurred such as variability in the striping of nitrocellulose membranes with test line antibody solutions of conjugate solutions. Subsequently, there are several challenges faced by manufacturers of such devices when investigating and determining the root cause of false positive/negative results, including: 

• • Single use nature of the device with established test development time limits/conditions (i.e. overdevelopment of the test beyond the intended interpretation time is itself a potential cause of false positive test results) 
  • Test disposal following immediate use by laboratory biohazard waste management controls (i.e. the specific test device cannot be supplied for investigation and is likely contaminated if available) 
  • The timeframe between test result interpretation, patient test result communication, and follow-up screening/testing (i.e. several months or years may pass between an individual testing event and follow-up screening or testing) 
  • Patient confidentiality (i.e. healthcare professionals or facilities may be reluctant to provide patient details relevant to the investigation, even when the data is anonymized, by their policies and procedures)

Therefore, FIT manufacturers typically resort to the following means to investigate false positive/negative complaints: 

• • Testing of retained samples from the same manufactured lot 
  • Follow-up with user facilities (e.g. laboratories) that received the same manufactured lot to verify the existence of any issues regarding false positives/negatives 

Where the manufacturer definitively identifies a fault condition as the root cause, including use-error due to ergonomic characteristics, this falls within the definition of an ‘incident’ and the manufacturer must subsequently determine whether it also represents a ‘serious incident’ that must be reported or is non-serious and is subject to reporting under trend reporting obligations. However, where manufacturer rules out a fault condition as the root cause, they should not necessarily believe that they are free of trend reporting requirements.

Normal device conditions with false positive and negative results

Where the manufacturer (to the best of its ability) has ruled out a fault condition cause of a complaint related to false positives/negatives, to determine whether the complaint is an ‘incident’ it still needs to determine whether there has been: 

• • Harm as a consequence of a medical decision, action taken or not taken based on information or result(s) provided by the device 
  • Deterioration in device characteristics or performance 

While the first of these can be verified via the details of the complaint and follow-up with the complainant, the deterioration in device characteristics or performance has a direct relationship with IVDR trend analysis (and reporting) and would require further investigation. 

As stated above, the clinical/diagnostic sensitivity and specificity are among the clinical performance characteristics of such devices and are included in the information provided to the user in the IFU. However, these characteristics are established through the realization of highly controlled clinical performance studies in which colonoscopy results are available to the investigators for every study participant – in contrast to real-world usage of the device. Manufacturers of such devices will typically be unable to obtain data on follow-up patient testing/screening results in general that could be compared to the original clinical performance studies. However, one piece of information that may be able to be obtained from the market is ‘positivity rates’, i.e. the number of positive tests reported vs. number of tests developed, over a specific timeframe and for specific healthcare facilities or customers.

Even in a normal device condition, positivity rates may be influenced by several external factors, including demographic/risk factors in the testing population. For example, regions where there is a higher than national average age in the population and a low initial screening rate, would likely have a higher FIT positivity rate than regions with younger average populations and high/strong screening rates. As such, the utilization of high-quality PMPF studies/surveys to establish baseline positivity rate trends in the testing population is a tool available to manufacturers to perform ongoing trend analysis and comply with trend reporting requirements under the IVDR. 

Learn more about trend reporting under the IVDR with MedEnvoy

The above example is just one such scenario demonstrating device performance considerations for trend reporting under the IVDR. If you have any questions regarding PMS or PMPF or require relevant training or consulting services, get in touch. 

Related articles

• • IVDD vs IVDR: Greatest Impacts on IVD Manufacturers
  • Leveraging Device Clinical Data in Great Britain for IVDR Performance Evaluations
  • What is the Difference Between PMS and PMPF Under the IVDR?