How fast is too fast? Innovation in IVF and the burden of proof

The field of assisted reproduction, and especially the introduction of new technologies in the clinical embryology laboratory, is as thrilling as it is oftentimes frustrating.

A recent randomised controlled trial aiming to test the effectiveness of a time-lapse system and its accompanying embryo ranking algorithm on the cumulative ongoing pregnancy and live birth rates has reignited the conversation on this tool. Published earlier this year in The Lancet, this study is a welcome piece of evidence, well designed and impeccably executed, on the value of time-lapse systems in improving clinically relevant outcomes. To me, it also represents a textbook example of how technological innovations are deployed in infertility care.

The study was carried out on 1731 couples undergoing IVF/ICSI, recruited from 15 clinics in the Netherlands, and compared in a three-arm design:

• embryo selection by a ranking algorithm in a time-lapse incubator;
• classical morphological ranking in a time-lapse incubator;
• classical morphological ranking in a time-lapse incubator with interrupted culture for daily assessments.

The time-lapse Geri+ incubator was used, along with the Eeva algorithm.

To date, this is the largest randomised control trial on time-lapse monitoring, providing cumulative ongoing pregnancy and livebirth results with a significant follow-up period of one year and sufficient statistical power. The results, in the words of the authors, 'provide no evidence that the use of a time-lapse monitoring incubator in an IVF laboratory increases the cumulative ongoing pregnancy or livebirth rate compared to standard embryo culture and selection'.

While this is not the first randomised control trial to indicate similar outcomes for embryos cultured in time-lapse systems vs conventional culture, it is nevertheless especially notable for its size, complexity, power and number of recruiting sites. The effort that goes into the coordination and execution of such a study is very significant.

It is also, alas, an example of a much too common occurrence in the IVF field, where plausible, promising even, technological innovations are introduced in clinical practice before a thorough evaluation of their efficacy and safety is performed, with problematic aspects for both patients and the field as a whole.

The pedal-to-the-metal mindset leading to premature application of technology in clinical practice leaves more cautious clinics and entire public healthcare systems to scramble for reliable data on the true value of each new 'toy'. Because manufacturers/developers data and premarket testing for clinically relevant outcomes are often incomplete, it is up to publicly funded initiatives to drive independent evaluations, and this comes with a necessary lag of time.

By the time the results of these post-commercialisation randomised control trial are out, the technology may have already gone through several iterations, sometimes the models or protocol tested are not in use anymore, have been replaced by upgrades and the results are simply 'old news'.

As much as some have criticised the design for using 'old' technology, the clinical protocols used by Dorit Kieslinger and colleagues (transfer of embryos on the third day of development), and the algorithm to select them for embryo transfer were very much up to date when the study was proposed and approved years ago.

Because of the speed of introduction of new innovations in IVF, we are now faced with the uncomfortable reality of an excellent, robust, worthy study whose practical value is questioned.

While some criticism can be safely dismissed as irrelevant, it is true that today's algorithms are supported by more complex, often artificial-intelligence-buttressed decision trees, and may indeed provide better ranking performance compared to the one tested. However, the study arm with time-lapse incubator and classical morphological scoring did not perform better than the interrupted control either, all while the time-lapse incubator continuous, undisturbed culture has been heralded as 'the best' for many years.

There are also ethical aspects associated with this approach toward testing new technologies. On the one hand the burden of proof, with all its complexities, costs and time is shifted to the users, and most specifically the public healthcare systems which apply a strict (or as strict as possible) evidence-based medicine approach to their portfolio.

Further, most technologies on the market have been initially developed in public settings, universities and research centres, with the direct personal contribution of individuals, such as patients and sample donors. The research at the base of the technology is also funded by public money, public infrastructure and resources. It seems perhaps excessive that the public should also fund what amounts to market validation of a technology while it is being sold for a profit.

Finally, and most importantly, we should never forget that plausible effects and wishful thinking are not neutral. Technologies that 'should' work may in fact end up harming patients. And the effect doesn't need to be directly from drugs or other treatments to be harmful: the opportunity cost can be more than enough to fail patients. By displacing other less glamorous, but proven, techniques, we may be increasing the cost of already expensive treatments, lengthening the time to live birth and ultimately limiting the possibility for some patients to continue. The already prohibitive barriers in fertility care will continue to rise for many.

We need to welcome randomised control trial, testing and gathering first degree evidence, but please, let's generate results before a decade is spent waxing and waning on the clinical advantages of unproven developments.