Trials and Tribulations: How Can We Build an Evidence Base?

This was the second session of the Progress Educational Trust annual conference 'Reality Check: A Realistic Look at Assisted Reproduction'. Chaired by Professor Peter Braude, emeritus professor of obstetrics and gynaecology at King's College, 'Trials and tribulations: how can we build an evidence base?' focused on the research needs of the fertility sector.

The first speaker was Dr James Duffy, the National Medical Director's Clinical Fellow at NHS England and NHS Improvement who gave a talk called 'Putting trials on trial: answering the demand for high-quality infertility research by developing a shared research agenda.' He argued that there is an unmet need for high-quality infertility research that can inform decisions in clinical practice to improve outcomes, and that increasing marketisation in the infertility sector has resulted in an era of unreliable systematic reviews.

Dr Duffy summarised results from studies about the use of evidence in obstetric patient care that he completed over the past 15 years with Cochrane UK as well as a recent priority-setting partnership. The studies pointed overwhelmingly to the need for new kinds of evidence that are more attuned to patient needs.

He argued that the fertility sector needs to repurpose its resources to deliver a priority-based, shared national and international research agenda, co-produced by health care professionals, people with infertility and others.

He drew five key conclusions in his talk: first, there is a significant gap between the research priorities identified in his studies and research performed in the field. Second, given the overwhelming and uninformative data currently available, we need fewer but better studies into the efficacy of treatment. Third, diverse research methodologies will be required. Fourth, variation in populations, outcomes and measures needs to be addressed. Finally, new randomised trial methodologies are needed.

The second speaker was Professor Nick Macklon, medical director of the London Women's Clinic and professor of obstetrics and gynaecology at the University of Copenhagen and at Zealand University Hospital in Denmark. His talk was titled 'Should randomised controlled trials be the sole arbiter of good clinical practice in IVF?'

Professor Macklon has been involved in randomised control trials (RCTs) throughout his career. RCTs are broadly considered the 'gold standard' standard method for medical trials, in which participants are divided into groups to compare different treatments. He argued that new methods were needed to address the gap in research to advise clinicians appropriately.

He conceptualised the current lack of appropriate evidence as a vicious cycle: a poor evidence-base means that research is lacking in key research areas, such as better understanding implantation. Thus, patient demand in the fertility sector is instead met by 'add-on' procedures, causing reputational damage to the field, making it harder to get research funding and perpetuating the sense that there is insufficient research and making clinicians ill-equipped to advise patients.

Professor Macklon suggested that a new metric is needed that measures 'effectiveness' (whether a technique works in practice) rather than what he called 'efficacy' (which he says only measures whether the technique could potentially work).

To address this, Professor Macklon called for a shift from a current preoccupation with evidence-based medicine, an approach that prioritises the use of explicit, judicious evidence in making decisions about individual patients, to a more patient-tailored medicine that designs studies to factor in patients' different contexts; combining scientific evidence with patients' values and preferences.

The clear need for a better evidence base expressed by many participants at the conference should be seen as an opportunity for the HFEA to move the field forward, he said. This is a chance to make prospective data collection mandatory, to cover many thousands of IVF cycles rather than small randomised groups at individual clinics; to use real-world data in context and to monitor what is being done.

The challenges will be what to measure, the resources needed, how to perform such complex analysis, and dealing with resistance from influential figures in the evidence-based medicine industry.

In the discussion with the audience that followed, the two main challenges to developing a new strategy identified were data access and what constitutes evidence.

Currently, data gathered at the bedside is coded in different ways, and translated from different languages, making interpretation difficult. Attendees said that clinics, regulators and the NHS should work to improve their databases, providing raw data for randomised cohorts of prospectively collected data.

Part of this would be deciding on a scale for future studies and a common infrastructure for data collection, as well as deciding which patients to include and how to characterise them. The biggest obstacles to this were lack of cooperation in the infertility sector and 'tethered' data that could not move between sectors.

The discussion considered how data currently gathered is not attuned to the ways in which patients themselves define the success or failure of IVF treatment. The lived experience of patients and the ways in which they negotiate options with their doctors is excluded from current metrics that only assess the scientific foundations of treatment or the money involved. These miss the fact that some patients consider an IVF cycle that doesn't end in pregnancy a success, given that they might feel that they have tried all the means of getting pregnant possible. Patients should be involved in defining ‘successful' treatment, the audience agreed.

Overall, the consensus was that the problems with randomised clinical trials had been widely established, and that now cooperation and funding would be needed to devise fewer but more encompassing and focused studies into the effectiveness of IVF treatments.

The Progress Educational Trust (PET) would like to thank the sponsors of its conference - the Anne McLaren Memorial Trust Fund, Edwards and Steptoe Research Trust Fund, CooperSurgical, the European Sperm Bank, Ferring Pharmaceuticals, the London Women's Clinic, NGA Law and the Institute of Medical Ethics.