Back to the drawing board: Redefining statistical significance

by

A recent panel debate moderated by Christian K. Schneider, head of Biopharma Excellence, explored emerging approaches to clinical research and data collection for therapies linked to rare diseases.

Key insights:

  • The panel considered the potential for supplementary data sets – such as pharmacodynamic read-outs, histological evidence and historical controls - in enabling regulators to reach robust decisions.
  • The panel also discussed on to the topic of reimbursement, which requires clarity about endpoints. Developers and regulators may be thinking about patient risk/benefit, while payers are more likely to be weighing up the cost/benefit.
  • Establishing endpoints is critical when studying rare diseases for which there have previously been no real treatment options; and include the patient voice as early as possible.

The debate panel:

  • Christian K Schneider: head of Biopharma Excellence and chief medical officer for biopharma at PharmaLex, is a former regulator at the British MHRA and the Danish Medicines Agency.
  • Nick Sireau: CEO and chair of trustees for the AKU Society which is dedicated to improving the lives of Alkaptonuria (AKU) patients.
  • Rachelle Jacques: president and CEO of Akari Therapeutics, a late-stage biotechnology company developing advanced therapies for auto-inflammatory and orphan diseases.
  • Daniel O’Connor: deputy director with responsibility for the Innovation Accelerator and Regulatory Science at UK healthcare agency, the MHRA.

In late 2022, Biopharma Excellence hosted a panel discussion featuring experts in biotech and the evolving regulatory environment (see above for panellist bios) to discuss the challenge of achieving statistical significance with novel and highly targeted therapies.

From a regulator’s perspective, Daniel O’Connor of the UK’s MHRA - with a remit of Innovation Acceleration within a regulatory context - noted that the challenge of achieving statistical significance when target populations are tiny is nothing new, but conceded that randomised trials aren’t always viable - especially with very rare conditions. He urged drug developers to seek scientific advice before finding workarounds.

Supplementing trial data

The panel considered the potential for supplementary data sets – such as pharmacodynamic read-outs, histological evidence and historical controls - in enabling regulators to reach robust decisions.

Nick Sireau, of AKU Society, talked about his early experiences of trying to get new drugs approved for Alkaptonuria (AKU); his two sons both suffer from the ultra-rare disease. His organisation relies heavily on grant funding and donations, with access to funds often difficult. It doesn’t help that the authorities’ requirements for clinical evidence have been quite restrictive up to now.

Fifteen years ago, 40 patients were recruited to an AKU drug trial that lasted three years and focused on a single endpoint (hip rotation), Nick explained. “The trouble is, that AKU affects patients very differently,” he noted. “So to just look at 40 patients with a single endpoint proved futile.” Although patients were reporting that they could walk further, that their pain had reduced, that they were feeling better since joining the trial, the study itself failed.

This led AKU Society to form a consortium of linked organisations across Europe, and a composite end point - a proposition which by then was well received in the EU. “That changed everything,” he said.

A progressive regulatory agenda

Daniel argued that the industry is now witnessing an evolution in regulatory thinking, resulting in a much more proportionate approach with regard to looking at the disease condition based on how much is already known: “not just thinking about either a randomised study or a single-arm study.”

Oncology’s use of basket studies has helped shine a light on what’s possible, he added. “But there’s also a lot of work now looking at shared molecular entities in rare diseases - something the International Rare Diseases Research Consortium (IRDiRC) is moving on quite quickly now.” The toolbox to get to the data now needed is visibly expanding, he said.

Likely next steps will need to come from a collaborative discussion and feasibility conversation - with patients; with researchers; with the regulators – about what's viable with a clinical development program. “We need to think about timelines, too,” Daniel added. “Three years is a long time to run a study.”

Reimbursement: identifying & aligning endpoints

The discussion moved on to the topic of reimbursement, which in turn requires clarity about endpoints. Developers and regulators may be thinking about patient risk/benefit, while payers are more likely to be weighing up the cost/benefit.

The industry as a whole must think more broadly in terms of endpoints that really matter, the panel concluded - particularly when it comes to very rare diseases for which there have previously been no real treatment options; and include the patient voice as early as possible.

Nick said he sensed that while regulators now seemed to accommodating the patient perspective more structurally, the big pharma companies tend to be less proactive about patient engagement. “Most drug companies will start working on something and then get in touch with us right at the last minute when things have started to go wrong,” he said. “There doesn't seem to be any systematic way that pharma companies engage with patient groups, particularly in the rare disease space.”

Involving patients at an earlier stage, finding out what’s most important to them and using this to direct research, is a growing area of focus among those thinking about all of this more strategically. Where cures aren’t possible or may still be a way off, for instance, factors such of quality of life can be hugely significant to patients.

Daniel talked about the Innovative Medicines Initiative (IMI), SISAQOL-IMI, which is working on recommendations about how to analyse and interpret data on health-related quality of life (HRQOL), led by the European Organisation for Research & Treatment Of Cancer around patient-reported outcomes.

“Not only in clinical studies, but also in a real-world setting - monitoring how patients are doing across their journey with a particular medicine after authorisation, something the MHRA is really keen on,” he noted.

A wish-list for change

The panellists ended by offering their own personal wish-lists for change if nothing was off the table. For Rachelle Jacques of Akari Therapeutics, multi-stakeholder approaches to overcoming practical barriers at a macro level is the way to go - rather than one company at a time; one patient advocacy organisation at a time; one regulatory body at a time – if the 95% of today’s remaining unmet needs are to be addressed sooner rather than later.

Nick said his wish would be finding a funding model that really works particularly for ultra-rare diseases, to fund studies that otherwise are just not commercially viable. He pointed to the Chan Zuckerberg Initiative in the US, which is doing much to build patient groups with significant funding, as something other regions might aspire to.