What is the value of Real-World Evaluation?

What exactly is Real World Evaluation? What does it cover? How will it benefit my business?

These are questions my colleagues and I across the AHSN network are asked regularly by health and care innovators looking to encourage the adoption and spread of their innovation.

Generating evidence to support the clinical efficacy and cost effectiveness of health technologies in the UK can be complex, time consuming and expensive. Once you have this evidence, innovators may still face the challenge of convincing customers of applicability in their organisation, which can hinder the spread of technology to help patients at scale across the health and social care system.

Traditionally, pharmaceutical companies and to a lesser extent, HealthTech businesses have used large randomised controlled trials (RCTs) as a gold-standard method to develop a strong evidence base that supports the clinical efficacy and cost effectiveness of a new medicine or product. For the HealthTech sector, this evidence does not always guarantee adoption at scale.

Depending on the context, RCTs may therefore be replaced, expanded, or supported by other types of evidence gathering, including single-arm studies, cohort studies, case series/studies, post-marketing surveillance, and increasingly, other Real-World Evaluations (RWE).

To answer the three questions above in simple terms:

RWE is a process of gathering evidence to demonstrate the value of a new product in real-world settings, in other words: outside of a controlled research environment. This might include practical elements such as understanding and describing training needs, or how much IT resource is required to integrate a new technology.

The definition we use within the AHSN Network is broad. Our view is that it encapsulates all activities that collect evidence outside of a research governance framework requiring consent of patients or healthcare professionals. It is important to bear in mind that one size does not fit all, and how to proceed with RWE should be adapted to the innovation, its setting, and the needs of the innovator but should always meet the NHS organisations standards and procedures that ensure good governance.

There are a few good reasons why companies should invest time and effort in RWE:

1. Activities carried out in a RWE increase buyer confidence that a product or solution will deliver the described benefits in their specific system.
2. RWE helps organisations to implement new technology as easily as possible by building on learnings from previous rollouts in other organisations’ RWE projects.
3. Digital technologies can potentially have much shorter product lifecycles in the market than medicines. This means that large, expensive, and time-consuming randomised control trials (RCTs) are often not feasible or affordable to generate evidence within an innovation’s lifespan, therefore RWE may be more appropriate.
4. Quantifying the value of RWE projects to businesses is challenging. The quality of projects carried out can be variable and therefore trust in these evaluations within the health and social care system can also vary. For this reason, we have been collaborating with key industry counterparts such as NICE and ABHI to develop a practical guide for innovators, that when followed can help provide assurance to commissioners and partners that the evaluation is of a high standard. This practical RWE guide compliments the recent publication by NICE of their Real-World Evidence Framework which really helps innovators think about the quality of the data they are collecting if they are hoping to use it to inform NICE guidance.

The RWE best practice guide has also been informed by companies who have completed robust RWE projects in partnerships with the AHSNs across England and are now seeing the business benefits:

One example is Qbtech who developed QbTest, a simple test to help support the rule in/out diagnosis of ADHD in children and young people. Following support from the AHSNs, including working closely with them on RWE projects, QbTest has spread to 63 NHS trusts and 123 sites in England (summer 2022.)

“If I were to advise other innovators, I would suggest that they really need to figure out what they need to prove and how they can show value before undertaking a Real-World Evaluation. We started with a small audit and the finding from that audit and then subsequent audits really gave us the confidence and informed what might we be looking for in terms of Real-World Evaluation.” Tony Doyle, Managing Director, Qbtech.

Another example is Roche Diagnostics, a large company with good products that were not being adopted by clinicians such as the PLGF blood test for pregnant women (the first of its kind in the world to rule out pre-eclampsia). They worked with the AHSN Network to establish why, and gather the real-world evidence they needed to turn this around and spread it nationally:

“I think most concerning for me was that the [PLGF] test was available for two years – NICE recommended it in 2016 – and we were working to make clinicians aware of it, but it simply wasn’t being taken off the shelf and used.

“We had to think about all the healthcare providers that touch that patient in that pathway. Not just the obstetricians and gynaecologists but midwives in our case. Who else within that pathway do we need to educate and convince of the benefit of the test, because each pregnant woman spends far more time with their midwife than they do their consultant or registrar, and for them to understand the test as well becomes quite a large training and updating exercise.” Julia Eades, Senior Market Access Manager – Women’s Health, Roche Diagnostics.

These and many other companies have realised the value in carrying out this type of evidence gathering. Well-designed, real-world evaluations, which respect the work that NICE have completed in their framework but are much more ‘practical’ in their approach, will help decision-makers in the NHS health and social care system make good, consistent decisions on innovation selection and, importantly, know how to implement them to ensure they return the high-impact health benefits promised.

• Read the guide: Real-World Evaluation to facilitate adoption at scale.
• Contact your local AHSN for advice on Real-World Evaluation projects.