Seizing an Opportunity to Collect User Experience Data
Guest author Amelia Mackenzie, an associate scientist in FHI 360’s Contraceptive Technology Innovation Department, is exploring two factors related to contraceptive acceptability—menstrual bleeding side effects and partner opposition to contraception—for her dissertation at the University of North Carolina at Chapel Hill.
Contraceptive clinical trials routinely collect vast amounts of data, but what new data can we collect about method acceptability during this research stage? If a method has reached the clinical trial phase, we’d hope formative acceptability research was already conducted to inform its development and to determine if a potential market exists. At this point in the game, few changes can be made to a method based on acceptability findings…so what’s left to learn?
Before pondering further, let’s consider the context. Clinical trials are designed to study the safety and efficacy of a drug or device. They’re conducted in a tightly-controlled research context to improve our confidence that results are due to the treatment itself and not some other external or unmeasured factor. Consequently, they differ from “the real world” in several ways.
First, clinics chosen for trials are generally high-quality, well-resourced facilities, often located at or associated with research institutions. This likely differs from smaller or resource-limited clinics, and from standard clinical practice more generally, where staff specifically dedicated to the clinical trial are not present.
Second, clinical trial protocols detail precise procedures for counseling participants about the drug or device under investigation and include follow-up visits and clinical tests. In contrast, counseling in a standard clinic setting may be limited, with follow-up visits and testing perhaps happening only if clinically indicated.
Third, trial participants differ from the wider potential user population in their access to clinic trial locations and their willingness and ability to participate. We know racial and ethnic minorities are usually underrepresented in clinical trials.
Within the contraceptive clinical trial context, how is product acceptability measured? We know that researchers routinely collect some narrowly-framed acceptability data, often using method continuation rates as an indicator of product acceptability. Given the highly-controlled and usually incentivized context of a clinical trial, continued use is a pretty low bar. Beyond that, researchers may collect data on trial participants’ attitudes about the method by asking them: (a) to rate their level of satisfaction on a Likert scale, (b) if they would choose to continue using the method after the trial, and/or (c) if they would recommend the method to a friend.
The problem is answers can be overstated due to social desirability bias, and people are generally bad at predicting their future behavior. So how can we improve upon these currently used measures and incorporate some aspects of the “real world” experience?
We need to take advantage of what is unique about the clinical trial stage; it’s the first time people experience the new method in its full form rather than as a hypothetical method described in words or pictures or as a prototype. Our goal should be to capture this unique user experience by gathering data that will be informative after approval by regulatory agencies.
We can ask trial participants about benefits and difficulties they faced while using the method and how these issues impact their lives. This is important because some side effects experienced by users may not be clinically relevant to the safety or efficacy of the contraceptive but very pertinent to acceptability and a user’s everyday life. We can ask participants about counseling information. How well did they understand the information? Based on their experience, what information was helpful, and what information was missing?
To illustrate, let’s take a common side effect of current methods we know is related to acceptability: menstrual bleeding changes. Contraceptives can cause changes in bleeding duration, volume, frequency and/or regularity, but measurement of these changes in clinical trials has been inconsistent and vague. Clinical trial data don’t capture how different changes can have vastly different lifestyle implications. Based on existing clinical trial data, a provider may counsel a woman about how she may experience “lighter bleeding” while using a certain method. But we also should be collecting data during clinical trials to see if a woman interprets “lighter” to mean a shorter duration or less volume and if she understands this could include irregular or unpredictable bleeding.
Granted, some of this user experience information is difficult to measure within the standard confines of a clinical trial. Incorporating more behavioral and social science methods into the research approach would be a good start. For clinical researchers who find this kind of research a foreign concept, perhaps they could gain insights from comparable methodology employed by consumer product market researchers or by approaches used during earlier stages of product development as discussed in previous blogs in this series.
Developing a safe method that is effective at preventing pregnancy is simply not enough to ensure uptake and continued use, post-regulatory approval. We should use the unique opportunity of the clinical trial to collect user experience data that could inform product introduction, including marketing and promotion strategies, programmatic planning, counseling guidelines, and policy recommendations. Introduction strategies based on data – not on researchers’ or programmers’ assumptions – can then be developed during the clinical trial stage for immediate use post-approval. Why wait to start collecting this data until after the method is on the market? Seize the day.