If you’ve been following the health economics industry for the past few years, you’ve heard about value-based pricing agreements between payers and the pharmaceutical industry. With value-based approaches, payment is linked directly to real-world treatment effectiveness. In this way, purchasers’ and manufacturers’ economic incentives are aligned based on clear, mutually defined patient outcomes.
Once or twice a year, I look around to see if anything major has changed in this area – Is the chatter on value-based agreements getting louder? Are value-based approaches becoming more common? My take on the current environment is that stakeholders are talking about value-based approaches more than ever, but their actual, successful implementation is stalling.
In a recent blog on payer value messaging strategies, HealthEconomics.com’s Patti Peeples indicated that a shift is clearly underway “towards more flexible risk-sharing and cost management mechanisms from the payer’s viewpoint.” But she proceeds to argue that substantial misperceptions persist regarding what payers actually want from pharma versus what pharma believes payers want. The ultimate problem? A lack of effective communication between the players.
In a 2011 case-based investigation published in Health Affairs, Peter Neumann and colleagues found very few examples of successful risk-sharing agreements, and noted that U.S. stakeholders continue to focus primarily on payment models not connected to performance assessment or data analytics. They: “The principal lesson thus far seems to be that risk sharing for pharmaceuticals is appealing in theory, but hard in practice.” The primary barriers identified? High implementation costs, lack of a data infrastructure, and challenges with outcomes selection.
To be fair, this issue is multifaceted and doesn’t have easy solutions. But I think a way exists to circumvent at least one key obstacle – the data infrastructure gap. According to Neumann et al: “Risk-sharing agreements require high-quality information systems, databases, and operational expertise.” It takes time to put these capacities in place, but there may be a workaround.
But before going any further, it’s important to understand that the way value is measured is undergoing a sea change.
The Old Ways of Measuring Value Are Changing
It could be said that value (treatment outcomes adjusted for treatment costs) is the newest currency in healthcare. But to be useful, value needs to be defined, measured, predicted, and optimized. Thanks in large part to novel study designs and the availability of advanced analytics, our notion of what aspects of value can be measured is evolving rapidly.
A poster presentation from Dinh and colleagues at the 2013 International Society for Pharmacoeconomics and Outcomes Research (ISPOR) conference outlines some key distinctions between past and future approaches to value strategy development.
The old way is reactive.
It does not consider value until late in the drug development process. This approach:
The new way is proactive.
It applies value prediction in early drug development (for example, by identifying “most valuable” subpopulations using data-based methods). This approach:
Getting Around the Information Technology Barrier
A 2012 Deloitte Center for Healthcare Solutions Issue Brief summarizes the information technology aspect of the value analytics gap as follows: “The availability of valid-real-time data for value metric assessments requires widespread electronic exchange of health information among stakeholders and disparate sources…this extensive network needs to be further built as it is not currently widespread.”
In some cases, the pharmaceutical industry and payers are teaming to overcome these data limitations. A collaboration between AstraZeneca and WellPoint, established in 2011, is using electronic medical records, claims information, and patient survey data to evaluate how currently available drugs affect healthcare costs and patient outcomes, particularly in chronic disease.
There are also some electronically enabled tools that are readily available to assess cost-effectiveness. Retrospective database analyses have the virtue of very large samples, substantial data inputs, and access to “real-world” patients. Interactive models simulate future outcomes and costs associated with specific products, and can be designed to reflect specific payers’ resource utilization patterns.
But these approaches have limitations when assessing the clinical and economic impact of new treatments. First, there’s always missing information – claims records don’t necessarily contain the biomarker and lab data needed to assess treatment efficacy, and interactive models are limited in scope. Also, in every case, someone has to take on the work and expense associated with designing and running new studies or building new models. And of course, if a treatment is not already on the market, time needs to pass before you can obtain useful information from retrospective claims data.
I think there is a way to bridge this gap. The data that players need can be augmented by incorporating accurately simulated data, using a tool that’s available today.
A Way To Look Data From Many Sides
When only limited real-world data are available, or the data doesn’t allow for the answering of specific questions, the judicious integration of simulated data can help payers and pharma work together to build a case.
Simulation models can answer the big “what if” questions that are cost-prohibitive or can’t be evaluated with traditional research. For example, what if you had a new dyslipidemia treatment, but had trouble discerning its efficacy against the background noise created by other cholesterol-lowering medications? Models enable you to run unlimited simulations of patient populations with user-defined characteristics.
The Archimedes Model is a large-scale, clinically realistic model of health and healthcare delivery. It contains detailed patient information obtained from public databases, clinical trials, observational studies, and epidemiologic data; this lets users conduct research on the general U.S. population or subpopulations. The Model also integrates with proprietary data, for example, claims or electronic health records containing biomarkers or clinical practice processes.
ARCHeS, the Model’s intuitive, online platform provides a natural venue for collaboration and value strategy development. The results of a recent Quintiles survey show that payers want more involvement during every stage of drug development. With ARCHeS, manufacturers and payers can sit down together (literally or virtually) at any point to “slice and dice” their customized data. ARCHeS can be used in two ways:
- With ARCHeS Population Explorer, users can model the impact of a new treatment on multiple populations (either the U.S. population or defined subpopulations, or users can upload their own patient data).
- With ARCHeS Trial Designer, users can set up virtual clinical trials for new products, compare the results for accuracy, and then set up new virtual trials to gain additional information—all within a few hours.
This is a new approach, and it could make it possible to examine some big economic and clinical questions early in the game:
- What cost savings are likely to be associated with a treatment, and what is the financial hit if any key assumptions are incorrect?
- What downstream effects, both positive and negative, are likely to be associated with a treatment?
When information like this is available up front, it becomes possible for stakeholders to negotiate based on accurate answers to “what if?” statements. For example, at what price point does an intervention become effective? What downstream effects can be expected to accompany a 9 mm Hg drop in blood pressure? What is the cost-benefit relationship of screening select subpopulations for certain cancers?
The next big experiment I’d like to see in value-based pricing involves the Archimedes Model and ARCHeS. With ARCHeS, you can integrate various types of data, hypothesize, add missing information, and receive answers to the questions that will help determine the specific value of an intervention in a particular population. The tools are there, just waiting to be used – and innovation has never been easier or more straightforward.
Caitlin Rothermel, MA, MPHc is a Northwest Washington-based medical and health economics writer. She has paid close attention to the Archimedes Model for years and gets completely geeked-out happy by Bayesian hierarchical modeling. You can learn more about Caitlin by visiting www.MedLitera.com.
Paid advertorial by Archimedes, Inc.
Maintaining high levels of patient recruitment, adherence and retention is essential for the successful completion of a clinical trial, yet it remains a significant challenge faced by researchers. Poor patient adherence and retention can adversely affect a trial by lengthening timelines, adding cost and risk to the validity of the data and delaying product approval.
There are numerous factors which contribute to the slow process of attracting the patient to consider participating in clinical research studies. These include lack of awareness of the critical role that clinical trials play, limited access to knowledge about clinical trials, lack of understanding about rights, safety and benefits and which are among the most common reasons for low participation. In terms of retention in the trial these include (i) patient-centred factors: such as demographic and psychosocial including beliefs, attitudes and motivation, severity; (ii) therapy-related factors: such as treatment complexity adverse reactions, lack of therapeutic impact; (iii) social and economic factors : including inability to take time off works, lifestyle patterns etc. (iii) Other factors: including clinical trial site location, frequency of clinic visits, clinic staff etc.
Development and integration of patient retention strategies that address the issues of patient perceived benefits, barriers and burden by leveraging both technology and communication is essential for addressing patient dropout. However, traditional thinking would have us believe that an individual’s decision to stay in or drop out of a clinical trial is based on the patient acting rationally.
At DHP Research we are bringing to our clients an approach which to quote Jules Berry “…shines a light on the factors that influence our actions” in an attempt to understand more the subtle and complex mechanisms that influences patient behaviour whether to stay or pullout of a clinical trial. It is to paraphrase Jules Berry, viewing the solutions through the prism of behavioural economics.
What is behavioural economics (BE)?
BE is a multi-factor approach to the understanding of human behaviour that represents a paradigm shift in the traditional thinking that our behaviour is rational. Rather the principles of BE are that our behaviour is to a large extent unconscious, irrational and socially driven. Therefore, for any clinical trial retention strategy to be truly patient-centric it must take account of these influences on patient behaviour throughout the different phases of a clinical trial.
The key characteristics of human behaviour outlined in BE are:
- Personal factors: We don’t like change, we live in the here and now, we are averse to loss, we want a positive and consistent self-image
- Social factors: We are heavily influenced by others
- Local and choice environment: The environment matters, it’s hard work to think and choices are guided by salience of information and mental shortcuts
Lets illustrate how these might apply in a real world setting.
Personal factors: We think short term and avoid loss rather than achieve gain (loss-aversion). In other words we feel loss more keenly than gain. For example, offering an amount of points at the start of an exercise programme which could be exchanged for items or money on programme completion – but would be withdrawn for failure to adhere to aspects of the programme – is likely to be more effective in achieving adherence than accumulating points from zero.
Social factors: In addition to our need to retain a positive self-image, individual decision making is heavily influenced by others – the power of the messenger – rather than the message itself. It is certainly true our behaviour can be be influenced by experts and authority, but the people like us (our peers) have tremendous influence over what we do.
Knowing which groups of people are likely to be be influential is a key question in developing a patient retention strategy and one that only research can address but, patient groups and their communications for example through social media and mobile phone apps could have a significant influence on trial participants to stay in.
Choice environment factors: When it comes to making a decision or choice as humans we don’t like to think too much. We can’t attend to and process all available information. Too much information, too many messages leaves us unable to cope and our system 1 thinking – perceptual, intuitive, influenced by emotion – kicks in to help us make decisions on judgements that come easily to mind.
When it comes to choice things that come to mind easily are considered important and decisions are often influenced by the salience of the information readily available. For example, we are less likely to participate in a clinical trial because we know someone who had a bad experience and withdrew. This is known as as the availability heuristic which results in the individual giving too much emphasis to small probabilities.
As individuals we priorities information that supports our existing beliefs and will filter information that supports those beliefs. Related to this is anchoring and occurs when we are presented with a piece of information we then use as an anchor for all subsequent information. For example, if patients are told that in previous studies that chances of adverse reactions was 8% this will serve as an anchor for the expected adverse reactions in the current study whether this is high or low.
Information such as this can be communicated in different ways and which can have a profound effect on choice For example, at a trial recruitment stage patients are told there’s a one in 25 chance of having an adverse reaction it’s more than likely the majority of patients will think this as extremely risky. However, informing patients the treatment is 96% reliable will more than likely be considered as very safe. Neither of these values is strictly “the truth” but they provide a perspective which is known as framing where the salience of information is created by presenting the more positive side of the problem.
Salience can also effect what we remember which is usually shaped not by the average way we felt about an experience but, rather at the peak and end of the experience. We use a mental shortcut to remember the most salient aspect of the experience. For example, having met interesting people at a party might be the salient thought about the party and will have significant impact on your choice to accept another invitation from the host. For trial participants this would not be the overall experience of participating in the trial to that point that comes to mind, but for example, how they were made to feel the last time they attended the trial site or experienced an adverse reaction.
The effect of salience can be seen in our everyday lives such as the most popular/expensive coffee and tea placed on the front of shelves, chocolate and crisps next to the till.
Ensuring trial participants have a positive experience at the trial site is critical for ongoing patient participation not only for increasing the salience of that positive experience for example through 1 to 1 discussions with patients about their well-being and attitudes and experiences of participating in the trial but, also as an opportunity to reinforce the salience of positive aspects of the trial.
This post has described just some of the key aspects to illustrate how BE can provide an added dimension to understanding behaviour. Through the careful assessment of these behavioural factors using different research techniques we can build a framework that helps us explain how these factors relate to behaviour in different situations. There are however, questions still to be answered such as the importance of each factor. Do the factors work together? How do the factors work at the individual level? Are the factors equally important for different disease groups and trial? Yet despite this and as yet not being able to make accurate predictions, behavioural economics does have the potential to be a real game changer in understanding some of the subtle and complex mechanisms that influences patient participation in clinical trials.
We hope to present more on this topic at the DIA 2014 Annual Meeting, June 15-19, San Diego, CA
For more information or to discuss this blog contact us at firstname.lastname@example.org
Are traditional conferences and trade shows dead? That thought probably has Conference Companies quaking in their well-traveled shoes, while industry execs are drumming on their laptops with glee at the thought of never having to go to another 2-day meeting in a Marriott ever again. But maybe it’s time to re-think conferences and how we share ideas. Let’s explore that for the HEOR and Managed Markets audience for a moment.
There are dozens of conferences held each month. These include major association meetings, like ISPOR, AMCP, DIA, ISPE, etc. as well as for-profit conference companies sponsoring events like CBI, eyeforpharma, IIR, marcusevans, and more. Just look at the conference calendar on HealthEconomics.Com for a peek at the sheer number of events on pricing, reimbursement, ACOs, PROs, and informatics. Many industry watchers bemoan the state of conferences in our field, complaining of low attendance, repetitive presentations by the same individuals hawking their wares, as well as the expense associated with registrations, airline and hotel stays. This doesn’t include the biggest expense, which is time away from the office with limited ROI. In fact, many government organizations in the United States forbid all but the most mission-critical conferences if they are further than a Metro ride away. These restrictions are in addition to an Office of Management and Budget memo promoting efficient agency travel spending, disseminated in May 2012.
Please take a Quick Poll about your company’s recent policies on conference attendance. When answering, consider whether your company has implemented policy changes, levels of approval, per diem spending limits, or recommended limiting travel/attendance in any way.
Of course, live conferences may offer the possibility of networking with individuals outside your own company, and occasionally hearing a nugget that might be the genesis of a big new idea or a re-thinking of an old approach. Nevertheless, forward-thinking executives have been wondering: How can we retain the benefits of conferences and trade shows, while tossing out the negatives?
One word: Virtual.
What does that have to do with HEOR and Managed Markets? One more word: HE-Xpo. HE-Xpo is a virtual conference, trade show and market place, just for the health economics, outcomes research, and managed markets community – worldwide.
Growth of Virtual Conferences & Tradeshows
More about HE-Xpo is below…but first, let’s back up. What exactly is a Virtual Conference and Trade Show and what is happening in this arena? According to a leading market research and technology research company, Market Research Media, the worldwide virtual conference market is forecasted to grow at a compound annual growth rate (CAGR) of 56% between the period of 2013 and 2018. This prediction was contained in their recent research report “Virtual Conference & Trade Show Market Forecast 2013-2018“. The worldwide virtual conference and trade show market was predicted to reach $18.6 Billion over the period 2013 – 2018.
The image below, derived from Market Research Media, shows a Venn diagram of sorts describing Virtual Conferences & Trade Show components.
In a recent Reader Satisfaction Survey by HealthEconomics.Com conducted April 2013, “Virtual Conferences” was the most requested new service by the >200 survey participants. As a result, we answered your call. HealthEconomics.Com has launched a new service, HE-Xpo, a virtual conference, trade show, and market place available 24 hours a day, 7 days a week, 365 days a year. HE-Xpo allows you to feature your company, products, and services in a media-rich, interactive environment that is SEO enhanced and offers powerful lead generation through real-time attendee profiling. HE-Xpo has Exhibit Booths, a Learning Center, an Auditorium, and a Lounge. You can create a compelling user experience by hosting live webinars or streaming a conference live in the Auditorium, uploading unlimited media to your exhibit booth and the HE-XPo Learning Center, including video, audio, images, PowerPoint slides, spreadsheets, PDFs and more, as well as network and share via social media (Twitter, Facebook, LinkedIn, Google+, more) in the Lounge. For a full description and price list, view our HE-XPo Media Pak.
HE-Xpo is going live this month (July 2013), with many HEOR company booths in the Exhibit Center and many items already uploaded into the Learning Center. Several industry associations are considering using HE-Xpo to live stream their annual meetings to expand involvement and access to those individuals in HEOR and Managed Markets who cannot get to the live meeting. Virtual attendance is the next best thing, and it may be even better (from an ROI) than attending in person. Contact Leslie Fine, HE-Xpo Marketing Manager, at email@example.com , to reserve your exhibit booth and get involved in the next big technology for the industry: virtual conferences and tradeshows.
HE-Xpo Grand Opening and Launch Party – September 19, 2013
Mark your calendars for the HE-Xpo Grand Opening & Launch Party, scheduled for September 19, 2013. Our HE-XPo Grand Opening is a day-long event, full of live presentations, exhibitor give-ways, learning opportunities, networking, and powerful lead-generation opportunities. We want you to be present and participate. HE-Xpo gives you all the next generation tools you need to showcase your product or service. We want to help you turn your booth attendees into customers and getting your booth ready and active is a snap. You are literally minutes away from having a live and accessible booth, available 365 days a year, 24 hours a day because it is extremely easy to set up. And, our experts are here to assist you in any way possible so that you can begin to see the benefits from your participation as soon as possible.
We can’t wait to have you join us with this special offer! If you are ready to raise the quality of your digital marketing experience to new heights, please contact Leslie@healtheconomics.com to get started.
Collecting information using a questionnaire to measure patient satisfaction, experience and health is now common practice. There is a variety of health questionnaire types that can be used. Whichever way questionnaires are used and for whatever purpose, the objective is to obtain reliable and valid information on the patient’s experience and reported outcomes. Below are some pointers as to why NHS patient experience surveys can fail in providing useful information.
- Not understanding the big picture: It is essential that the overall objectives of the patient experience survey are defined at the outset (the research question). This will include establishing the purpose of the survey e.g. measuring patient satisfaction, experience and outcomes, clarifying the target population the health questionnaire will be administered to e.g. patient group, disease type, how the information will be collected e.g. paper/pencil, interview, web and how that information will be used e.g. improve patient experience.
- Using inappropriate data collection methodology: It is obvious but, telephone surveys are inappropriate for the hard of hearing and elderly. If you are using a postal survey how reliable is your data source? What is the literacy level of your target population?
- Choosing the wrong question type: Choosing the correct type of question for your health survey will involve making decisions such as whether to use an open or closed question, a ‘don’t know’ response option, rating scales or grids etc. Remember complexity leads to non-response.
- The questions are lengthy and difficult to understand: Remember respondents need to understand what the question is asking to give you the correct information. A well crafted questions needs to be no longer than 20 words, should be written using plain and simple language. The question should ask one question e.g. How would you rate the receptionist’s helpfulness? NOT How would you rate the receptionist’s and doctor’s helpfulness?
- The survey questions are not relevant: Survey questions must be relevant and specific to the target population. When developing a new patient health questionnaire patient input is essential to ensure content validity. If using an existing questionnaire then establish content validity via some focus groups.
- Not pre-testing the questionnaire: Pre-testing the patient questionnaire can highlight any problems with it, including length, understanding, missing questions etc. Pre-tests can be carried out on a small sample of the target population.
- Getting a low response rates: Response rates are critical to the success of a patient experience survey and with some thought and planning can be as high as 70%. This includes, a good introduction letter to the patient explaining the survey and confidentiality, a stamped addressed envelope if a postal survey and reminder letters.
For more information on patient experience questionnaire design download the 7 Tips you need to know for successful questionnaire design
Dr Keith Meadows is founder of DHP Research & Consultancy specialising in the measurement of patient reported outcomes and experience.
A key hurdle facing the entire pharmaceutical industry is non-adherence by patients to medication. This problem is only likely to be surmounted if patients believe that taking medication will lead to immediate benefits through reduction of symptoms, improvement in physiological functioning and quality of life.
The diabetes mellitus (DM) marketplace, for example, is becoming saturated with multiple medications in both the insulin and pre-insulin space, particularly as analogues start to lose their patents. Differentiation in clinical outcomes within classes is often unclear or minimal. This means that differentiation of therapeutic options is likely to focus more on frequency or mode and method of administration, as opposed to statistically significant differences in glucose control, which are clinically relevant.
What is a measurement strategy?
An effective way of establishing the link between the measured outcome, such as the patient’s health status or quality of life following an intervention programme, is the development of a measurement strategy which requires a clear understanding of the disease and the relevant primary outcomes (e.g. reduced hypoglycemia and secondary outcomes e.g. reduced anxiety).
A patient-reported outcomes (PRO) measurement strategy provides a framework to support the selection of an appropriate PRO for a clinical trial through which treatment effectiveness in terms of health status or quality of life for example, can be demonstrated in relation to the desired primary outcomes.
Components of each of the key stages of the strategy are shown below. This strategy makes explicit the expected treatment effects (e.g. primary biomedical endpoint(s)) AND IMPORTANTLY the secondary endpoints such as reduced anxiety, to be measured by the PRO.
A critical aspect of the measurement strategy is selecting the appropriate PRO that captures the benefits of the primary physiological endpoint of treatment i.e. secondary endpoint(s). However, outcome teams are frequently faced with a plethora of potential PROs each purporting to measure – often without a sound theoretical or measurement model – specific health constructs such as health status or quality of life. As a consequence the choice of a PRO is often made according to:
- the instrument having been used in previous studies
- its name appears to be appropriate for the intended use
- The supporting psychometric data looks o.k.
Furthermore, there is the tendency for those conducting clinical trials to treat the more commonly measured health constructs such as quality of life (QoL), health-related quality of life (HRQoL) and health status as interchangeable in the PRO selection process which they are not. Examples of this include the SF-36 and EQ-5D which are frequently referred to as indicators of QoL, but in fact are more indicators of health status, which of course can impact on the individual’s QoL. Health status is a measure of the quality of health yet while there is no universally accepted definition of QoL, there is the general consensus that it is based on the individual’s subjective evaluation of the psychological and social aspects of their life including work, school and family.
As described above, essential to selecting the appropriate PRO is to make explicit the expected treatment effects e.g. primary biomedical endpoint(s) and the resulting secondary endpoint(s) which should be articulated through the endpoint model from which the most appropriate PRO can be selected. For an example of a simple endpoint model see below.
In this simple model, the objective is to reduce recurrent hypoglycemia which – as the primary endpoint – will be a reduction in the various effects of hypoglycemia including sweating, shaking. etc. Then, as a result, an improvement in the the desired secondary endpoint of the patient’s quality of life will occur. Clearly the model requires further expansion to include which elements of QoL are to be measured as well as health status. Having specified the secondary endpoints the appropriate PRO can then be selected. Central however, to selecting the PRO is the PROs conceptual framework.
What is a PRO conceptual framework?
The PROs conceptual framework shows the item content in relation to the specified concepts/domains the instrument is purported to measure. Therefore, if the PRO has been selected to measure aspects of sleep disturbance as a secondary endpoint, then there must be clear conceptual and psychometric evidence that the items of the PRO should relate directly to these specific construct. Below as an example is the conceptual framework of the Diabetes Health Profile -a PRO developed to assess the psychological and behavioural impact of living with diabetes that was derived on the basis of significant patient input and psychometric evidence.
Not all PROs make explicit their conceptual framework, but it’s worth bearing in mind that evidence 0f a conceptual framework is an essential requirement by the FDA for labeling claims
Selecting the most appropriate PRO to provide evidence of treatment effectiveness based on the patient’s perspective is a complex process requiring an explicit measurement strategy. This will include defining the primary endpoints and their relationship(s) with the desired secondary endpoints and linking these with conceptual framework of the selected PRO.
Dr Keith Meadows is Founder and Director of DHP Research and Consultancy Ltd. Keith has extensive experience in the field of patient reported outcome measurement with a particular emphasis on the psychological impact of living with diabetes.
If you’re reading this, chances are you’ve heard about the Oregon Medicaid study recently published in the New England Journal of Medicine. In case you’ve been on vacation, however, the results were, at best, a disappointment to advocates of Medicaid: a large, multi-year randomized controlled trial failed to conclude that coverage through the program produced meaningful health benefits, relative to remaining uninsured. As landmark publications of policy interventions are pretty rare, and insofar as the ACA health reform is counting on a dramatic expansion of Medicaid to achieve its goal of universal coverage, the paper has predictably set off a spirited debate within the health policy community. Those on the right are pointing to the findings in calling for a halt to the ACA Medicaid expansion. Among mainstream analysts, we’ve seen a call for caution in interpreting these results (though similar restraint was not advocated by the same crowd when more positive interim results were published in 2011). As in the case of other studies published in areas of contentious debate (see also: screening mammography), having a high-quality trial testing the question of interest, and a p-value associated with that test, is not enough to clarify the matter. I’ve written at length about the Oregon study already, but I’ll be honest: I don’t much care about the study’s findings. For the simple reason that there’s no way a single study can offer a conclusive answer to a research question.
One reason for this is what you might call the random-effects problem. If you ran the Oregon study again, the influences of random error suggest you’d end up with another set of estimates; run it again in a different sample of people, and you’d get still another set of findings. If you replicated the study a bunch of times, eventually you’d start to see the results converge on the “true” effect. As such, findings from single studies are best thought of as single draws from a distribution of possible results. This, of course, implies that it’s really hard to conclude anything from empirical studies, and that you need an enormous amount of data to say anything with confidence, but … that’s actually completely the case. When you estimate a treatment effect from a trial, there’s no real way of knowing whether that estimate is close to the mean of that theoretical distribution of results, or whether it’s an outlier. (Note: if you’re familiar with the work of John Ioannidis, this won’t surprise you much.)
Many policy researchers, of course, don’t really care about this; they care less about whether the effect estimate is perfectly accurate than about whether it’s “significant”. The debate over whether or not Medicaid has downstream effects on health has had an either/or flavor, and we’ve been told over and over that the ACA’s Medicare pilots and comparative effectiveness research will tell us “what works” in medical care. In practice, concluding that something “works” from empirical research involves looking at the p-value and seeing whether its effect is “significant” (i.e., small enough to conclude that the observed effect is unlikely due to random influences). But does a “significant” test result really tell you that?
I know this runs counter to everything smart people tell you about statistics, but bear with me. The process of inferring “significance” starts with a null hypothesis H, which states that the groups don’t differ. If the null hypothesis is true, then the probability of observing no difference between the groups is high (with “equality between groups” E defined via reference to a statistical distribution suggesting the threshold at which an observed difference is unlikely to occur by chance). Then we do our study, and observe a difference between groups that’s larger than our “likely” threshold (i.e., not-E). The conclusion most researchers will draw is: “then H is probably not true”.
One obvious problem here is that H isn’t a random variable — hypotheses are either true or they aren’t – so discussing it in terms of probabilities is nonsense. But even if we toughen up the inference from our hypothesis test (i.e., “if we see not-E, we assume that H is false”), our conclusion still doesn’t really follow. In many ways, hypothesis testing is a sort of game that researchers agree to play, wherein observing a difference believed to be unlikely based on sampling error alone leads to the conclusion that the null is false. Yet proving a conjecture requires a lot more than this. The conjecture either needs to be impossible to dispute without contradicting yourself, or you need to be able to demonstrate it inductively by observing it over samples of homogeneous subjects. Clearly, the first approach rarely works in health services research, since few claims about services or policy are necessarily true. To prove a contention via the second approach, you’d need to go through all the subjects in each treatment group and show that the difference between groups is maintained throughout the sample. This is a first step – generalizing to different subject samples is one problem, as is the fact that people adapt to policy interventions over time (meaning the underlying relationships aren’t constant) – but the bigger point is that a conclusion drawn from a comparison of mean differences between groups doesn’t come close to either proving or refuting the hypothesis. Put simply, the information from a hypothesis test is not sufficient for proving the truth or falsity of H. And as such, we still don’t know whether or not Medicaid has any effect on health status.
When I write things like the above, I’m sometimes accused of being anti-research or anti-science. Nothing could be further from the truth. What I’m opposed to is the misuse of science. Had the Oregon study found significant effects on health status, I’d feel the same way about the trial. Insofar as physicians, patients and policymakers all carry biases into their work, and most people tend to generalize wildly from their own experiences, repeated observation over time can perform a valuable service in helping us to understand health care delivery in a more objective way. But this is very different from using limited sets of observations to leap to broad conclusions, and from asserting the truth or falsehood of theories without doing anything resembling a rigorous proof. If health services researchers want the responsibility of their work being used to guide medical practice, they really need to start stepping up their game.
In yet another sign of just how dysfunctional and Kafkaesque our healthcare system has become, consider a study recently published in the Journal of the American Medical Association.
Researchers conducted a retrospective analysis of data for all inpatient surgical discharges during 2010 from a 12-hospital system to compare costs, revenues, and contribution margins (revenue minus variable costs, representing financial resources available to pay fixed costs) between patients who developed surgical complications and those who didn’t. For the record, the greater the positive contribution margins, the better the hospital is doing financially.
Bottom line: hospitals earned more—a lot more—on patients with surgical complications. Although patients with 1 or more complications incurred higher costs, those with private insurance were also associated with a contribution margin (profit) of $39,017 each; those with Medicare, a contribution margin of $1,749 each. Looking just at margins (revenues minus fixed and variable costs), patients with private insurance who had 1 or more complication brought in $25,000 more than those without complications, although Medicare, Medicaid, and private pay patients brought in less.
So, I ask you, where is the incentive for hospitals to improve the quality of their care to reduce surgical complications?
Now, I’m not suggesting that surgeons are deliberately introducing complications or not providing the best care they can. However, numerous studies find that performance improvement and reengineering efforts can reduce the rate of surgical complications. They also show that when hospitals face a financial penalty for complications (such as hospital-acquired pneumonia) or readmissions, they figure out a way to reduce or prevent them.
But we’re not incentivizing hospitals to take those steps when it comes to surgical complications.
My primary frustration is with the commercial payers. Their reimbursement rates ranged from 100% to 250% of Medicare rates. If we had a single-payer system, of course, everyone would get the same fee for the same diagnosis-related group (DRG) (with adjustment for severity mix and geographic location, of course).
The authors of this study offer 2 suggestions for addressing the problem: bundle the average cost of complications into the DRG payment for surgical procedures, and/or limit the hospital’s ability to upcode the surgery when complications occur.
What do you think we should do?
The passage of the ACA in 2010 has to feel like a high-water mark for proponents of preventive medicine. The Obama health reform essentially makes free preventive care the law of the land, with insurers required to completely cover the cost of a laundry list of interventions intended to avert dreaded chronic illnesses like diabetes, cancer, and cardiovascular disease. The rationale for this policy, often repeated, is that the chronically ill account for 75% of health care spending. Manage their illnesses more effectively through evidence-based treatments and lower-cost non-specialist providers, the argument goes, and you’ll solve health care’s cost crisis, as well as creating a healthier population. As such, by 2014 most insurers will be required to pay for utilization of services like blood pressure, cholesterol, and depression screening, routine cancer screening, obesity and tobacco cessation counseling, and a number of services specific to women and children. The expectation being that a ton of costly chronic illness will never occur as a result.
Unfortunately, there are a few problems with this concept. For one, it directly contradicts the mandate for evidence-based care that most of these folks support: there’s almost no evidence suggesting that these screening interventions have any benefit to patients. A common refrain to this is that engaging patients via preventive screening makes them more conscious of the need for self-management. But this is also unproven: a decade or so ago, “disease management” was a fad that fell short of expectations (as most health policy fads do). In many ways, the push for preventive care is of the “we don’t know what to do, but we have to do something” variety.
But preventive medicine’s problems with comparative effectiveness don’t end there. One of the premises of evidence-based care (like, one of the really, really basic premises) is to avoid providing services with no value to patients. It’s why we care about patient-reported outcomes and using resources efficiently. But providing large volumes of care to asymptomatic, average-risk populations is pretty much guaranteed to waste a ton of resources. If you insist, for example, on regularly screening women for breast cancer without any indication of elevated risk, what you’re mostly doing is inventing a complicated way to keep radiologists busy. Yes, you’ll detect some disease (not all of it will be cancer, of course), and you’ll likely prevent premature death in at least some women. Of course, you’ll also overdiagnose a lot of lesions that will never become malignant, and turn up a ton of false positives. And, importantly, by removing cost considerations from the equation, you’ll be subtly discouraging women from weighing those risks against potential benefits. Interestingly, prevention advocates don’t run from this implication of their work, often embracing the numbers-needed-to-screen statistic in their arguments. Yet implicit in an NNS of, say, 600 is that 599 people receive a worthless intervention. Acute treatments aren’t perfect on numbers-needed either, but they do far better than prevention does, since the populations are smaller.
Unfortunately, the cost rationale for prevention is almost certainly overblown as well. For one thing, the 75% figure is absurd; it includes all the costs of care for people defined as chronically ill, not just the costs of their hospitalizations for uncontrolled disease. For another, the types of analyses that usually establish “cost-effectiveness” don’t really work without evidence of effectiveness. And take it from me: I used to work for a large medical group whose screening recommendations were frequently criticized as heartlessly conservative, and I can tell you that screening programs for large numbers of people are staggeringly expensive. And the patients in our medical group had to cover part of the cost. It’s far from a certainty that providing a wide array of services to a broadly defined population, whose marginal cost of consuming those services is zero, is going to be less costly than acute treatment of a much smaller number of chronically ill patients. Not to mention that preventing some costly conditions earlier on may leave people vulnerable to other costly conditions in later life. I’m thinking specifically of the recent RAND study in the New England Journal of Medicine, which estimated the economic burden of dementia at somewhere between “astronomical” and “economically crippling”. As far as I know, there’s no screening program that can forestall Alzheimer’s disease in early life.
And honestly, I’m surprised by how rarely someone points out that the cost-saving justification for medical interventions is complete nonsense. Since when is “being able to pay for itself” a criterion of value in medical care? Let’s say we have two treatments, A and B; A costs nothing and has no clinical benefits, but receiving it prevents a patient from receiving B, which costs money but treats the patient’s illness effectively. This line of thinking would say that providing A is preferable, since it’s “saved” us the cost of B. This sounds dumb, of course, but that’s because we provide medical care to relieve illness and suffering, not for the purpose of saving money. If the latter is your goal, frankly, you’re better off providing no services at all. If, on the other hand, you recognize the patient’s relief as something of value, simple economics would suggest that you’ll have to exchange something of value to obtain it.
None of this is to say that preventive care is pointless. With an appropriately engaged patient and a clear discussion of their risks and benefits, these interventions can clearly help people to live longer, healthier lives. But consideration of the finer points of the doctor-patient relationship is a far cry from mandating that third parties provide an unlimited quality of services to unlimited numbers of people at no cost. If the end result of this latest effort at health reform is to massively subsidize the provision of services with no real benefit, it’ll be hard to look back on it as a success.