Case #1: Cardiac rehabilitation program

Suppose a proposal is made to offer a cardiac rehabilitation program of uncertain effectiveness to people who have experienced a heart attack.  Following are five statements derived from five randomized trials published in medical journals. On the basis of each, please take a moment ton indicate how likely you are to agree to the implementation of a cardiac rehabilitation program based on this information. For the sake of the example, assume that the costs of each program are the same and that each result was deemed to be statistically significant. Please rate from 0 (would not) to 10 (would) the degree of support you would be willing to recommend each of the following programs if during a 10-year follow-up [Slide1]:

1. Program A reduced the death rate by 13%.
2. Program B produced an absolute reduction in deaths of 4%
3. Program C increased the rate of patient survival from 70% to 74%
4. Program D prevented one death for every 25 people who entered a rehabilitation program.
5. Program E reduced the odds of death by 18%.

      Case #2: Breast screening program

Suppose a proposal is made to offer a breast screening program of uncertain effectiveness to women older than 50 years-old [Slide2]. Following are five statements derived from five randomized trials published in medical journals. On the basis of each, indicate how likely you are to agree to the implementation of a breast screening program. Assume that the costs of each program are the same and each result was deemed to be statistically significant. Rate from 0 (would not) to 10 (would) the degree of support you would be willing to give the screening program if during a 7-year-follow-up [Slide3]:

1. Program A reduced the death rate by 34%
2. Program B produced an absolute reduction in deaths from breast cancer of 0.06%
3. Program C increased patient survival rate of breast cancer from 99.82% to 99.88%
4. Program D prevented one death for every 1,588 individuals screened.
5. Program E reduced the odds of death by 34%.

The results of the five programs in Case 1 are typical of how the results of new research studies are reported in professional medical journals and subsequently quoted in the popular press. In fact these are actual results taken from the published medical literature. The translation of research findings into improved quality and efficiency of care is critical for research to be of ultimate practical value. How findings like those in Case 1 are interpreted is critical when physicians and patients male health care decision.  The underlying assumption is to provide the highest quality, most effective health care given a particular patient’s situation. What may not be evident is that all five results presented in Case 1 are derived from exactly the same underlying data.

The two problems introduced in Cases 1 and 2 typify the language used to report new clinical research findings and were modified from those presented by Fahey and colleagues. What may not be so readily apparent is that the five results for each case are actually alternative representations of the same research findings. This fact implies that your preference for each intervention, whether therapy or screening, should be the same regardless of how the information is presented or framed.

Results of studies using problems like these and others suggest that the manner in which information is framed can affect the decisions people make and that preference reversals do occur when the same “evidence” is presented in different ways. These studies suggest that people are significantly more supportive of treatments when outcomes are expressed in terms of relative risk reduction but less supportive when outcomes are expressed as the absolute risk difference or the number of patients needed to treat to prevent one patient from experiencing and adverse event. We address how these preference reversals might occur after explaining how the various representations of the same results are derived.

The study results reporting various measures of effect in Cases 1 and 2 are derived from 2 x 2 tables that can be constructed from data provided in the original articles. For illustration purposes, the 2 x 2 table for Case 2 is provided in table 1.

Table 1

The table is based on results reported in the original Scandinavian multinational randomized trial conducted by Tabar et al. that was designed to examine the effectiveness of mammography screening in reducing breast cancer-specific deaths after 7 years of follow-up. These effect size measures are (1) relative risk reduction, (2) risk difference, (3) survival rates, (4) number of patients needed to treat (or screen) to prevent one adverse event, and (5) reduction in odds.

 Program A reduced the death rate by 34%. This is the common language used to describe what is more technically referred to as the relative risk reduction. To determine the relative risk, simply divide (or take the ratio of) the risk of an adverse outcome for people randomized to one group by the risk of an adverse event for people randomized to the other group and then subtract this from one to estimate the reduction in relative risk [Slide4]:

Risk (mammography) = 71/58,148 = 0.00122

Risk (controls) = 76/41,104 = 0.00185

Relative risk = 0.00122/0.00185 = 0.659

Relative risk reduction = 1 – 0.659 = 0.341 = 34%

Program B produced an absolute reduction in deaths from breast cancer of 0.06%. Rather than take the radio of the risks for the two groups, simply subtract the risk difference, which is sometimes referred to as the absolute risk reduction. [Slide5]:

 Risk (mammography) = 71/58,148 = 0.00122

Risk (controls) = 76/41,104 = 0.00185

Risk difference = 0.00122 – 0.00185 =  - 0.00063 = - 0.06%

Program C increased the patient survival rate of breast cancer from 99.82% to 99.88%.  Divide the number of patients not experiencing the adverse event by the total number of people in that group and take the difference between the two groups:

Survival rate (mammography) = 58,077/58,148 = 0.9988 = 99.88%

Survival rate (controls) = 41,028/41,104 = 0.9982 = 99.82%

Program D prevented one death for every 1,588 individuals screened. The number of patients needed to be treated or screened to prevent one adverse outcome is simply the inverse of the absolute risk difference [Slide6]:

 Risk difference = 0.00122 – 0.00185 = - 0.00063 (0.06%)

Risk difference = absolute risk reduction

Number needed to treat = 1 / (absolute risk reduction)

Number needed to treat = 1 / 0.00063 = 1,588

The number needed to treat can be adjusted for the underlying risk group that is most representative of a particular patient’s characteristics and risk if the patient were to go untreated (or receive standard care).

Program E reduced the odds of death by 34%. Estimate the odds of an adverse event for each group by dividing the number of adverse events for each group by the number of people not experiencing that adverse event in that group and then take the ratio of these odds to estimate the odds ratio. The reduction in odds is simply one minus the odds ratio [Slide7]:

Odds (mammography) = 71 / 58,077 = 0.00122

Odds (controls) = 76 / 41,028 = 0.00185

Odds ratio = 0.00122 / 0.00185 = 0.659

Odds reduction = 1 – 0.659 = 0.341 = 34%

This breast cancer screening problem illustrates the special case in which the risk and odds are identical. This only occurs when the number of events is small and the sample sizes are large for both groups, because it does not matter whether the number of events (e.g., deaths) in patients randomized to one group or another (e.g., mammography group) is divided by the total number of people in that group (to calculate relative risk) or by the total number of people who did not experience the event (i.e., were still living), because these numbers are both large and tend to approximate each other [Slide8] .

 So why does this matter? People, including physicians, generally tend to confuse odds with risks. It is argued that the concept of risk is more easily understood than odds and therefore is a more desirable indicator to use to communicate the degree of effectiveness of an intervention. Odds, however, have certain statistical properties that make their calculation and use easier (e.g., in multivariate prediction modeling), whereas calculating “risks” may not be appropriate in some circumstances – for example, in the estimation of effects based on results from case0 control studies. Estimates of risk should not be based on results from case-control studies because it is possible to obtain “any value we like for the risk by varying the number of cases and controls that we choose to study. That is, the investigators choose the number of patients to include in each group and can inflate or deflate the denominator, which does not allow us to calculate an unbiased estimate of risk. All this is made even more problematic because of the well-know difficulty people (both lay and professional) have in understanding statistical and probabilistic concepts.