Hand's paradox
In statistics, Hand's paradox arises from ambiguity when comparing two treatments. It shows that a comparison of the effects of the treatments applied to two independent groups that can contradict a comparison between the effects of both treatments applied to a single group.
Paradox
Comparisons of two treatments often involve comparing the responses of a random sample of patients receiving one treatment with an independent random sample receiving the other. One commonly used measure of the difference is then the probability that a randomly chosen member of one group will have a higher score than a randomly chosen member of the other group. However, in many situations, interest really lies on which of the two treatments will give a randomly chosen patient the greater probability of doing better. These two measures, a comparison between two randomly chosen patients, one from each group, and a comparison of treatment effects on a randomly chosen patient, can lead to different conclusions.
This has been called Hand's paradox,[1][2] and appears to have first been described by David J. Hand.[3]
Examples
Example 1
Label the two treatments A and B and suppose that:
Patient 1 would have responded values 2 and 3 to A and B respectively. Patient 2 would have responded values 4 and 5 to A and B respectively. Patient 3 would have responded values 6 and 1 to A and B respectively.
Then the probability that the response to A of a randomly chosen patient is greater than the response to B of a randomly chosen patient is 6/9 = 2/3. But the probability that a randomly chosen patient will have a greater response to A than B is 1/3. Thus a simple comparison of two independent groups may suggest that patients have a higher probability of doing better under A, whereas in fact patients have a higher probability of doing better under B.
Example 2
Suppose we have two random variables, [math]\displaystyle{ x_A \sim N(1,1) }[/math] and [math]\displaystyle{ x_B \sim N(0,1) }[/math], corresponding to the effects of two treatments. If we assume that [math]\displaystyle{ x_A }[/math] and [math]\displaystyle{ x_B }[/math] are independent, then [math]\displaystyle{ \Pr(x_A \gt x_B) = 0.76 }[/math], suggesting that A is more likely to benefit a patient than B. In contrast, the joint distribution which minimizes [math]\displaystyle{ Pr(x_A\gt x_B ) }[/math] leads to [math]\displaystyle{ Pr(x_A\gt x_B )\ge 0.38 }[/math]. This means that it is possible that in up to 62% of cases treatment B is better than treatment A.
References
- ↑ Fay MP, Brittain EH, Shih JH, Follmann DA, and Gabriel EE (2018) Causal estimands and confidence intervals associated with Wilcoxon-Mann-Whitney tests in randomized experiments. Statistics in Medicine, 37, 2923-2937.doi:10.1002/sim.7799
- ↑ Greenland S., Fay M.P., Brittain E.H., Shih J.H., Follmann D.A., Gabriel E.E, & Robins J.M. (2020) On Causal Inferences for Personalized Medicine: How Hidden Causal Assumptions Led to Erroneous Causal Claims About the D-Value, The American Statistician, 74:3, 243-248, DOI: 10.1080/00031305.2019.1575771 doi:10.1080/00031305.2019.1575771
- ↑ Hand D.J. (1992) On comparing two treatments, The American Statistician, 46, 190–192.doi:10.1080/00031305.1992.10475881
 |  |
