Home About us Editorial board Ahead of print Current issue Search Archives Submit article Instructions Subscribe Contacts Login 
5 years IF: 1.036 (® Thomson Reuters)
IF 2016: 1.064 (® Thomson Reuters)
Total Cites: 7140
Follow Us
Follow Us
  • Users Online: 1368
  • Home
  • Print this page
  • Email this page

 Table of Contents  
Year : 2017  |  Volume : 130  |  Issue : 8  |  Page : 993-996

Postrandomization Confounding Challenges the Applicability of Randomized Clinical Trials in Comparative Effectiveness Research

1 Center for Clinical Epidemiology and Evidence-based Medicine, Beijing Children's Hospital, Capital Medical University, Beijing 100045; Key Laboratory of Major Diseases in Children, Ministry of Education, Beijing 100045, China
2 Center for Clinical Epidemiology and Evidence-based Medicine, Beijing Children's Hospital, Capital Medical University, Beijing 100045, China
3 Center for Medical Safety and Risk Administration, National Institute of Hospital Administration, Beijing 100091, China

Date of Submission12-Jan-2017
Date of Web Publication10-Apr-2017

Correspondence Address:
Prof. Xiao-Xia Peng
Center for Clinical Epidemiology and Evidence-based Medicine, Beijing Children's Hospital, Capital Medical University, Beijing 100045
Login to access the Email id

Source of Support: None, Conflict of Interest: None

DOI: 10.4103/0366-6999.204117

Rights and Permissions

Keywords: Comparative Effectiveness Research; Postrandomization Confounding; Randomized Clinical Trials

How to cite this article:
Peng YG, Nie XL, Feng JJ, Peng XX. Postrandomization Confounding Challenges the Applicability of Randomized Clinical Trials in Comparative Effectiveness Research. Chin Med J 2017;130:993-6

How to cite this URL:
Peng YG, Nie XL, Feng JJ, Peng XX. Postrandomization Confounding Challenges the Applicability of Randomized Clinical Trials in Comparative Effectiveness Research. Chin Med J [serial online] 2017 [cited 2017 Nov 24];130:993-6. Available from: http://www.cmj.org/text.asp?2017/130/8/993/204117

  What Is the Problem? Top

Different from trials for regulatory approvals of new interventions aimed to test the efficacy, comparative effectiveness research (CER) is the direct comparison of existing health-care interventions (compared with active controls) to examine which treatment works best, for whom, and under what settings.[1] Therefore, CER is indispensable to assist consumers, clinicians, purchasers, and policy makers to make evidence-based decisions. Actually, an obvious uprising tendency of CER after 2004 was displayed and a majority of CER was performed by randomized clinical trials (RCTs) with active controlled interventions.

It is known to us that major advantages of RCTs to demonstrate causality include low risk of selective bias and minimized the influence of baseline confounding by randomly assigning the intervention, low risk of performance bias, or detection bias due to blinding. Whatever Phase II and III clinical trials or CER, properly designed and carried out RCTs can provide the most definitive causal inference. Compared with RCTs of Phase II and III, which can produce persuasive causal inference, the challenge for CER should be considered cautiously because the cleanest comparison occurs only when standard interventions are performed, i.e., there are no unexpected co-interventions, such as medications, supplementary therapies, and behaviors during trials. RCTs are only expected to be free from baseline confounding but not from postrandomization confounding when we study the long-term effects of sustained clinical interventions in typical patients and care settings.[2]

On the other hand, the control group in CER receives active treatments and aims to examine the effectiveness of intervention rather than efficacy so that it is difficult to give sustained standard interventions like that in clinical trials. The baseline equilibrium after randomization could be broken and the disequilibrium of interventions among different trial arms might occur or be magnified due to uncontrollable concomitant therapies. For example, unexpected concomitant therapies can occur when patients admitted in the Intensive Care Unit are enrolled as participants. This may impact the estimation of effect size due to postrandomization confounding.

Therefore, we attempt to more clearly define postrandomization confounding so that we are able to give serious consideration to this potential bias resource. The postrandomization confounding could be considered as an error introduced by disequilibrium concomitant therapies after a random assignment in RCTs, especially in trials aiming to compare effectiveness rather than efficacy. The impact of postrandomization confounding on causal inference is shown in [Figure 1].
Figure 1: The impact of postrandomization confounding on causal inference in comparative effectiveness research.

Click here to view

  Current Examples of the Postrandomization Confounding Top

Some examples of the postrandomization confounding in CER based on RCTs design we discussed here will help us to propose a viewpoint that the postrandomization confounding could actually impact results of CER. We should not place undue emphasis on the known advantages attributing to randomly assigning and blinding, while ignoring other potential bias when we are designing the RCTs for CER. Moreover, we hope to cause concerns on how to design and report cautiously CER.

In this view, one RCT on Chinese traditional medicine acupuncture for seasonal allergic rhinitis (SAR) presented potential postrandomization confounding by concomitant medicine therapies. Participants diagnosed with SAR were allocated randomly into real acupuncture (RA) or sham acupuncture (SA) with a good design randomization and blinded procedure to assess the primary outcome, i.e., the severity of SAR symptoms. Some symptom relief medications, short-acting antihistamine loratadine, or a decongestant nasal spray were required for participants when needed. Four weeks of acupuncture treatment was assessed as a safe and effective option for clinical management of SAR due to the reduction scores of SAR symptom severity measurements.[3] However, the efficacy of the acupuncture might also be underestimated due to the potential bias introduced from postrandomization confounding of concomitant medicine. [Figure 2] showed the weekly symptom relief medication score, which was calculated by one tablet loratadine equating to one point and two sprays per nostril of oxymetazoline equaled one point also and the decreased spector sneezing score, one of the symptoms severity primary outcomes, from baseline till follow-up. From the third treatment period, the mean difference of weekly symptom relief medication score between groups was enlarged about one point, which indicated the participants in RA group applied less medication to relieve the symptoms compared with those in SA group. Meanwhile, the reduction of symptom severity was significant (P = 0.02 for sneezing score and 0.006 for itchiness of ears and palate score separately) in RA group during the same period. It could be inferred that benefit of decreased symptom score might be estimated conservatively in this trial because the mean difference of symptom score between RA and SA group could be traded off by more weekly symptom relief medication in SA group, which may be also the bias by postrandomization confounding.
Figure 2: Weekly symptom relief medication score (left) and symptom severity over time using the spector sneezing score (right) as an example.

Click here to view

In another large-scale multicenter RCT, the Systolic Blood Pressure Intervention Trial (SPRINT), interventions were aimed at antihypertension and examination of the ratio of benefit to harm by different blood pressure (BP) target values. Logically, the groups were divided according to the BP target values (<140 mmHg in standard treatment group vs. <120 mmHg in the intensive treatment group) rather than different interventions.[4] SPRINT showed targeting a systolic blood pressure of <120 mmHg, compared with <140 mmHg, reduce the rates of major cardiovascular events (MACE) and death from any cause.

Based on common sense, it is imperative to standardize the pharmaceuticals algorithms. Therefore, all antihypertensive medications administration in these two studies was evidence-based. For example, angiotensin-converting enzyme-I (ACE-I) and angiotensin receptor blocker (ARB) agents were interchangeably recommended as the first-line treatments for antihypertension by guidelines.[5],[6],[7] Evidence that ACE-I [8],[9],[10],[11] and ARB [12],[13] prevented cardiovascular events and death showed these kinds of medication could influence the risk of MACE. Strikingly, the appendix results of SPRINT showed the proportion of more than three antihypertensive agents using were 56.1% in the intensive group and 24.1% in the standard group at most recent visit.[4] Moreover, one additional antihypertensive drug on average was used in the intensive treatment group. At the same time, there were different in ACE-I using (76.7% vs. 55.2%) and ARB using (8.7% vs. 4.0%).[4] The disequilibrium of concomitant drugs after randomization leads us to doubt that the efficacy of reduced MACE in SPRINT could be clearly attributed to the target BP values or to the effects of antihypertensive agents against MACE by itself. The apparent difference of pharmaceuticals algorithms between groups could introduce the potential postrandomization confounding.

  What Needs to Happen Next? From Extensive Use to Wise Use Top

As the upper level of evidence hierarchy, RCTs always have been a competitive and persuasive study design so that more and more stakeholders preferred it. However, it is really not almighty for any setting. In recent decades, the methodology of RCTs is indeed improved with the progress of the clinical study and much more normatively statistical approaches, whatever in design, performance, statistical analysis, reporting, even assessment of the risk of bias. Controls, randomization, and blinding, these three main principles were almost focused and noted in RCTs, but other conditional settings might be omitted, such as clean comparison with standardized interventions and follow-up procedures. Postrandomization confounding was one of the potential biases introduced by ignorance of conditioned on variables.

RCTs could be still regarded as the first choice to compare the effectiveness among different interventions because RCTs can provide the most definitive causal inference. What we proposed is that RCTs might be more applicable in strict premarket clinical trials, rather than universal, even abused, especially in some CERs. The following scenarios should be more cautiously considered when RCTs were designed for CER. First, the results of CER might be challengeable when some unavoidable supplementary therapies or behaviors will occur after randomization, such as symptom relief medications, which was not constrained but was designed as a secondary outcome in RCTs aimed to evaluate the effectiveness of acupuncture against SAR. Second, the bias would be more likely to occur in long-term trials conducted in critical care settings [14] or in long-term chronic disease prevention [2],[15] because it is difficult to sustain clean intervention to deal with unexpected disease progression. Third, in some special conditions, such as that of SPRINT study, trial arms are determined by target value of BP rather than medical interventions so that it is predestined that disequilibrium of treatments or behaviors for the targets could occur. This would confuse us that the effect size of RCTs is attributed to different target values or difference of medicine.

Therefore, the potential confounding and bias due to postrandomization confounding in CER should be considered more cautiously and reported. Some restrictions of the concomitant therapies or stratification in both assigning and statistical analysis could be considered to deal with this problem. Moreover, meticulous design for the target values settings would be helpful to get sound results. Most important, a standard framework for design, performance, and reporting of CER by RCTs will be required to control this kind of confounding.

Financial support and sponsorship


Conflicts of interest

There are no conflicts of interest.

  References Top

Greenfield S, Rich E. Welcome to the journal of comparative effectiveness research. J Comp Eff Res 2012;1:1-3. doi: 10.2217/cer.11.13.  Back to cited text no. 1
Hernán MA, Hernández-Díaz S, Robins JM. Randomized trials analyzed as observational studies. Ann Intern Med 2013;159:560-2. doi: 10.7326/0003-4819-159-8-201310150-00709.  Back to cited text no. 2
Xue CC, Zhang AL, Zhang CS, DaCosta C, Story DF, Thien FC. Acupuncture for seasonal allergic rhinitis: A randomized controlled trial. Ann Allergy Asthma Immunol 2015;115:317-24.e1. doi: 10.1016/j.anai.2015.05.017.  Back to cited text no. 3
SPRINT Research Group, Wright JT Jr., Williamson JD, Whelton PK, Snyder JK, Sink KM, et al. A randomized trial of intensive versus standard blood-pressure control. N Engl J Med 2015;373:2103-16. doi: 10.1056/NEJMoa1511939.  Back to cited text no. 4
James PA, Oparil S, Carter BL, Cushman WC, Dennison-Himmelfarb C, Handler J, et al. 2014 evidence-based guideline for the management of high blood pressure in adults: Report from the panel members appointed to the Eighth Joint National Committee (JNC 8). JAMA 2014;311:507-20. doi: 10.1001/jama.2013.284427.  Back to cited text no. 5
Mancia G, Fagard R, Narkiewicz K, Redón J, Zanchetti A, Böhm M, et al. 2013 ESH/ESC Guidelines for the management of arterial hypertension: The Task Force for the management of arterial hypertension of the European Society of Hypertension (ESH) and of the European Society of Cardiology (ESC). J Hypertens 2013;31:1281-357. doi: 10.1097/01.hjh.0000431740.32696.cc.  Back to cited text no. 6
Eckel RH, Jakicic JM, Ard JD, de Jesus JM, Houston Miller N, Hubbard VS, et al. 2013 AHA/ACC guideline on lifestyle management to reduce cardiovascular risk: A report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. Circulation 2014;129 25 Suppl 2:S76-99. doi: 10.1161/01.cir.0000437740.48606.d1.  Back to cited text no. 7
Yusuf S, Sleight P, Pogue J, Bosch J, Davies R, Dagenais G. Effects of an angiotensin-converting-enzyme inhibitor, ramipril, on cardiovascular events in high-risk patients. The Heart Outcomes Prevention Evaluation Study Investigators. N Engl J Med 2000;342:145-53. doi: 10.1056/NEJM200001203420301.  Back to cited text no. 8
Fox KM; EURopean Trial on Reduction of Cardiac Events with Perindopril in Stable Coronary Artery Disease Investigators. Efficacy of perindopril in reduction of cardiovascular events among patients with stable coronary artery disease: Randomised, double-blind, placebo-controlled, multicentre trial (the EUROPA study). Lancet 2003;362:782-8. doi: 10.1016/S0140-6736(03)14286-9.  Back to cited text no. 9
Savarese G, Costanzo P, Cleland JG, Vassallo E, Ruggiero D, Rosano G, et al. A meta-analysis reporting effects of angiotensin-converting enzyme inhibitors and angiotensin receptor blockers in patients without heart failure. J Am Coll Cardiol 2013;61:131-42. doi: 10.1016/j.jacc.2012.10.011.  Back to cited text no. 10
van Vark LC, Bertrand M, Akkerhuis KM, Brugts JJ, Fox K, Mourad JJ, et al. Angiotensin-converting enzyme inhibitors reduce mortality in hypertension: A meta-analysis of randomized clinical trials of renin-angiotensin-aldosterone system inhibitors involving 158,998 patients. Eur Heart J 2012;33:2088-97. doi: 10.1093/eurheartj/ehs075.  Back to cited text no. 11
ONTARGET Investigators, Yusuf S, Teo KK, Pogue J, Dyal L, Copland I, et al. Telmisartan, ramipril, or both in patients at high risk for vascular events. N Engl J Med 2008;358:1547-59. doi: 10.1056/NEJMoa0801317.  Back to cited text no. 12
Bangalore S, Kumar S, Wetterslev J, Messerli FH. Angiotensin receptor blockers and risk of myocardial infarction: Meta-analyses and trial sequential analyses of 147 020 patients from randomised trials. BMJ 2011;342:d2234. doi: 10.1136/bmj.d2234.  Back to cited text no. 13
Ma PL, Peng XX, Du B, Hu XL, Gong YC, Wang Y, et al. Sources of heterogeneity in trials reporting hydroxyethyl starch 130/0.4 or 0.42 associated excess mortality in septic patients: A systematic review and meta-regression. Chin Med J 2015;128:2374-82. doi: 10.4103/0366-6999.163387.  Back to cited text no. 14
Manson JE, Shufelt CL, Robins JM. The potential for postrandomization confounding in randomized clinical trials. JAMA 2016;315:2273-4. doi: 10.1001/jama.2016.3676.  Back to cited text no. 15


  [Figure 1], [Figure 2]


Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
Access Statistics
Email Alert *
Add to My List *
* Registration required (free)

  In this article
What Is the Problem?
Current Examples...
What Needs to Ha...
Article Figures

 Article Access Statistics
    PDF Downloaded39    
    Comments [Add]    

Recommend this journal