American Journal of Kidney Diseases
Volume 53, Issue 3 , Pages 377-388 , March 2009

Randomized and Observational Studies in Nephrology: How Strong Is the Evidence?

  • Tom Greene, PhD

      Affiliations

    • Corresponding Author InformationAddress correspondence to Tom Greene, PhD, Division of Clinical Epidemiology, University of Utah School of Medicine, 85 North Medical Dr, East Rm 201, Salt Lake City, UT 84122-5350

References 

  1. Sacks H, Chalmers TC, Smith H. Randomized versus historical controls for clinical trials. Am J Med. 1982;72:233–240
  2. Black N. Why we need observational studies to evaluate the effectiveness of health care. BMJ. 1996;312:1215–1218
  3. McKee M, Britton A, Black N, McPherson K, Sanderson C, Bain C. Interpreting the evidence: Choosing between randomised and nonrandomized studies. BMJ. 1999;319:312–315
  4. Collins R, MacMahon S. Reliable assessment of the effects of treatment on mortality and major morbidity, I: Clinical trials. Lancet. 2001;357:373–380
  5. Concato J, Shah N, Horwitz R. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med. 2000;342:1887–1892
  6. Benson K, Hartz AJ. A comparison of observational studies and randomized, controlled trials: Special articles. N Engl J Med. 2000;342:1878–1886
  7. Barton S. Which clinical studies provide the best evidence? (The best RCT still trumps the best observational study). BMJ. 2000;321:255–256
  8. MacMahon S, Collins R. Reliable assessment of the effects of treatment on mortality major morbidity (II: Observational studies). Lancet. 2001;357:455–462
  9. Pocock SJ, Elbourne DR. Randomized trials or observational tribulations?. N Engl J Med. 2000;342:1907–1909
  10. Dunn D. The dangers of inferring treatment effects from observational data: A case study in HIV infection. Control Clin Trials. 2002;23:106–110
  11. Laine C. Postmenopausal hormone replacement therapy: How could we have been so wrong?. Ann Intern Med. 2002;137:290
  12. Machens K, Schmidt-Gollwitzer K. Issues to debate on the Women's Health Initiative (WHI) Study (Hormone replacement therapy: An epidemiological dilemma?). Hum Reprod. 2003;18:1992–1999
  13. Prentice RL. Methodologic challenges in chronic disease population research. Biostatistics. 2001;2:365–381
  14. Greene T. Are observational studies ‘just as effective’ as randomized clinical trials?. Blood Purif. 2000;18:317–322
  15. Wolfe R. Observational studies are just as effective as randomized clinical trials. Blood Purif. 2000;18:323–326
  16. Taubes G: Do we really know what makes us healthy? New York Times September 16, 2007: 52-59, 74, 78, 80
  17. Humphrey LL, Chan BK, Sox HC. Postmenopausal hormone replacement therapy and the primary prevention of cardiovascular disease. Ann Intern Med. 2002;137:273–284
  18. Whittemore A, Valerie M. Observational studies and randomized trials of hormone replacement therapy: What can we learn from them?. Epidemiology. 2003;14:8–10
  19. Grodstein F, Stampfer MJ. The epidemiology of postmenopausal hormone therapy and cardiovascular disease. In:  Goldhaber SZ,  Ridker PM editor. Thrombosis and Thromboembolism. New York, NY: Dekker; 2002;p. 67–68
  20. Grodstein F, Clarkson T, Manson J. Understanding the divergent data on postmenopausal hormone therapy. N Engl J Med. 2003;348:645–650
  21. Writing Group for the Women's Health Initiative Investigators. Risk and benefits of estrogen plus progestin in healthy postmenopausal women: Principal results from the Women's Health Initiative randomized controlled trial. JAMA. 2002;288:321–333
  22. Stamper M, Colditz G. Estrogen replacement therapy and coronary heart disease: A quantitative assessment of the epidemiologic evidence. Prev Med. 1991;20:47–63
  23. Eknoyan G, Beck G, Cheung A, et al. Effect of dialysis dose and membrane flux in maintenance hemodialysis. N Engl J Med. 2002;347:2010–2019
  24. Collins A, Liao M, Umen A. Urea index (Kt/V) and other predictors of hemodialysis patient survival. Am J Kidney Dis. 1994;23:272–282
  25. Parker TF, Husni L, Huang W, Lew N, Lowrie EG. Survival of hemodialysis patients in the United States is improved with a greater quantity of dialysis. Am J Kidney Dis. 1994;23:670–680
  26. Held PJ, Port FK, Wolfe RA, et al. The dose of hemodialysis and patient mortality. Kidney Int. 1996;50:550–556
  27. Wolfe RA, Ashby VB, Agodoa LYC, Jones CA, Port FK. Body size, dose of hemodialysis and mortality. Am J Kidney Dis. 2000;35:80–88
  28. Port FK, Ashby VB, Dhingra RK, Roys EC, Wolfe RA. Dialysis dose and body mass index are strongly associated with survival in hemodialysis in patients. J Am Soc Nephrol. 2002;13:1061–1066
  29. Evidence-Based Medicine Working Group. Evidence-based medicine: A new approach to teaching the practice of medicine. JAMA. 1992;268:2420–2425
  30. Preventive Services Task Force. Guide to Clinical Preventive Services: Report of the U.S. Preventive Services Task Force. (ed 2). Baltimore, MD: Williams & Wilkins; 1996;
  31. Holland P. Statistics and causal inference. J Am Stat Assoc. 1986;81:945–960
  32. Rubin D. Estimating causal effects of treatments in randomized and nonrandomized studies. J Educ Psychol. 1974;66:688–701
  33. Pocock S. Clinical Trials: A Practical Approach. New York, NY: Wiley; 1983;
  34. Freidman L, Furberg C, DeMets D. Fundamentals of Clinical Trials. (ed 3). New York, NY: Springer; 1998;
  35. Grimes D, Schultz K. Bias and causal associations in observational research. Lancet. 2002;359:248–252
  36. Lawlor D, Smith G, Bruckdorfer K, Kundu D, Ebrahim S. Those confounded vitamins: What can we learn from the differences between observational versus randomised trial evidence?. Lancet. 2004;363:1724–1727
  37. Marshall J, Hoover G, Hoover D. Survivor treatment selection bias in observational studies: Examples from the AIDS literature. Ann Intern Med. 1996;11:999–1005
  38. Rothman KJ, Greenland S. Modern Epidemiology. (ed 2). Philadelphia, PA: Lippincott-Raven; 1998;
  39. Hernan M, Hernandez-Diaz S, Robins J. A structural approach to selection bias. Epidemiology. 2004;15:615–625
  40. Rubin D. Matched Sampling for Casual Effects. Cambridge, NY: Cambridge University; 2006;
  41. Kleinbaum D, Kupper L, Muller K, Nizam A. Applied Regression Analysis and Other Multivariable Methods. 3rd ed.. Pacific Grove, CA: Brooks/Cole Publishing Company; 2008;
  42. Grobbee DE, Hoes AW. Confounding and indication for treatment in evaluation of drug treatment for hypertension. BMJ. 1997;315:1151–1154
  43. Greene T, Daugirdas J, Depner T, et al. Association of achieved dialysis dose with mortality in the Hemodialysis Study: An example of “dose-targeting bias. ” J Am Soc Nephrol. 2005;16:3371–3380
  44. Lee Y, Ellenberg J, Hirtz D, Nelson K. Analysis of clinical trials by treatment actually received: Is it really an option?. Stat Med. 1991;10:1595–1605
  45. Besarab A, Bolton W, Browne J, et al. The effects of normal as compared with low hematocrit value in patients with cardiac disease who are receiving hemodialysis and epoetin. N Engl J Med. 1998;339:584–590
  46. Appel LJ, Bakris G, Douglas-Baltimore J, et al. AASK Research Group The relationship between achieved blood pressure and renal outcomes in the African American Study of Kidney Disease (AASK). J Am Soc Nephrol. 2002;13:423A;(abstr)
  47. Concato J, Horwitz R. Beyond randomised versus observational studies. Lancet. 2004;363:1660–1661
  48. Feinstein AR, Horwitz RI. Problems in the “evidence” of “evidence-based medicine. ” Am J Med. 1997;103:529–535
  49. US Renal Data System. USRDS 2008 Annual Report. Bethesda, MD: The National Institutes of Health, National Institute of Diabetes and Digestive and Kidney Diseases; 2008;
  50. Ioannidis J, Haidich A, Pappa M, et al. Comparison of evidence of treatment effects in randomized and nonrandomized studies. JAMA. 2001;286:821–830
  51. Greene T, Daugirdas J, Cheung A, et al. HEMO Study Group Indices derived from modeled urea distribution volume are associated with mortality in the HEMO Study. J Am Soc Nephrol. 2003;14:212A;(abstr)
  52. Rosenbaum P, Rubin D. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70:41–55
  53. Rubin D. Estimating causal effects from large data sets using propensity scores. Ann Intern Med. 1997;127:757–763
  54. Rosenbaum P, Rubin D. Reducing bias in observational studies using subclassification on the propensity score. J Am Stat Assoc. 1984;79:516–524
  55. Dehejia R. Estimating causal effects in nonexperimental studies. In:  Gelman A,  Meng X editor. Applied Bayesian Modeling and Causal Inference From Incomplete-Data Perspectives. New York, NY: Wiley; 2004;p. 25–35
  56. Hill J, Reiter J, Zanutto E. A comparison of experimental and observational analyses. In:  Gelman A,  Meng X editor. Applied Bayesian Modeling and Causal Inference From Incomplete-Data Perspectives. New York, NY: Wiley; 2004;p. 49–71
  57. Robins J, Hernan M, Brumback B. Marginal structural models and causal inference in epidemiology. Epidemiology. 2000;11:550–560
  58. Robins J, Hernan M. Estimation of the causal effects of time-varying exposures. In:  Fitzmaurice G,  Davidian M,  Verbeke G,  Molenberghs G editor. Longitudinal Data Analysis. New York, NY: Chapman & Hall/CRC; 2009;p. 553–599
  59. Reeves G, Cox D, Darby S, Whitley E. Some aspects of measurement error in explanatory variables for continuous and binary regression models. Stat Med. 1998;17:2157–2177
  60. Carroll R, Ruppert D, Stefanski L, Crainiceanu C. Measurement Error in Nonlinear Models: A Modern Perspective. (ed 2). New York, NY: Chapman & Hall; 2006;
  61. Angrist J, Imbens G, Rubin D. Identification of causal effects using instrumental variables. J Am Stat Assoc. 1996;87:328–336
  62. Brooks J, Irwin C, Hunsicker L, Flanigan M, Chrischille E, Pendergast J. Effect of dialysis center profit-status on patient survival: A comparison of risk-adjustment and instrumental variable approaches. Health Services Res. 2006;41:2267–2289
  63. Frangakas C, Rubin D. Addressing complications of intention-to-treat analysis in the combined presence of all-or-nothing treatment non-compliance and subsequent missing outcomes. Biometrika. 1999;86:365–389
  64. Gareen I. Noncompliance in cancer screening trials. Clin Trials. 2007;4:341–349
  65. Victoria C, Habicht J, Bryce J. Evidence-based public health: Moving beyond randomized trials. Am J Public Health. 2004;94:400–405
  66. Peterson JC, Adler S, Burkart JM, et al. Modification of Diet in Renal Disease Study Group Blood pressure control, proteinuria and the progression of renal disease: The Modification of Diet in Renal Disease Study. Ann Intern Med. 1995;123:754–762
  67. Wright J, Agodoa L, Contreras G, et al. AASK Study Group Achieved blood pressure control in the African American Study of Kidney Disease and Hypertension (AASK). Arch Intern Med. 2002;162:1636–1643
  68. Depner T, Daugirdas J, Greene T, et al. HEMO Study Group Dialysis dose and effect of gender and body size on outcome in the HEMO Study. Kidney Int. 2004;65:1386–1394
  69. Suki W, Zabaneh R, Cangiano J, et al. Effects of sevelamer and calcium-based phosphate binders on mortality in hemodialysis patients. Kidney Int. 2007;72:1130–1137
  70. Jafar TH, Stark PC, Schmid CH, et al. AIPRD Study Group Angiotensin-converting enzyme inhibition and progression of renal disease: Proteinuria as a modifiable risk factor for the progression of non-diabetic renal disease. Kidney Int. 2001;60:1131–1140
  71. Locatelli F, Pozzoni P, Di Filippo S. What are we expecting to learn from the MPO Study?. In:  Ronco C,  Brendolan A,  Levin N editor. Cardiovascular Disorders in Hemodialysis. Basel, Switzerland: Karger; 2005;p. 83–89
  72. Greene T. What did we learn from the HEMO Study? (Implications of secondary analyses). Contrib Nephrol. 2005;149:1–14
  73. Rothwell P. External validity of randomized controlled trials: To whom do the results of this trial apply?. Lancet. 2005;365:82–93
  74. Vandenbourcke J. When are observational studies as credible as randomized trials?. Lancet. 2004;363:1728–1731
  75. Prentice R, Langer R, Stefanick M, et al. Women's Health Initiative Investigators Combined analysis of Women's Health Initiative observational and clinical trial. Am J Epidemiol. 2006;163:589–599
  76. Port FK, Wolfe RA, Hulbert-Shearon TE, McCullough KP, Ashby VB, Held PJ. High dialysis dose is associated with lower mortality among women but not among men. Am J Kidney Dis. 2004;43:1014–1023

 Originally published online as doi:10.1053/j.ajkd.2008.12.011 on January 30, 2009.

PII: S0272-6386(08)01695-8

doi: 10.1053/j.ajkd.2008.12.001

American Journal of Kidney Diseases
Volume 53, Issue 3 , Pages 377-388 , March 2009

Article Tools

* Image Usage & Resolution