American Journal of Kidney Diseases

Developing a Set of Core Outcomes for Trials in Hemodialysis: An International Delphi Survey

Published:February 24, 2017DOI:


      Survival and quality of life for patients on hemodialysis therapy remain poor despite substantial research efforts. Existing trials often report surrogate outcomes that may not be relevant to patients and clinicians. The aim of this project was to generate a consensus-based prioritized list of core outcomes for trials in hemodialysis.

      Study Design

      In a Delphi survey, participants rated the importance of outcomes using a 9-point Likert scale in round 1 and then re-rated outcomes in rounds 2 and 3 after reviewing other respondents’ scores. For each outcome, the median, mean, and proportion rating as 7 to 9 (critically important) were calculated.

      Setting & Participants

      1,181 participants (202 [17%] patients/caregivers, 979 health professionals) from 73 countries completed round 1, with 838 (71%) completing round 3.

      Outcomes & Measurements

      Outcomes included in the potential core outcome set met the following criteria for both patients/caregivers and health professionals: median score ≥ 8, mean score ≥ 7.5, proportion rating the outcome as critically important ≥ 75%, and median score in the forced ranking question < 10.


      Patients/caregivers rated 4 outcomes higher than health professionals: ability to travel, dialysis-free time, dialysis adequacy, and washed out after dialysis (mean differences of 0.9, 0.5, 0.3, and 0.2, respectively). Health professionals gave a higher rating for mortality, hospitalization, decrease in blood pressure, vascular access complications, depression, cardiovascular disease, target weight, infection, and potassium (mean differences of 1.0, 1.0, 1.0, 0.9, 0.9, 0.8, 0.7, 0.4, and 0.4, respectively).


      The Delphi survey was conducted online in English and excludes participants without access to a computer and internet connection.


      Patients/caregivers gave higher priority to lifestyle-related outcomes than health professionals. The prioritized outcomes for both groups were vascular access problems, dialysis adequacy, fatigue, cardiovascular disease, and mortality. This process will inform a core outcome set that in turn will improve the relevance, efficiency, and comparability of trial evidence to facilitate treatment decisions.

      Index Words

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to American Journal of Kidney Diseases
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Chalmers I.
        • Bracken M.
        • Djulbegovic B.
        • Garattini S.
        • Grant J.
        • Gulmezoglu A.
        How to increase value and reduce waste when research priorities are set.
        Lancet. 2014; 383: 156-165
        • Chalmers I.
        • Glasziou P.
        Avoidable waste in the production and reporting of research evidence.
        Lancet. 2009; 374: 86-89
        • Macleod M.R.
        • Michie S.
        • Roberts I.
        • et al.
        Biomedical research: increasing value, reducing waste.
        Lancet. 2014; 383: 101-104
        • Bell C.M.
        • Chapman R.H.
        • Stone P.W.
        • Sandberg E.A.
        • Neumann P.J.
        An off-the-shelf help list: a comprehensive catalog of preference scores from published cost-utility analyses.
        Med Decis Making. 2001; 21: 288-294
        • Nissenson A.
        Improving outcomes for ESRD patients: shifting the quality paradigm.
        Clin J Am Soc Nephrol. 2014; 9: 430-434
        • Wong G.
        • Howard K.
        • Chapman J.
        • Pollock C.
        • Chadban S.
        • Salkeld G.
        How do people with chronic kidney disease value cancer-related quality of life?.
        Nephrology. 2012; 17: 32-41
        • Wyld M.
        • Morton R.
        • Hayen A.
        • Howard K.
        • Webster A.
        A systematic review and meta-analysis of utility-based quality of life in chronic kidney disease treatments.
        PLoS Med. 2012; 9: e1001307
        • Gargon E.
        • Gurung B.
        • Medley N.
        • Altman D.
        • Blazeby J.
        • Clarke M.
        Choosing important health outcomes for comparative effectiveness research: a systematic review.
        PLoS One. 2014; 9: e99111
        • Ghandi G.
        Patient-important outcomes in registered diabetes trials.
        JAMA. 2008; 299: 2543-2549
        • Boers M.
        • Kirwan J.
        • Wells G.
        • Beaton D.
        • Gossec L.
        • d'Agostino M.
        Developing core outcome measurement sets for clinical trials: OMERACT filter 2.0.
        J Clin Epidemiol. 2014; 67: 745-753
        • Svensson S.
        • Menkes D.
        • Lexchin J.
        Surrogate outcomes in clinical trials: a cautionary tale.
        JAMA Intern Med. 2013; 173 (612-612)
        • Yudkin J.
        • Lipska K.
        • Montori V.
        The idolatory of the surrogate.
        BMJ. 2011; 343: d7995
        • Halimi J.M.
        • Sautenet B.
        • Gatault P.
        • Roland M.
        • Giraudeau B.
        Renal endpoints in renal and cardiovascular randomized clinical trials: time for a consensus?.
        Fundam Clin Pharmacol. 2012; 26: 771-782
        • EVOLVE Trial Investigators
        Effect of cinacalcet on cardiovascular disease in patients undergoing dialysis.
        N Engl J Med. 2012; 367: 2482-2494
        • Palmer S.C.
        • Hayen A.
        • Macaskill P.
        • et al.
        Serum levels of phosphorus, parathyroid hormone, and calcium and risks of death and cardiovascular disease in individuals with chronic kidney disease: a systematic review and meta-analysis.
        JAMA. 2011; 305: 1119-1127
        • Palmer S.C.
        • Teixeira-Pinto A.
        • Saglimbene V.
        • et al.
        Association of drug effects on serum parathyroid hormone, phosphorus, and calcium levels with mortality in CKD: a meta-analysis.
        Am J Kidney Dis. 2015; 66: 962-971
        • Perkovic V.
        • Neal B.
        Trials in kidney disease–time to EVOLVE.
        N Engl J Med. 2012; 367: 2541-2542
        • Morton R.
        • Tong A.
        • Howard K.
        • Snelling P.
        • Webster A.
        The views of patients and carers in treatment decision making for chronic kidney disease: systematic review and thematic synthesis of qualitative studies.
        BMJ. 2010; 340: c112
        • Morton R.
        • Tong A.
        • Webster A.
        • Snelling P.
        • Howard K.
        Characteristics of dialysis important to patients and family caregivers: a mixed methods approach.
        Nephrol Dial Transplant. 2011; 26: 4038-4046
        • Saini P.
        • Loke Y.
        • Gamble C.
        • Altman D.
        • Williamson P.
        • Kirkham J.
        Selective reporting bias of harm outcomes within studies: findings from a cohort of systematic reviews.
        BMJ. 2014; 349: g6501
        • Clark M.
        Standardising outcomes for clinical trials and systematic reviews.
        Trials. 2007; 8: 39
        • Prinsen C.
        • Vohra S.
        • Rose M.
        • King-Jones S.
        • Ishaque S.
        • Bhaloo Z.
        Core Outcome Measures in Effectiveness Trials (COMET) initiative: protocol for an international Delphi study to achieve consensus on how to select outcome measurement instruments for outcomes included in a ‘core outcome set.'.
        Trials. 2014; 15: 247
        • Gorst S.L.
        • Gargon E.
        • Clarke M.
        • Blazeby J.M.
        • Altman D.G.
        • Williamson P.R.
        Choosing important health outcomes for comparative effectiveness research: an updated review and user survey.
        PloS One. 2016; 11: e0146444
        • Kirkham J.
        • Boers M.
        • Tugwell P.
        • Clarke M.
        • Williamson P.
        Outcome measures in rheumatoid arthritis randomised trials over the last 50 years.
        Trials. 2013; 14: 324
        • Dalkey N.
        • Helmer O.
        An experimental application of the DELPHI method to the use of experts.
        Manage Sci. 1963; 9: 458-467
        • Chiarotto A.
        • Terwee C.
        • Deyo R.
        • Boers M.
        • Lin C.
        • Buchbinder R.
        A core outcome set for clinical trials on non-specific low back pain: study protocol for the development of a core domain set.
        Trials. 2014; 15: 511
        • Harman N.
        • Bruce I.
        • Callery P.
        • Tierney S.
        • Sharif M.
        • O'Brien K.
        MOMENT–Management of Otitis Media with Effusion in Cleft Palate: protocol for a systematic review of the literature and identification of a core outcome set using a Delphi survey.
        Trials. 2013; 14: 70
        • Waters A.
        • Tudur Smith C.
        • Young B.
        • Jones T.
        The CONSENSUS study: protocol for a mixed methods study to establish which outcomes should be included in a core outcome set for oropharyngeal cancer.
        Trials. 2014; 15: 168
        • Tong A.
        • Manns B.
        • Hemmelgarn B.
        • et al.
        Standardised Outcomes in Nephrology - Haemodialysis (SONG-HD): study protocol for establishing a core outcome set in haemodialysis.
        Trials. 2015; 16: 364
        • Urquhart-Secord R.
        • Craig J.C.
        • Hemmelgarn B.
        • et al.
        Patient and caregiver priorities for outcomes in hemodialysis: an international nominal group technique study.
        Am J Kidney Dis. 2016; 68: 444-454
      1. Schunemann H, Brozek J, Oxman A. GRADE Handbook for Grading Quality of Evidence and Strength of Recommendation. 2009. Accessed February 2017.

        • Morton R.L.
        • Snelling P.
        • Webster A.C.
        • et al.
        Dialysis modality preference of patients with CKD and family caregivers: a discrete-choice study.
        Am J Kidney Dis. 2012; 60: 102-111
        • Tong A.
        • Manns B.
        • Hemmelgarn B.
        • et al.
        Establishing core outcome domains in hemodialysis: report of the Standardized Outcomes in Nephrology-Hemodialysis (SONG-HD) Consensus Workshop.
        Am J Kidney Dis. 2017; 69: 97-107
        • Lopez-Vargas P.A.
        • Tong A.
        • Howell M.
        • et al.
        Patient awareness and beliefs about the risk factors and comorbidities associated with chronic kidney disease - a mixed-methods study.
        Nephrology (Carlton). 2017; 22: 374-381
        • Breckenridge K.
        • Bekker H.L.
        • Gibbons E.
        • et al.
        How to routinely collect data on patient-reported outcome and experience measures in renal registries in Europe: an expert consensus meeting.
        Nephrol Dial Transplant. 2015; 30: 1605-1614
        • Flythe J.E.
        • Powell J.D.
        • Poulton C.J.
        • et al.
        Patient-reported outcome instruments for physical symptoms among patients receiving maintenance dialysis: a systematic review.
        Am J Kidney Dis. 2015; 66: 1033-1046
        • Purnell T.S.
        • Auguste P.
        • Crews D.C.
        • et al.
        Comparison of life participation activities among adults treated by hemodialysis, peritoneal dialysis, and kidney transplantation: a systematic review.
        Am J Kidney Dis. 2013; 62: 953-973
        • Chiarotto A.
        • Deyo R.A.
        • Terwee C.B.
        • et al.
        Core outcome domains for clinical trials in non-specific low back pain.
        Eur Spine J. 2015; 24: 1127-1142
        • Harman N.
        • Bruce I.
        • Kirkham J.
        • et al.
        The importance of integration of stakeholder views in core outcome set development: otitis media with effusion in children with cleft palate.
        PloS One. 2015; 10: e0129514