Methods:

Article Outline

• Clinical practice guidelines defined
• K/DOQI guiding principles
• Evidentiary basis for guidelines
  • Phase I: Work group member selection
  • Phase II: Targeting
  • Phase III: Literature review, selection, and abstraction
    • Critical appraisal method for articles concerning prognosis
    • Critical appraisal methods for articles concerning nutritional assessment
    • Critical appraisal methods for articles concerning nutritional treatment
    • Results of the systematic review
  • Phase IV: Formulation of guidelines
  • Phase V: Draft report with supporting rationale
  • Phase VI: Peer review
    • Stage one: Primary review
    • Stage two: Organizational review
    • Stage three: Open review
  • Phase vii: Issue final guidelines
• K/DOQI implementation planning

Back to Article Outline

Clinical practice guidelines defined 

The institute of Medicine has defined practice guidelines as “systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances.” The American Medical Association endorsed this definition by describing practice guidelines as “systematically developed statements, based on current professional knowledge, that assist practitioners and patients to make decisions about appropriate health care for specific clinical circumstances.” Put simply, practice guidelines constitute an effort to advise health-care providers and patients as to what constitutes optimal clinical practice, based on the best information available. As a result, practice guidelines can not only improve both quality and cost-effectiveness of care, but can also facilitate continuous improvement in clinical practice as new information becomes available.

Back to Article Outline

K/DOQI guiding principles 

Four principles guided decision-making in the conduct of the NKF-DOQI and will be retained for the K/DOQI guidelines:

Back to Article Outline

Evidentiary basis for guidelines 

The guidelines were developed using an evidence-based approach similar to the one used by The Federal Agency for Health Care Research and Quality (AHCRQ). That is, before formulating recommendations, the Work Groups reviewed all published evidence pertinent to the topics being considered and critically appraised the quality and strength of that evidence. For many issues that the Work Groups chose to address, there either was no pertinent literature available or available evidence was flawed or weak. As a result, in many instances the Work Groups formulated their recommendations based on the opinions of the Work Group members and comments received from the peer reviewers. In all instances, the Work Groups have documented the rationale for their recommendations. That is, they have articulated each link in the chain of logic they used as the evidentiary or opinion-related basis for their recommendation. This approach helps readers of the guidelines determine the quantity and quality of evidence underlying each recommendation.

Although some of the DOQI guidelines are clearly based entirely on evidence or entirely on opinion, many are based in part on evidence and in part on opinion. Such “hybrid” guidelines arise when some (or even most) of the links in the chain of logic underlying a guideline are based on empirical evidence, but some (ie, at least one) are based on opinion. The opinion of the Work Group members can enter the chain of logic that supports a guideline either to fill in a gap in available evidence on some scientific or clinical issue, or in the form of a value judgment regarding what they feel is appropriate clinical practice based on available evidence. Thus, many opinion-based guidelines may have substantial empirical evidence underlying them. These guidelines were developed using a seven-stage process.

Phase I: Work group member selection 

The DOQI Steering Committee selected a Chair to lead the Adult and Pediatric Nutrition Work Group and suggested names of individuals with particular expertise to serve on the Work Group. Final decisions on the membership of the Work Group were made by the Work Group Chair. In recognition of the different bodies of literature and expertise for nutrition issues in adult and pediatric ESRD and MD patients, the Work Group Chairs appointed separate nutrition Work Groups for adult and pediatric patients. Two Vice Chairs, for protein-energy nutrition and for carnitine, were appointed for the Adult Work Group, and one Vice Chair was appointed for the Pediatric Work Group.

Support for the Work Groups in coordinating and performing the systematic literature review, synthesizing data abstracted from the literature into evidence tables, facilitation of the guideline development process, conducting meetings of the Work Groups, and analyzing results of the guideline development meetings was provided by personnel from the RAND Corporation and Cedars-Sinai Medical Center. Both of these institutions are associated with the Southern California Evidence-Based Practice Center.

Phase II: Targeting 

The Work Groups defined the specific topics on which guidelines would focus and the specific questions on which the systematic literature would focus. The following clinical questions were formulated:

Question 1. Which of the following measures of nutritional status best predicts patient morbidity/mortality (and growth rate in children) in MD patients?

Serum albumin, serum prealbumin, anthropometric measures (height, weight, skinfold thickness, body mass index [BMI], percent of normal body weight, percent of desirable body weight, postdialysis body weight), bioelectrical impedance (BIA), urea nitrogen appearance, serum creatinine and creatinine index, subjective global nutritional assessment (SGA), dietary diaries and interviews, serum cholesterol, serum transferrin, serum IGF in pediatric patients, protein equivalent of total nitrogen appearance (PNA/PCR), prognostic nutrition index, serum acute-phase proteins (C-reactive protein), serum alpha-1 glycoprotein, dual energy x-ray absorptiometry (DXA), a combination of more than one of these measures.

Question 2. Which of the following measures is the best diagnostic test for protein/energy nutritional status in MD patients?

Serum albumin, serum prealbumin, anthropometric measures (height, weight, skinfold thickness, BMI, percent of normal body weight, percent of ideal body weight, postdialysis body weight), BIA, urea nitrogen appearance, serum creatinine and creatinine index, SGA, dietary diaries and interviews, serum cholesterol, serum transferrin, serum IGF, PNA, prognostic nutrition index, serum acute phase proteins (C-reactive protein), serum alpha-1 glycoprotein, DXA, a combination of more than one of these measures.

Question 3. What is the effect of acid/base status on nutritional measures in MD patients?

Question 4. Which levels of intake of protein and energy in MD patients produce the following:

The lowest morbidity/mortality, the most optimum changes in nutritional status using measures from Question 1 above, positive nitrogen balance, the most optimal growth in children?

Question 5. Which levels of protein and energy intake in predialysis patients produce the lowest morbidity at the initiation of dialysis? (This question was included because of evidence that nutritional status at the onset of MD therapy is a strong predictor of nutritional status and mortality during the course of MD therapy.)

Question 6. What is the energy expenditure of MD patients during resting and other activities, and how does it compare with energy expenditure in normal individuals?

Question 7. Is interdialytic weight gain a good measure for dietary compliance or a good prognostic indicator?

Question 8. Does carnitine supplementation in adult MD patients improve morbidity or mortality?

Question 9. What are the toxic/adverse effects of L-carnitine, if any, in adult MD patients?

Question 10. Which nutritional interventions produce the lowest morbidity/mortality (and best growth in children) or the most optimum changes in nutritional status in MD patients using measures from Question 1 above?

Question 11. Does GH therapy improve growth or morbidity/mortality in pediatric MD patients?

Question 12. Does vitamin or mineral supplementation (exclusive of calcium, magnesium, and vitamin D) improve morbidity/mortality in pediatric MD patients?

Phase III: Literature review, selection, and abstraction 

A structured database search of two computerized bibliographic databases (MEDLINE and EMBASE) was performed with the following specifications: language : English and non-English articles; dates : 1966 through 1997; subjects : human; article types : letters, editorials, reviews, case reports, and abstracts of meeting proceedings were excluded. The literature search was performed in collaboration with a librarian experienced in searching computerized bibliographic databases and performing “evidence-based” systematic reviews. The Journal of Renal Nutrition was hand-searched, because, at the time, it was not indexed in the bibliographic databases listed above. Additionally, referrals from DOQI Work Group members through August 1999 were reviewed.

After loading articles from MEDLINE, EMBASE, Work Group referrals, and the Sigma Tau bibliography into an electronic database, one reviewer performed an initial title review of these articles. Two independent reviewers then reviewed the abstracts of articles whose titles were selected. Selection disagreements were resolved by consensus. English language articles for which the abstracts were selected were then obtained and categorized based on the clinical question the article addressed. Two independent reviewers then reviewed these articles. Information was abstracted from the articles (see below) by one abstracter and verified by a second. Disagreements were resolved by consensus. Articles that were rejected at this stage were coded using the following codes:

In order to increase precision and reduce systematic errors, the language of manuscripts was not limited to English.^1,2 The English titles and English abstracts of foreign language articles, when available, were sent to all Work Group members for review. The abstracts of foreign language manuscripts were translated into English if any Work Group member thought that the paper might contribute positively to the evidence base. Selections were further based on study design. For prognostic articles, only those with prospective cohort or historical prospective cohort designs were included for further analysis. For assessment of nutritional status, only manuscripts in which a nutritional parameter was compared to a recognized standard nutritional measure or to a clinical outcome were included for further analysis. For manuscripts examining nutritional treatment, only those with a prospective design with concurrent controls were analyzed further. Because there were smaller numbers of these types of studies for carnitine treatment or pediatric renal nutrition, these requirements were not as rigidly applied for this literature.

After article abstraction (see below), evidence tables were produced from a subset of abstracted data elements and evaluated by the Work Group during meetings in Los Angeles in August 1998 (Adult Work Group), in October 1998 (Pediatric Work Group), and during a series of subsequent conference calls. The Work Group accepted or rejected articles based on the study design and methods and the adequacy with which it addressed the clinical questions. The final selected articles are indicated by an asterisk in the reference section. Other citations, that are not asterisked, were not used for guideline development, but were used to more fully explain the background or rationale for a guideline.

For each prognostic article, the following characteristics were ascertained³: (1) the study type; (2) the three main co-morbid conditions; (3) whether there was a representative and well-defined sample of patients at a similar phase in the course of disease; (4) the characteristics of the study population and dialysis procedures that might have affected the study results; (5) the duration of the follow-up period; (6) whether the outcomes were objective and the interpretation of the outcomes was unbiased; (7) whether adjustment was made for important known prognostic factors; and (8) the results of the study.

For each article concerning nutritional assessment, the following information was obtained^4,5: (1) the type of study; (2) the three main co-morbid conditions; (3) whether there was an independent blinded comparison with a reference (gold) standard; (4) the characteristics of the study population and the dialysis procedures that might have affected the study results; (5) whether the results of the nutritional measure that was studied influenced the decision to measure the reference standard; (6) whether characteristics and variety of the patients' standard is similar to those found in dialysis centers; (7) whether the test methodology are described well enough to be reproducible; and (8) the results of the study.

For each treatment article, the following information was obtained^6,7: (1) the type of study; (2) the three main co-morbid conditions; (3) the Jadad quality scores⁸; (4) the randomization score; (5) the double blind score; (6) the score for whether all patients were accounted for; (7) an intention-to-treat score; (8) whether the treatment groups were similar at baseline; (9) the characteristics of the study population, dialysis procedure, and other ancillary treatment that might have affected the study results; (10) whether the treatment groups were treated similarly except for the study intervention; and (11) the results of the study.

The Jadad quality scores address issues most important in demonstrating the validity of randomized clinical trials and have been demonstrated to reflect methodological quality. Empirical evidence demonstrates that when these quality features are not met in clinical trials, bias and an exaggeration of the effect sizes often result.^8-12

The initial literature search identified 19,272 MEDLINE and 4,943 EMBASE articles. In addition, the Work Groups referred 134 articles for review, and the Sigma Tau Pharmaceutical Corporation submitted a bibliography that contained 138 additional references that were included in the analysis. Of these 24,487 references, 22,362 titles were rejected as not meeting the inclusion criteria, leaving 2,125 titles. Abstracts of these articles were reviewed and 1,021 were rejected as not meeting the inclusion criteria, thus leaving 1,104 articles. One hundred and seventy of these were foreign language articles whose titles and abstracts were sent to the Adult or Pediatric Work Groups. Of these, 102 were not selected for further evaluation. Two were selected but could not be translated, and 66 were further evaluated. Of the remaining 1,000 manuscripts (including the 66 mentioned above), 29 were unobtainable, leaving 971 to be abstracted. Of these, 640 were rejected because they were classified as an editorial, letter, review, case report, or abstract, did not answer a clinical question of relevance, did not have a valid study design, or did not involve humans. The remaining 331 articles were sent to the Adult or Pediatric Work Groups along with evidence tables for these articles created from the abstraction forms. The Work Groups rejected 81 additional articles for one or more of the same reasons indicated above, leaving 250 accepted articles.

Phase IV: Formulation of guidelines 

The group process used to develop the guidelines is a modification of the RAND/UCLA Appropriateness Method. This group process method has the following essential features: multidisciplinary, iterative, quantitative, and each panelist has equal weight in determining the final result.¹³

In conjunction with the Work Groups, RAND and Cedars-Sinai staff developed draft guidelines based on the results of the systematic review. The draft guidelines corresponded to the key questions developed by each Work Group. The draft guidelines included all possible topics articulated by the Work Groups during the targeting phase and at the Work Group meetings to discuss the evidence. These draft guidelines were then transmitted to the Work Group members, who used the evidence tables and their expert judgment to rate each guideline statement for validity on a 1-to-9 scale. The RAND staff then compiled summaries for the face-to-face meetings of the Work Groups. At these meetings, Work Group members were provided with the summaries of these first round ratings of validity. These summary ratings were used to key a point-by-point discussion of the evidence and opinion surrounding each potential guideline statement. After each discussion, the Work Group members privately re-rated each guideline statement for validity. These votes form the basis for the final guidelines. Statements were accepted as valid if the median panel rating on validity was 7 or greater on the 1-to-9 scale. “Complete agreement” was defined as occurring when all Work Group members rated a guideline statement within the same three-point range of the scale (for example, all members' ratings were in the range of 7, 8, or 9). After determining the final guideline statements, Work Group members went through a similar two-step rating process to assess the level of evidence. A rating of “Evidence ” was defined as “mainly convincing scientific evidence, limited added opinion”; “Opinion ” was defined as “mainly opinion, limited scientific evidence”; and “Evidence plus Opinion ” was defined as “about equal mixtures of scientific evidence and opinion.”

Phase V: Draft report with supporting rationale 

Following the development of the guidelines, the Work Group drafted a report that included the supporting rationale for each guideline. While writing the rationale for each guideline, Work Group members cited additional references that had either not been identified previously in the literature search efforts, or had been identified but rejected. These citations contained information that was felt to be important either as background material, or to further explain the rationales. However, these additional references were not part of the evidence base that was used to either formulate the guideline statements or the votes on the validity or the rating of evidence versus opinion for each guideline.

Phase VI: Peer review 

The purpose of the peer review process was to identify:

The nutrition guidelines were subjected to a three-stage peer review process:

NKF-DOQI's multidisciplinary Steering Committee was assigned to review the draft report. Drafts were distributed to the committee in August 1999 and members had the opportunity to offer oral comments at a face-to-face meeting in mid-September. The draft report was also sent to the NKF-DOQI Advisory Council, the NKF Scientific Advisory Board, and selected experts in the field. Many substantive comments were received, and this resulted in substantive changes in the organization and content of some of the guidelines and rationales. Given the large volume of comments received, the Work Group vice-chairs reviewed the comments first and entered them into a computer database separating these according to whether they had a potential minor or substantive impact. Comments were sorted by guideline topic and then provided to the Work Groups for analysis and response.

Close to 200 individuals representing nearly 50 end-stage renal disease (ESRD)-related organizations reviewed the second draft of the guidelines in December 1999. Organizations that were invited to participate in the second round of peer review were selected by the Steering Committee based on suggestions from the Advisory Council and the Work Groups. Organizations included various nephrology professional societies (eg, Renal Physicians Association, American Society of Nephrology, American Nephrology Nurses Association, and American Renal Administrators Association), the American Association of Kidney Patients, the ESRD Networks, NKF Councils, dialysis chains, managed care organizations, and private industry organizations selected their own reviewers.

In the final round of review, in December 1999, approximately 400 individuals received copies of the revised draft guidelines. Within 3 weeks, 30% of these reviewers provided comments. The Work Group vice-chairs sorted and organized these comments and the Work Group analyzed the responses.

Phase vii: Issue final guidelines 

The Work Group and staff performed several tasks to complete the guidelines. The guidelines were edited to ensure clarity and consistency. The Work Group carefully reviewed the final draft and made the indicated changes. Accuracy of the literature citations for each guideline document were also verified.

Back to Article Outline

K/DOQI implementation planning 

The NKF plans to undertake three types of activities to promote implementation of these recommendations.

The development of the K/DOQI practice guidelines is a cooperative, rewarding, and unifying effort for the participants and the community of health care workers who are involved in the care of the individual with kidney disease. We hope this spirit of cooperation and commitment to improvement of dialysis patient outcomes will help the K/DOQI in efforts to put its quality improvements into practice.