Predicting Outcomes in CKD

Part of medicine can be viewed as a science of predictions. For example, in a given geographic location, a physician could make a reasonable estimate of the risk that an individual entering his/her office has a particular disease (ie, based on the prevalence of that disease in that region) even without asking specific questions or conducting any diagnostic tests. Learning a patient's age and sex further defines this risk. Physicians can then use their knowledge and experience, results of diagnostic testing, and perhaps the patient's response to specific medications to increase or decrease their estimate of the risk of that individual truly having the disease. In another situation, clinicians may be interested in predicting outcome events among a specific patient population. Two broad aims of such prediction modeling can be distinguished: first, the identification of factors that associate with an outcome of interest, which may eventually lead to the identification of specific causes of disease and development of preventive strategies by targeting individual factors; and second, the identification of patients who are at particular risk for the outcome, which may lead to changes in the management of this patient group. These 2 distinct scientific questions translate into different methodological approaches.1, 2, 3, 4

When the aim is to identify individual determinants for an outcome of interest, the focus lies on the strength of association between individual factors and the outcome. One may initially limit the set of potential determinants to factors that are plausibly linked to the disease of interest or to factors that can be modified. When the aim is to classify patients into risk groups for the outcome, the focus lies on improving the ability to distinguish between patients who will or will not experience the outcome or to classify patients correctly into risk levels for the outcome. While these 2 approaches may identify a similar core set of determinants, individual factors may have a strong relationship with the outcome but may not improve the discrimination between individual patients or the classification of patients into risk levels.4, 5, 6 In this issue of the American Journal of Kidney Diseases, 2 articles present prediction models, one aiming to identify and describe individual determinants, and one aiming to classify patients into risk levels for adverse outcomes among patients with chronic kidney disease.

Levin and colleagues7 focus on the question of identifying individual predictors for kidney disease progression or death among referred patients with stage 4 chronic kidney disease. The authors utilized a large provincial registry to identify a subset of patients who reached an estimated glomerular filtration rate of less than 30 mL/min/1.73 m2. Using a backward elimination approach, the authors developed a model that predicted loss of kidney function from a large set of variables containing information about personal characteristics, medical history, and laboratory values. The following variables significantly predicted the outcome of dialysis or kidney transplantation: age, sex, estimated glomerular filtration rate, systolic blood pressure, diastolic blood pressure, hemoglobin, phosphate, parathyroid hormone, proteinuria, use of angiotensin-converting enzyme inhibitors or angiotensin II type 1 receptor blockers, as well as combination terms of diabetes/proteinuria and albumin/proteinuria.

Johnson and colleagues8 developed a prediction model based on data from the Kaiser Permanente Northwest Health Maintenance Organization (HMO) that allows classification of patients with chronic kidney disease into risk levels for the outcome of 5-year risk of dialysis or kidney transplantation. From a large set of patient characteristics collected during routine clinical practice, clinically important determinants of the outcome were initially selected for inclusion in the model. The decision to retain a determinant in the model was based on accuracy of prediction using the c statistic, and the final model was validated using a re-sampling technique. The following 6 variables were included in the final model: age, sex, diabetes, estimated glomerular filtration rate, anemia, and hypertension. Information from these individual predictors was translated into a point score and the sum of the points allowed the classification of patients in specific risk levels. In this point score, continuous variables were transformed into categories, leading to some loss of information, but this approach increases its clinical utility since users do not need to employ a complex equation. The authors show the consequence of selecting different cut-points from greater than or equal to 1% to greater than or equal to 20% risk of dialysis or kidney transplantation. Specifically, they show that the predicted risk from the final prediction model agrees well with the observed risk and that the model can discriminate well between the high-risk and low-risk patients.

Because the 2 prediction models aim to answer different questions and consequently utilize different modeling techniques, readers should not be surprised that different variables have been included in the final models. Both of these publications are important contributions in identifying individual predictors and patients at high risk for needing dialysis and kidney transplantation. These studies will help to target future research of adverse outcomes among patients with chronic kidney disease. Since the construction of prediction models follows statistical and not necessarily biological or clinical properties, future studies should aim to translate individual statistical predictors into biological risk factors. Such studies may focus on strength of the association between a proposed risk factor and outcome, prevalence of a risk factor in specific populations, attributable risks, and perhaps causal modeling approaches.9

While the proposed risk classification model8 performs well in the provided setting of the Kaiser Permanente Northwest HMO, it remains to be established whether this model performs equally well in other populations. In addition, other researchers may wish to focus on different measures of model performance to judge risk classification models or to assess whether novel risk factors help to improve prediction of dialysis and kidney transplantation. There is, however, an ongoing discussion, mainly about studies of cardiovascular outcomes,4, 10, 11, 12 regarding the optimal approach for evaluating such prediction models. Johnson and colleagues reasonably chose to use the c statistic as a measure of prediction. The c statistic compares the ranks of predicted probabilities in individuals with and without the outcome and ranges from 0.5 (no discrimination) to 1.0 (perfect discrimination). However, the c statistic is an insensitive measure for assessing the impact of new predictors to a score13 and the utilization of relative risk measures in relating predictors and disease has limited impact on the c statistic.6 Recently, other measures of model performance have been suggested.5, 10 These mainly focus on the amount of reclassification of individuals into different risk levels or the improvement of predicted versus observed risk in reclassified individuals.

In summary, prediction models can target different aspects of specific clinical questions. Depending on the aims, researchers should select adequate model-building strategies and consider the purpose of the model when evaluating model performance. Most importantly, however, the results of prediction models should lead to future studies evaluating whether implementation of interventions or different management of patients will prevent disease or adverse outcomes.