Predicting Incident CKD

What Does This Important Study Show? 

Using baseline data from 2 community-based cohorts, the Atherosclerosis Risk in Communities (ARIC) Study and the Cardiovascular Health Study (CHS), the investigators created a combined sample of 14,155 middle-aged and older adults in which to derive and validate their risk score.2 Of the total sample, just under 60% were women, approximately one-fifth was nonwhite, and all had a baseline estimated glomerular filtration rate (eGFR) of at least 60 mL/min/1.73 m2. Over a follow-up period of up to 10 years, the investigators identified 1,605 cases of incident CKD, defined as an eGFR less than 60 mL/min/1.73 m2.

Participants from each original cohort were randomly assigned 2:1 to make up the derivation and validation samples, respectively. Within the derivation sample, logistic regression models showed that incident CKD was associated with 10 baseline predictors: age, female sex, white race/ethnicity, systolic blood pressure, high-density lipoprotein (HDL) cholesterol, diabetes, anemia, peripheral vascular disease, history of cardiovascular disease, and history of heart failure. Not surprisingly, age was by far the strongest single predictor. For instance, age over 70 years conferred a 4-fold adjusted risk of incident CKD. Continuous variables were converted to clinically meaningful categorical variables, and then all 10 variables were used together to create a predictive model. For further simplification, race/ethnicity and HDL cholesterol were removed to create an 8-predictor model. This final model formed the basis for a risk score that assigned points to levels of each of the 8 predictors (Table 1).

Table 1. A Proposed Algorithm for Predicting Incident CKD

	Risk Factor Scoring
	Risk Factor	Odds Ratio (95% CI)	Assigned Score
	Age, y
	50-59	1.9(1.5-2.4)	1
	60-69	3.8(3.0-4.8)	2
	≥70	4.3(3.3-5.6)	3
	Female sex	1.1(1.0-1.3)	1
	Anemia	1.6(1.1-2.4)	1
	Hypertension	1.7(1.5-2.0)	1
	Diabetes mellitus	1.4(1.2-1.7)	1
	History of cardiovascular disease	1.3(1.1-1.6)	1
	History of heart failure	1.6(1.0-2.7)	1
	Peripheral vascular disease	1.5(1.2-1.9)	1

	Risk Prediction
	Total Score	Sensitivity, %	Specificity, %	PPV, %	NPV, %
	≥5	20	93	28	90
	≥4	45	80	22	92
	≥3	69	58	17	94
	≥2	89	29	14	96

Source: Kshirsagar et al.2 Abbreviations: CI, confidence interval; PPV, positive predictive value; NPV, negative predictive value.

As noted, the risk score would be easy to calculate in any clinical setting. Whether it adds incremental prognostic information is more challenging to assess. Because the score consists of risk factors that are associated with CKD risk, the score itself has a strong statistical association with incident CKD. However, a statistically significant association does not ensure prognostic utility.4 Two additional features of the score are important to consider: (1) discrimination, the model's ability to separate individuals who eventually develop CKD from those who do not; and (2) calibration, the model's ability to provide risk estimates that are close to actual, observed event rates.

The best-known measure of discrimination is the area under the receiver operating characteristic curve (also known as the c statistic), which is an overall index of sensitivity and specificity across the range of diagnostic thresholds. Values for the c statistic range from 0.5 (no discrimination) to 1.0 (perfect discrimination). One way to interpret the c statistic is to consider a random pair of individuals, one of whom will develop the event of interest; the c statistic is the probability of correctly selecting the individual who will develop the event. Thus, the c statistic for the CKD algorithm (0.69) denotes a 69% probability of assigning a higher risk score to the individual who develops CKD. Because a completely uninformative risk score would still have a 50% chance of correctly identifying the higher-risk individual, values for the c statistic between 0.50 and 0.70 correspond to only moderate discrimination. In comparison, c statistics for cardiovascular risk models (such as the Framingham Heart Study risk score) typically range from 0.75 to 0.85.

The CKD algorithm performs better with respect to calibration. In models that are well calibrated, only small differences exist between event rates predicted by the model and observed event rates. The Hosmer-Lemeshow statistic is a measure of calibration. Hosmer-Lemeshow statistics that are not statistically significant denote good calibration (ie, no significant difference between predicted and observed event rates). For the CKD risk score, Hosmer-Lemeshow statistics were nonsignificant (P > 0.20) for both the full and simplified models. Furthermore, observed event rates appeared close to actual event rates when plotted across the possible point scores.

Thus, the CKD risk score proposed by Kshirsagar and colleagues provides moderate discrimination and good calibration. However, to answer the question of whether the risk score provides incremental information, it is also useful to consider what alternatives exist for predicting the risk of CKD. Age stands out as a particularly powerful predictor of CKD risk, raising the possibility that the predictive value of the final risk score is largely dependent on knowing the age of the patient. Therefore, it would be interesting and informative to see how the final algorithm would perform compared to a predictive model based only on age.

With respect to study limitations, the authors note that they were unable to include urinary indices such as albuminuria, which defines the presence of CKD even in the setting of a normal eGFR. Therefore, design of the risk score may be limited by misclassification of individuals in both the derivation and validation samples. Another limitation is that the generalizability of the study findings to nonwhite individuals is uncertain, because the overall number of nonwhite participants was relatively low.

How Does This Study Compare With Prior Studies? 

Although there have been no previously published prediction models for incident CKD in the community, several prior studies have reported on individual risk factors. Many of these studies are cited by Kshirsagar et al,2 and a majority of the previously identified variables were included in the initial regression models tested in the derivation sample. One notable exclusion was baseline kidney function.5, 6, 7 A prior study that also evaluated multiple risk factors for CKD used data from 2,585 middle-aged participants of the Framingham Heart Study and identified the following independent predictors: age, baseline eGFR, diabetes, body mass index, low HDL, hypertension, and smoking.5 Importantly, the strongest predictor overall was baseline kidney function, where a baseline eGFR less than 90 mL/min/1.73 m2 was associated with an adjusted odds ratio of 3.01 (95% CI, 1.98-4.58) for incident CKD. The value of considering a subclinical measure of an outcome has been similarly demonstrated in analyses that identified baseline blood pressure as predictive of hypertension8 and baseline fasting glucose as predictive of diabetes.9

Thus, several factors identified by the present model—such as anemia, history of heart failure, and history of cardiovascular or peripheral arterial disease—could be markers for the presence of subclinical renal dysfunction. Indeed, Kshirsagar and colleagues have previously identified many of the same variables as being independently associated with occult CKD.10 Accordingly, it is possible, and perhaps likely, that a simple score based on either current eGFR alone or on both age and current eGFR would outperform the CKD score proposed by Kshirsagar and colleagues.

What Should Clinicians and Researchers Do? 

A key question is what a clinician would do with such information, and whether it would change management. Current guidelines for hypertension and diabetes, the primary causes of CKD in US adults, already call for screening to detect CKD as well as more intensive therapy for individuals found to have CKD.11, 12, 13 Therefore, the degree to which such a model would impact on clinical decision making—besides reinforcing the importance of risk factor management—has yet to be determined. Although the authors argue that a high CKD score could motivate more intensive risk factor modification, clinicians should already be aggressive about treating patients with hypertension, diabetes, established cardiovascular disease, or peripheral vascular disease, to name a few components of the model.

While studies such as the one by Kshirsagar and colleagues provide valuable information, there is clearly more work to be done. The modest discriminative ability of the clinical CKD prediction model underscores the need for better measures of subclinical renal dysfunction and novel risk markers that could be incorporated into future algorithms. Furthermore, improvements in risk stratification may yield little clinical gain unless better therapies for preventing CKD are developed.