Table of Contents

HK J Paediatr (New Series)
Vol 28. No. 3, 2023

HK J Paediatr (New Series) 2023;28:157-164

Original Article

Comparison of Three Critical Illness Scoring Systems for Assessing Septic Acute Kidney Injury

X Peng, X Sun, X Chen, L Liu, J Lu, L Dong


Abstract

Introduction: Sepsis is the most common critical illness in clinical settings, and septic acute kidney injury (AKI) is a major cause of mortality in paediatric patient. Methods: We aimed to investigate scoring systems for determining the severity of septic AKI through mortality prediction using Pediatric Risk of Mortality III (PRISM III), Pediatric Multiple Organ Dysfunction Score (P-MODS), and Pediatric Critical Illness Score (PCIS). The clinical data of 102 paediatric patients with septic AKI admitted to the paediatric intensive care unit from January 2014 to December 2018 were collected. Receiver operating characteristic (ROC) curves were plotted to determine the optimal cutoff values of the scoring systems for assessing mortality. Results: There were 25.64% death rates among patients with stage 1 disease, 45% with stage 2 disease, and 58.14% with stage 3 disease, with a significant difference (x2=8.8409, p=0.012). The cutoff values of the ROC curves of PRISM III, P-MODS, and PCIS were 12, 6, and 82, respectively, in patients with not staged septic AKI; 12, 5, and 84, respectively, in patients with stage 1 septic AKI; 17, 9, and 72, respectively, in patients with stage 2 septic AKI; and 12, 7, and 74, respectively, in patients with stage 3 septic AKI. Conclusions: PRISM III was the best mortality risk assessment system for paediatric patients with not staged septic AKI. PCIS was better in predicting the mortality risk of paediatric patients with stage 1 AKI, whereas PRISM III was better for paediatric patients with stage 2 and 3 AKI.

Keyword : Acute kidney injury; Pediatric Critical Illness Score; Pediatric Multiple Organ Dysfunction Score; Pediatric Risk of Mortality III; Sepsis


Introduction

Sepsis is a major healthcare problem, affecting millions of people around the world each year.1 It is a typical complication of severe trauma, burns, shock, and major surgery, and an important cause of mortality in patients with trauma and burns. Statistics show that about 8 million children under the age of five dead each year in the world, nearly 70% are due to sepsis caused by infection. Even in the United States, a country with advanced medical technology, 42,000 children have severe sepsis every year, with a mortality rate of up to 10% among them. Meanwhile the mortality rate of paediatric patients with severe sepsis in paediatric intensive care units (PICUs) reaches up to 50% in developing countries.2 In China, sepsis among children is also an important concern. An article published in November 2014 in Pediatric Critical Care Medicine evaluated the prevalence of sepsis in paediatric patients in Huai'an District of Jiangsu Province in China. The study showed that not only was there a large number of paediatric patients with sepsis but also that the patients were young, with 80% <5-year-old and 36% <1-year-old. The mortality rate among paediatric patients with sepsis was also extremely high (33% died of severe sepsis, 74% died of respiratory failure with or without septic shock), and most patients did not have underlying diseases.3,4 In 2013, a prospective multicentre study on paediatricacute kidney injury (AKI) in China was published in BMC Urology, which showed that sepsis was the leading cause of mortality in children with AKI, accounting for 34.9% of deaths among paediatric patients.3,4

Severe sepsis is a major cause of death in PICUs in both developing and developed countries, and septic AKI increases the risk of in-hospital death six to eight-fold.5 The third generation of the Pediatric Risk of Mortality (PRISM III), published in 1996, is one of the widely used scoring systems for assessing condition and prognosis for critically ill children admitted to PICU.6 The Pediatric Critical Illness Score (PCIS) is the most widely used and effective scoring method for paediatric critical illness in China. The objective of the present study was to compare three paediatric critical illness scoring systems in combination with the staging criteria for AKI diagnosis developed by the Kidney Disease Improving Global Outcomes (KDIGO) Work Group, in order to evaluate the suitability of scoring systems in predicting the mortality of patients admitted in PICU. Receiver operating characteristic (ROC) curves were plotted to compare the predictive effect of the area under the ROC curve (AUC) of each assessment system on the mortality of paediatric patients with septic AKI, in order to select the appropriate critical illness scoring system for different AKI stages.

Data and Methods

Study Populations and Groups
A total of 102 paediatric patients with septic AKI who were treated in the PICU of West China Second University Hospital of Sichuan University from January 2014 to December 2018 were included. This study was conducted in accordance with the provisions of the revised 2013 World Medical Association Declaration of Helsinki.

Methods
Diagnostic Criteria
The diagnosis of sepsis was based on the "Expert Consensus on the Diagnosis and Treatment of Pediatric Septic Shock (2015 edition)".7 The diagnosis and staging of septic AKI in paediatric patients were based on the 2012 KDIGO guidelines (Table 1).8 The baseline creatinine was derived by urine volume assessment, or according to European Renal Best Practice (ERBP) Working Group recommended that the first recorded serum creatinine value as a baseline.

Table 1 AKI staging in the 2012 KDIGO guidelines
Stage Serum creatinine criteria Urine output criteria
Stage 1 1.5-1.9 times baseline or ≥0.3 mg/dl (≥26.5 μmol/L) increase <0.5 mL/kg.h, time 6-12 h
Stage 2 2.0-2.9 times baseline <0.5 mL/kg.h, time >12 h
Stage 3 3.0 times baseline or ≥4.0 mg/dl (≥353.6 μmol/L) increase in or initiation of renal replacement therapy or in patients <18 years decrease in eGFR <35 ml/min per 1.73 m2 Oliguria (<0.3 mL/kg.h) >24 h or anuria >12 h
AKI, acute kidney injury; KDIGO, the Kidney Disease Improving Global Outcomes

Inclusion and Exclusion Criteria
Paediatric patients who met the diagnostic criteria for sepsis and were diagnosed with AKI based on the 2012 KDIGO guidelines were included. Paediatric patients diagnosed with sepsis but not AKI, or paediatric patients diagnosed with AKI but without clinical manifestations of sepsis were excluded.

Data Collection and Relevant Diagnostic Parameters
The clinical data of all paediatric patients were retrospectively collected, including basic epidemiological data (sex, age of onset), vital signs on admission to the intensive care unit, and results of auxiliary examinations. The disease was evaluated and scored by the bedside clinician using three critical illness assessment systems – PRISM III, PCIS, and Pediatric Multiple Organ Dysfunction Score (P-MODS), and the bedside clinician were blind to the outcome and blinded to the stage of AKI.

Statistical Analysis
Statistical analyses were performed using SAS 9.4 software. Sample size was calculated using the formula for determining the number of samples required for a diagnostic test.9 Qualitative data, such as sex, age, AKI, and mortality, were expressed as rate or percentage (%). Differences in rates were compared using the chi-square test. ROC curves were plotted to determine the optimal cutoff values of the PRISM III score, PCIS, and P-MODS. The cut-off values and the areas under the ROC curve were used to assess the mortality in paediatric patients with septic AKI that had not been staged (not staged septic AKI). Moreover, the best scoring system among PRISM III, PCIS, and P-MODS for predicting mortality in paediatric patients with different stages of AKI was also evaluated. The maximum value of the Youden index was used as the optimal cutoff value.10

Results

Clinical Data Analysis
Baseline Clinical Data
A total of 102 paediatric patients with septic AKI were included in this study, including 65 boys (63.73%) and 37 girls. The age of onset was 5.32±4.88 years. The youngest patient was 1 month old, and the oldest patient was 17 years old. A total of 62 children (60.78%) were ≤5 years old (Table 2).

Table 2 Analysis of clinical data of 102 paediatric patients with septic AKI [number of patients (%)]
  Group   Died Survived X2 p
  n   44 58    
Sex   Male 30 (46.15) 35 (53.85) 2.039 0.3606
  Female 14 (37.84) 23 (62.16)
Age   ≤5 years 26 (41.94) 36 (58.06) 0.1010 0.9508
  >5 years 18 (45.00) 22 (55.00)
AKI stage   1 10 (25.64) 29 (74.36) 8.8409 0.012
  2 9 (45.00) 11 (55.00)
    3 25 (58.14) 18 (41.86)    
  Group   Died Survived X2 p
  n   44 58    
PCIS   ≤82 34 (62.96) 20 (37.04) 17.9025 <0.0001
  >82 10 (20.83) 38 (79.17)
PRISM III   ≥12 33 (75.00) 11 (25.00) 32.0266 <0.0001
  <12 11 (18.97) 47 (81.03)
P-MODS   ≥6 29 (61.70) 18 (38.30) 12.2473 0.0005
  <6 15 (27.27) 40 (72.73)
AKI, acute kidney injury; PCIS, Pediatric Critical Illness Score; PRISM III, Pediatric Risk of Mortality III; P-MODS, Pediatric Multiple Organ Dysfunction Score

Among the paediatric patients with septic AKI, 44 (43.14%) died, 26 of whom were ≤5 years old, accounting for 59.09% of all deaths. According to AKI stages, there were 10 deaths (25.64%) among 39 patients with stage 1 disease, 9 deaths (45%) among 20 patients with stage 2 disease, and 25 deaths (58.14%) among 43 patients with stage 3 disease. The mortality rate significantly increased with higher AKI stages (x2=8.8409, p=0.012). The mortality rate showed no difference between different sex (P=0.3606) and age (P=0.9508). The increase in mortality rate was statistically significant when PCIS was <82, PRISM III score was >12, and P-MODS was >6.

Best Scoring System Based on AUC
Evaluation of Each Assessment System for Not Staged Septic AKI
Figure 1 shows the ROC curves of PRISM III, P-MODS, and PCIS for mortality assessment in paediatric patients with not staged septic AKI. The cutoff values were 12, 6, and 82, respectively, and the AUCs were 0.8452, 0.7414, and 0.8184, respectively. Therefore, the best mortality risk assessment system for paediatric patients with not staged AKI was PRISM III.

Figure 1 Cutoff values and ROC curves of each assessment system for not staged septic AKI.

For paediatric patients with not staged septic AKI, the AUCs for PRISM III, P-MODS, and PCIS were 0.845, 0.741, and 0.818, respectively (Table 3).

Table 3 ROC curves of PRISM III, P-MODS, and PCIS for mortality assessment in not staged septic AKI
Assessment system AUC SE P 95% CI
PRISM III 0.845 0.040 0.000 0.767-0.923
P-MODS 0.741 0.048 0.000 0.739-0.836
PCIS 0.818 0.040 0.000 0.739-0.898
AKI, acute kidney injury; PCIS, Pediatric Critical Illness Score; PRISM III, Pediatric Risk of Mortality III; P-MODS, Pediatric Multiple Organ Dysfunction Score

The PRISM III score, P-MODS, and PCIS for mortality assessment in paediatric patients with not staged septic AKI showed cutoff values of 12, 6, and 82, respectively. PRISM III had a sensitivity of 75% and a specificity of 81.03%. P-MODS had a sensitivity of 65.91% and a specificity of 68.97%. PCIS had a sensitivity of 77.27% and a specificity of 65.51%. When the three scoring systems were used jointly for assessment, the specificity increased to 96.55% and the sensitivity decreased to 38.64% (Table 4).

Table 4 Utility of PRISM III, P-MODS, and PCIS for mortality assessment in paediatric patients with not staged septic AKI
Scoring system Cutoff value Sensitivity (%) Specificity (%) Positive predictive value (%) Negative predictive value (%)
PRISM III 12 75.00 81.03 75.00 81.03
P-MODS 6 65.91 68.97 61.70 72.73
PCIS 82 77.27 65.51 62.96 79.17
Joint assessment with the three systems - 38.64 96.55 89.47 67.47
AKI, acute kidney injury; PCIS, Pediatric Critical Illness Score; PRISM III, Pediatric Risk of Mortality III; P-MODS, Pediatric Multiple Organ Dysfunction Score

Evaluation of Each Assessment System for Septic AKI of Different Stages
In paediatric patients with stage 1 septic AKI, the cutoff values of the ROC curves of PRISM III, P-MODS, and PCIS for mortality assessment were 12, 5, and 84, respectively, and the AUCs were 0.8414, 0.7362, and 0.8810, respectively (Figure 2). Therefore, the best mortality assessment system for paediatric patients with stage 1 AKI was PCIS.

Figure 2 Cutoff values and ROC curves of each assessment system for paediatric patients with stage 1 septic AKI.

In paediatric patients with stage 2 septic AKI, the cutoff values of the ROC curves of PRISM III, P-MODS, and PCIS for mortality assessment were 17, 9, and 72, respectively, and the AUCs were 0.8232, 0.7576, and 0.6869, respectively (Figure 3). Therefore, the best mortality assessment system for paediatric patients with stage 2 AKI was PRISM III.

Figure 3 Cutoff values and ROC curves of each assessment system for paediatric patients with stage 2 septic AKI.

In paediatric patients with stage 3 septic AKI, the cutoff values of the ROC curves of PRISM III, P-MODS, and PCIS for mortality assessment were 12, 7, and 74, respectively, and the AUCs were 0.8367, 0.6778, and 0.7822, respectively (Figure 4). Therefore, the best mortality assessment system for paediatric patients with stage 3 AKI was PRISM III.

Figure 4 Cutoff values and ROC curves of each assessment system for paediatric patients with stage 3 septic AKI.

Discussion

Sepsis is the major cause of critical illness in children. Severe sepsis occurs in approximately 8% of paediatric patients in PICUs. The prevalence of sepsis has decreased owing to relative advances in medical technology and supportive therapies, and sepsis-associated mortality has been greatly reduced. However, it has led to increased attention paid to the organ function and prognosis of surviving children.11-13 Severe septic AKI is a common comorbidity of sepsis, accounting for more than half of the cases in adult patients. Among paediatric patients with severe diseases in PICUs, AKI occurs in approximately 16%; however, no precise data are available for septic AKI.14,15 Many studies have demonstrated an extremely high mortality rate in children with severe sepsis. In PICUs, the mortality rate of critically ill paediatric patients without AKI is 58.7%, whereas that of critically ill paediatric patients with AKI can reach 73.4%. No definitive mortality rate in children with septic AKI has been reported.16,17 Given the high mortality rate of septic AKI, early predictive assessment and active strengthening of supportive therapies are needed to reduce the mortality rate of paediatric patients with sepsis and improve the poor long-term prognosis of survivors.

The Physiological Stability Index (PSI) is the first worldwide-recognised critical illness severity scoring method in children; however, it requires the assessment of many parameters and is cumbersome to use in the clinic. In 1988, Pollack et al simplified the PSI and proposed the first generation of PRISM, which has since evolved to the third generation (PRISM III). PRISM III comprises 17 physiological parameters and 26 physiological parameter ranges and is widely used in various parts of the world. It has become a standard tool for assessing the condition and prognosis of paediatric patients in PICUs. PCIS was developed by the Emergency Medicine Group of the Chinese Pediatrics Society and the Pediatrics Group of the Chinese Society of Emergency Medicine in 1995. PCIS comprises 10 physiological parameters and is the most widely used and effective paediatric critical illness scoring method in China owing to its simplicity, effectiveness, and compatibility with the national condition. P-MODS is primarily used to assess the degree of organ dysfunction in children by evaluating parameters such as bilirubin, lactic acid, fibrinogen, urea, and oxygenation index.

In the present study, the mortality rate of paediatric patients with septic AKI was 43.14%. This result was similar to the epidemiological results from a survey using claims data of the National Health Insurance system in Taiwan between 2006 and 2010, which showed that the incidence of AKI in critically ill paediatric patients was 1.4%, of which 46.5% of the AKI cases was caused by sepsis. The survey also showed that the rate of mortality associated with critical illness was 44.2%. In addition, Julie et al reported that septic AKI was an independent risk factor for death in paediatric patients with severe sepsis.16 The mortality rate increased with higher AKI stages. Through ROC curve and AUC analyses, we showed that the AUC of PRISM III was 0.845 when the cutoff score was 12, whereas the AUCs of P-MODS and PCIS were 0.741 and 0.818, respectively, which were lower than the AUC of PRISM III. Thus, PRISM III was the most useful system for mortality prediction in paediatric patients with septic AKI.

A study by Kaur et al has reported that PRISM III score has excellent capacity to discriminate between survival and mortality. PRISM III score can be used to predict length of stay among survivors.6 A prospective observational cohort study by Graciano et al showed that P-MODS scores could be objectively reflected organ dysfunction and predict the risk of death accurately.18 Another study showed that PCIS was a protective factor (OR: 0.88; 95% CI: 0.86-0.90) for paediatric AKI.19 Up to now, few studies have been made to compare three systems in predicting mortality in paediatric patients with different stages septic AKI. In this study, we compared the performance of PRISM III, P-MODS, and PCIS for the prediction of mortality risk in paediatric patients with septic AKI of different stages. For stage 1 septic AKI, the mortality rate was 25.64%. Our result shows that PCIS performed better in predicting mortality risk in paediatric patients with stage 1 septic AKI, and the mortality risk of paediatric patients increased when the PCIS was <84. For stage 2 septic AKI, the mortality rate was 45.0%. Our study suggested that PRISM III performed better than P-MODS and PCIS in predicting mortality risk in paediatric patients with stage 2 septic AKI. When the PRISM III score was >17, the mortality risk of paediatric patients increased with higher scores. For stage 3 septic AKI, the mortality rate was 58.14%. When the PRISM III score was >12, the mortality risk due to stage 3 septic AKI significantly increased. PRISM III performed better than the other two systems in predicting mortality in paediatric patients with stage 3 septic AKI. The P-MODS score can be used to evaluate five body functions, namely, circulation, breathing, liver function, blood coagulation, and kidney function. Because the P-MODS score does not include an assessment of the nervous system, its prognostic value in children with conditions related to the nervous system diseases may be limited. So that there are limitations in predicting prognosis in P-MODS score. For the PRISM III score, data for the following 16 variables: temperature, systolic blood pressure, heart rate, partial pressure of arterial oxygen (PaO2), partial pressure of arterial carbon dioxide (PaCO2), Glasgow Coma Scale (GCS) score, pupillary reaction, prothrombin time (PT) and activated partial thromboplastin time (APTT), serum creatinine, serum urea nitrogen, serum potassium, blood glucose, and serum bicarbonate levels, white blood cell and platelet counts. Therefore, PRISM III score is relatively accurate in predicting prognosis. The PCIS scoring system included more electrolytes, and the scoring was greatly affected by electrolytes. There were few electrolyte disturbances in patients with stage 1 septic AKI. So that PCIS performed better in predicting mortality risk in paediatric patients with stage 1 septic AKI.

Prediction models of mortality provide great insight to health care administrators regarding the prognosis of the patient and may greatly benefit the decision process as well as the outcome of the patient. The conclusion of our study can help in-charge physician to choose more suitable scoring systems to better predict the clinical outcomes in children facing different stages of septic AKI. Choosing suitable scoring systems according to different stages, physicians can assess the survival chances of the patient. In settings where there is a shortage of medicines and staff, such models enable physicians to decide how and where to direct their limited resources.

In summary, this study retrospectively compared the results of three critical illness scoring systems (PRISM III, P-MODS, and PCIS) for mortality assessment in paediatric patients diagnosed with septic AKI. Our results showed that PRISM III had the best assessment performance in paediatric patients with not staged, stage 2, and stage 3 AKI, whereas PCIS performed better in predicting mortality risk in paediatric patients with stage 1 AKI. However, our study is limited by the small sample size and single centre. Moreover, the urinary assessment was not accurate because the urine output may not be measured precisely in infant which have the risk of mixing the urine with stool. In addition, baseline creatinine value can be a bias as patient may already develop AKI on PICU admission. In next study, we will increase the sample size and collect data from a multi-centre. Furthermore, we will find a better way to measure the urine output. And we will try our best to get the baseline creatinine value from the healthy period of the patient.

Declaration of Interest

The authors declare no conflicts of interests.


References

1. Rhodes A, Evans LE, Alhazzani W, et al. Surviving Sepsis Campaign: International Guidelines for Management of Sepsis and Septic Shock: 2016. Intensive Care Med 2017;43:304-77.

2. Ashraf M, Shahzad N, Irshad M, Hussain SQ, Ahmed P. Pediatric acute kidney injury: a syndrome under paradigm shift. Indian J Crit Care Med 2014;18:518-26.

3. Wang Y, Sun B, Yue H, et al. An epidemiologic survey of pediatric sepsis in regional hospitals in China. Pediatr Crit Care Med 2014;15:814-20.

4. Cao Y, Yi ZW, Zhang H, Dang XQ, Wu XC, Huang AW. Etiology and outcomes of acute kidney injury in Chinese children: a prospective multicenter investigation. BMC Urol 2013;13:41.

5. Gómez H, Kellum JA. Sepsis-induced acute kidney injury. Curr Opin Crit Care 2016;22:546-53.

6. Kaur A, Kaur G, Dhir SK, et al. Pediatric Risk of Mortality III Score - Predictor of Mortality and Hospital Stay in Pediatric Intensive Care Unit. J Emerg Trauma Shock 2020;13:146-50.

7. Emergency Medicine Group of the Chinese Pediatrics Society; Pediatrics Group of the Chinese Society of Emergency Medicine; Division of Pediatric Critical Care Physician of the Chinese Medical Doctor Association. Expert consensus for the diagnosis and management of septic shock (infectious shock) in children (2015). Chin J Pediatr 2015;53:576-80.

8. Kidney Disease: Improving Global Outcomes (KDIGO) Acute Kidney Injury Work Group. KDIGO clinical practice guideline for acute kidney injury. Kidney Int 2012;Suppl 2:1.

9. Malhotra RK, Indrayan A. A simple nomogram for sample size for estimating sensitivity and specificity of medical tests. Indian J Ophthalmol 2010;58:519-22.

10. Aggarwal R, Ranganathan P. Understanding diagnostic tests - Part 3: Receiver operating characteristic curves. Perspect Clin Res 2018;9:145-8.

11. Weiss SL, Fitzgerald JC, Pappachan J, et al; Sepsis Prevalence, Outcomes, and Therapies (SPROUT) Study Investigators and Pediatric Acute Lung Injury and Sepsis Investigators (PALISI) Network. Global epidemiology of pediatric severe sepsis: the sepsis prevalence, outcomes, and therapies study. Am J Respir Crit Care Med 2015;191:1147-57.

12. Balamuth F, Weiss SL, Neuman MI, et al. Pediatric severe sepsis in U.S. children's hospitals. Pediatr Crit Care Med 2014;15:798-805.

13. Ruth A, McCracken CE, Fortenberry JD, Hall M, Simon HK, Hebbar KB. Pediatric severe sepsis: current trends and outcomes from the Pediatric Health Information Systems database. Pediatr Crit Care Med 2014;15:828-38.

14. Kolhe NV, Stevens PE, Crowe AV, et al. Case mix, outcome and activity for patients with severe acute kidney injury during the first 24 hours after admission to an adult, general critical care unit: application of predictive models from a secondary analysis of the ICNARC Case Mix Programme database. Crit Care 2008;12(suppl 1):S2.

15. Schneider J, Khemani R, Grushkin C, Bart R. Serum creatinine as stratified in the RIFLE score for acute kidney injury is associated with mortality and length of stay for children in the pediatric intensive care unit. Crit Care Med 2010;38:933-9.

16. Fitzgerald JC, Basu RK, Akcan-Arikan A, et al. Acute kidney injury in pediatric severe sepsis: an independent risk factor for death and new disability. Crit Care Med 2016;44:2241-50.

17. Al-jboor W, Almardini R, Al Bderat J, Frehat M, Masri HA, Alajloni MS. Acute kidney injury in critically ill child. Saudi J Kidney Dis Transpl 2016;27:740-7.

18. Graciano AL, Balko JA, Rahn DS, Ahmad N, Giroir BP. The Pediatric Multiple Organ Dysfunction Score (P-MODS): development and validation of an objective scale to measure the severity of multiple organ dysfunction in critically ill children. Crit Care Med 2005;33:1484-91.

19. Wei C, Hongxia G, Hui F, Xianhui Q, Danqun J, Haipeng L. Impact of and risk factors for pediatric acute kidney injury defined by the pROCK criteria in a Chinese PICU population. Pediatr Res 2021;89:1485-91.

 

 
 

©2024 Hong Kong Journal of Paediatrics. All rights reserved. Developed and maintained by Medcom Ltd.