Clinical paperValidation of the CaRdiac Arrest Survival Score (CRASS) for predicting good neurological outcome after out-of-hospital cardiac arrest in an Asian emergency medical service system
Introduction
Out-of-hospital cardiac arrest (OHCA) is a time-critical medical emergency, with an average incidence of 67 to 170 per 100,000 persons per year in Europe1., 2. and a 2010 global incidence of 96 emergency medical services (EMS)-attended OHCAs per 100,000 population.3 A more recent study by the International Liaison Committee on Resuscitation (ILCOR) found that this incidence ranged from 30.0 to 97.1 individuals per 100,000 population across its surveyed registries.4 Due to its poor prognosis, OHCA is one of the leading causes of death in adults worldwide.5 Timely intervention and effective prehospital emergency care are vital for saving lives.6., 7. Many factors associated with successful management of OHCA have been described in conjunction with the chain of survival.5 The most important indicators of successful resuscitation are overall survival and survival with favorable neurological outcomes, therefore early and accurate prediction of such outcomes are key to efficient OHCA management.8., 9., 10..
Numerous prognostic models have been developed to predict survival,8., 9., 11. neurologic recovery,12 and the return of spontaneous circulation (ROSC).13., 14. These models served as clinical scoring tools to provide valuable support in decision-making regarding the proper allocation of resources, optimal treatment and communication with families.9 Clinical prognostic models are commonly formulated to quantify the likelihood that an event will occur, for which both traditional logistic regression13., 15., 16. and sophisticated machine learning techniques (e.g., random forest and deep learning)17., 18. have been employed. There are a variety of OHCA scoring models, some of which serve as operational tools that can be used by EMS personnel at the scene,16., 19. while others are designed for comparison between EMS services or hospitals as quality assessment tools.13., 15. However, none of these scoring models have been widely adopted in clinical practice, even though some have been externally validated.8 The gap between validation and adoption is then predominantly driven by how clinical decisions in OHCA need to be made with an extremely high degree of certainty.8 Bringing scoring models that were developed and validated based on a specific cohort into clinical practice then raises questions about their validity in other clinical cohorts of diverse cultures, patient populations, geographical locations and EMS systems. OHCA survival rates, for instance, differ dramatically between regions,2., 20. therefore a score developed using data from one region may not be able to accurately reflect the survival rate in another region. As a remedy for the wide variations in patient outcomes and characteristics, Liu et al.21 proposed adjusting clinical scores with local historical information before actual implementations. It is therefore essential to ensure that clinical scoring models are validated, calibrated, and adjusted appropriately.
Among the scores that predict neurological outcomes following OHCA, the CaRdiac Arrest Survival Score (CRASS) is one of the newest additions.15 CRASS is a tool developed with data from the German Resuscitation Registry22 (EMS in Germany involves both paramedics and an EMS physician being dispatched to the scene) that uses patient demographics and OHCA-related details to estimate the probability of good neurological survival. CRASS was originally developed not as a triage scoring system but rather as a quality assessment tool for comparing treatment regimens in EMS and hospital systems. This is unlike other related scoring systems like the UB-ROSC score, which was created with the intention of being directly utilized by EMS personnel.16 However, it should be noted that in its development and validation, the score was still evaluated based on individual, patient-level prediction.15 As CRASS has not been externally validated in other countries, its performance in different EMS systems and cultural settings is unknown. This study aims to evaluate CRASS in predicting good neurological outcomes using data collected from OHCA patients over eight years in Singapore.
Section snippets
Study design and setting
We conducted a retrospective population-based cohort study using data of EMS-attended OHCA patients in Singapore. The data were retrieved from the Pan-Asian Resuscitation Outcomes Study (PAROS),23 an international prospective registry for cardiac arrest. This study analyzed data from the Singapore cohort collected between April 2010 and December 2018.
Singapore is an urbanized city-state located in Southeast Asia with a population of approximately 5.5 million on a land area of 728.6 square
Cohort characteristics
A total of 18,359 OHCA patients were included in the Singaporean cohort of the PAROS registry during the study period. A total of 6,955 patients were excluded: 338 were under the age of 18 years old, 5,970 did not have ROSC or ongoing CPR at hospital arrival, and 1,085 had no information pertinent to the patient's age, neurological status at discharge, or initial rhythm. Finally, 11,404 patients were included for analysis, with 260 (2.3%) who achieved favorable neurological outcome (Fig. 1).
CRASS score as a metric for quality control
The evaluation and calibration of the predictive performance of the CRASS score at the patient level is a necessary step before its utilization in comparing health systems and treatment strategies.15 As described in the original CRASS study, the score was intended as a quality control tool instead of a triage score calculator for patient-level clinical decision support. In our study, we used the CRASS score to evaluate the efficacy of healthcare institutions. While the observed percentages of
Discussion
In this study, we validated the CRASS score in a Singaporean cohort, where the prehospital care practice and patient characteristics differ from those in the German cohort, from which CRASS was developed. In contrast to the Singaporean EMS (fire-based and paramedic-staffed), in the case of an OHCA in Germany, paramedics and an EMS physician are dispatched to the scene. We observed excellent discrimination in determining the probability of favorable neurological outcomes. Despite this, CRASS
Conclusion
In this study, CRASS showed good discriminatory ability and moderate calibration performance in predicting favorable neurological outcomes among the Singapore cohort. By adjusting the constant coefficient in the CRASS score, the calibration performance improved. Future large-scale, cross-country validation studies of the CRASS score in a wide range of cultural, geographical, and clinical settings are needed.
Acknowledgements
The authors would like to thank Ms Nur Shahidah, Ms Pin Pin Pek and the late Ms Susan Yap from Department of Emergency Medicine, Singapore General Hospital; Ms Nurul Asyikin, Ms Liew Le Xuan, Ms Noor Azuin and Ms Joann Poh from Unit for Prehospital Emergency Care, Singapore General Hospital; Ms Woo Kai Lee from Department of Cardiology, National University Heart Centre Singapore and Ms Charlene Ong previously from Accident & Emergency, Changi General Hospital for their contributions and support
Funding
This work was supported by the National Medical Research Council, Clinician Scientist Awards, Singapore (NMRC/CSA/024/2010, NMRC/CSA/0049/2013 and NMRC/CSA-SI/0014/2017) and Ministry of Health, Health Services Research Grant, Singapore (HSRG/0021/2012).
The funders had no roles in the study design; in the collection, analysis, and interpretation of data; in the writing of the manuscript; and in the decision to submit the manuscript for publication.
Conflict of Interest
MEH Ong reports funding from the Zoll Medical Corporation for a study involving mechanical cardiopulmonary resuscitation devices; grants from the Laerdal Foundation, Laerdal Medical, and Ramsey Social Justice Foundation for funding of the Pan-Asian Resuscitation Outcomes Study; an advisory relationship with Global Healthcare SG, a commercial entity that manufactures cooling devices; and funding from Laerdal Medical on an observation program to their Community CPR training Centre Research
Singapore PAROS Investigators
Han Nee Gan (Changi General Hospital, Singapore); Si Oon Cheah (Urgent Care Clinic International, Singapore); Wei Ming Ng (Ng Teng Fong General Hospital, Singapore); Wei Ling Tay (Ng Teng Fong General Hospital, Singapore); Benjamin SH Leong (National University Hospital); Gayathri Nadarajan (Singapore General Hospital, Singapore); Nausheen Edwin Doctor (Sengkang General Hospital, Singapore); Lai Peng Tham (KK Women’s & Children’s Hospital, Singapore); Shalini Arulanandam (previously Chief
References (35)
- et al.
European Resuscitation Council Guidelines 2021: Executive summary
Resuscitation
(2021) - et al.
Survival after out-of-hospital cardiac arrest in Europe - Results of the EuReCa TWO study
Resuscitation
(2020) - et al.
Global incidences of out-of-hospital cardiac arrest and survival rates: Systematic review of 67 prospective studies
Resuscitation
(2010) - et al.
Out-of-hospital cardiac arrest across the World: First report from the International Liaison Committee on Resuscitation (ILCOR)
Resuscitation
(2020) - et al.
Out of hospital cardiac arrest: Past, present, and future
Resuscitation
(2021) - et al.
Out-of-hospital cardiac arrest: prehospital management
Lancet
(2018) - et al.
Out-of-hospital cardiac arrest: A systematic review of current risk scores to predict survival
Am Heart J
(2021) - et al.
Predictive value of hospital discharge neurological outcome scores for long-term neurological status following out-of-hospital cardiac arrest: A systematic review
Resuscitation
(2020) - et al.
Development and Validation of an Interpretable Prehospital Return of Spontaneous Circulation (P-ROSC) Score for Out-of-Hospital Cardiac Arrest Patients Using Machine Learning
eClinicalMedicine
(2022) - et al.
CaRdiac Arrest Survival Score (CRASS) - A tool to predict good neurological outcome after out-of-hospital cardiac arrest
Resuscitation
(2020)
An Utstein-based model score to predict survival to hospital admission: The UB-ROSC score
Int J Cardiol
Deep-learning-based out-of-hospital cardiac arrest prognostic system to predict clinical outcomes
Resuscitation
Outcome prediction of out-of-hospital cardiac arrest with presumed cardiac aetiology using an advanced machine learning technique
Resuscitation
Validation of the ROSC after cardiac arrest (RACA) score in Pan-Asian out-of-hospital cardiac arrest patients
Resuscitation
A national resuscitation registry of out-of-hospital cardiac arrest in Germany—A pilot study
Resuscitation
Machine learning algorithms performed no better than regression models for prognostication in traumatic brain injury
J Clin Epidemiol
A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models
J Clin Epidemiol
Cited by (3)
Artificial intelligence in emergency medicine. A systematic literature review
2023, International Journal of Medical InformaticsThe Intelligent Heart Rate Monitoring Model for Survivability Prediction of Cardiac Arrest Patients Using Deep Cardiac Learning Model
2023, Proceedings of the 2023 International Conference on Intelligent Systems for Communication, IoT and Security, ICISCoIS 2023
- 1
Contributed equally.
- 2
Singapore PAROS Investigators are listed after the conflict of interest statment.