Inter-Rater Agreement on Cincinnati Prehospital Stroke Scale (CPSS) and Prehospital Acute Stroke Severity Scale (PASS) Between EMS Providers, Neurology Residents and Neurology Consultants

Martin Gude; Hans Kirkegaard; Rolf Blauenfeldt; Anne Behrndtz; Jeppe Mainz; Ingunn Riddervold; Claus Z Simonsen; Niels Hjort; Søren P Johnsen; Grethe Andersen; Jan Brink Valentin

doi:10.2147/CLEP.S418253

Inter-Rater Agreement on Cincinnati Prehospital Stroke Scale (CPSS) and Prehospital Acute Stroke Severity Scale (PASS) Between EMS Providers, Neurology Residents and Neurology Consultants

Clin Epidemiol. 2023 Sep 7:15:957-968. doi: 10.2147/CLEP.S418253. eCollection 2023.

Authors

Martin Gude¹, Hans Kirkegaard^{1

2}, Rolf Blauenfeldt³, Anne Behrndtz³, Jeppe Mainz³, Ingunn Riddervold⁴, Claus Z Simonsen^{2

3}, Niels Hjort^{2

3}, Søren P Johnsen⁵, Grethe Andersen^{2

3}, Jan Brink Valentin⁵

Affiliations

¹ Department of Research and Development, Prehospital Emergency Medical Services, Central Denmark Region; and Aarhus University Hospital, Aarhus, Denmark.
² Department of Clinical Medicine, Aarhus University, Aarhus, Denmark.
³ Danish Stroke Center, Department of Neurology, Aarhus University Hospital, Aarhus, Denmark.
⁴ Norwegian Air Ambulance Foundation, Oslo, Norway.
⁵ Danish Center for Clinical Health Services Research, Department of Clinical Medicine, Aalborg University, Aalborg, Denmark.

Abstract

Objective: To examine the agreement between emergency medical service (EMS) providers, neurology residents and neurology consultants, using the Cincinnati Prehospital Stroke Scale (CPSS) and the Prehospital Acute Stroke Severity Scale (PASS).

Methods: Patients with stroke, transient ischemic attack (TIA) and stroke mimic were included upon primary stroke admission or during rehabilitation. Patients were included from June 2018 to September 2019. Video recordings were made of patients being assessed with CPSS and PASS. The recordings were later presented to the healthcare professionals. To determine relative and absolute interrater reliability in terms of inter-rater agreement (IRA), we used generalisability theory. Group-level agreement was determined against a gold standard and presented as an area under the curve (AUC). The gold standard was a consensus agreement between two neurology consultants.

Results: A total of 120 patient recordings were assessed by 30 EMS providers, two neurology residents and two neurology consultants. Using the CPSS and the PASS, a total of 1,800 assessments were completed by EMS providers, 240 by neurology residents and 240 by neurology consultants. The overall relative and absolute IRA for all items combined from the CPSS and PASS score was 0.84 (95% CI 0.80; 0.87) and 0.81 (95% CI 0.77; 0.85), respectively. Using the CPSS, the agreement on a group-level resulted in AUCs of 0.83 (95% CI 0.78; 0.88) for the EMS providers and 0.86 (95% CI 0.82; 0.90) for the neurology residents when compared with the gold standard. Using the PASS, the AUC was 0.82 (95% CI 0.77; 0.87) for the EMS providers and 0.88 (95% CI 0.84; 0.93) for the neurology residents.

Conclusion: The high relative and absolute inter-rater agreement underpins a high robustness/generalisability of the two scales. A high agreement exists across individual raters and different groups of healthcare professionals supporting widespread applicability of the stroke scales.

Keywords: emergency medical service provider; neurologists; observer variation; prehospital emergency care; stroke.

Grants and funding

The study was supported by a grant from the Danish non-profit foundation TrygFonden (grant number 117615) and by a grant from the Laerdal Foundation. The foundations did not influence the study, the drafting of the manuscript or the interpretation of the results.