Research Article
Austin J Cerebrovasc Dis & Stroke. 2017; 4(4): 1068.
Non-enhanced CT Maximum Intensity Projections for the Detection of Large Vessel Occlusions
Ho JP*, Nguyen DT, Pirastefahr M, Narula R, Hailey L, Mortin M, Rapp K, Agrawal K, Huisa- Garate B, Modir R, Meyer DM, Hemmen TM and Meyer BC
Department of Neurosciences, University of California San Diego, USA
*Corresponding author: Ho JP, Department of Neurosciences, University of California, San Diego 200 West Arbor Drive Medical Offices North, 3rd Floor, Suite 3 San Diego, CA 92103, USA
Received: May 24, 2017; Accepted: June 30, 2017; Published: July 13, 2017
Abstract
Background: Identification of large vessel occlusions (LVO) is important with recent guidelines supporting endovascular therapy in selected acute ischemic stroke patients. Many stroke centers perform CT angiography (CTA) in patients with suspected LVO; however this requires additional time and contrast administration. Non-enhanced CT maximum intensity projection (NECT-MIPs) may offer a rapid alternative to CTA.
Methods: We retrospectively reviewed acute stroke patients with LVO in the UCSD Stroke Registry, presenting between 6/2014-7/2016. NECT-MIPs were evaluated for presence of LVO. Gold standard comparison was to CTA. Results were stratified by level of training (Faculty, Fellow and Acute Care Practitioners [ACPs]). Inter-rater agreement was assessed using Fleiss’ Kappa Coefficient.
Results: We reviewed 24 patients using NECT-MIPs for the detection of LVO. Faculty had a sensitivity and specificity of 95% & 92% for ICA/M1, 42% & 100% for M2, and 67% & 96% for basilar occlusions. Fellows and ACPs had a sensitivity and specificity of 61% & 94% for ICA/M1, 19% & 83% for M2, and 75% & 95% for basilar occlusions. Inter-rater agreement among Faculty readers was k=0.75 for ICA/M1, k=0.79 for M2 and k=0.14 for basilar occlusions. Among Fellows and ACPs, k=0.57 for ICA/M1, k=0.40 for M2, and k=0.27 for basilar occlusions.
Conclusion: NECT-MIPs have high sensitivity and specificity for the detection of LVO when compared to CTA. Inter-rater agreement is fair and higher amongst more experienced reviewers. These results suggest that NECTMIPs may be helpful to streamline the identification of LVO and reduce door to needle and door to intervention times.
Keywords: CT; CTA; Acute ischemic stroke; Neuroradiology; Reliability
Introduction
Rapid detection of large vessel occlusions (LVO) in acute stroke has become increasingly important in light of recent endovascular trials [1-5] with AHA/ASA guidelines recommending endovascular therapy in select patients with a LVO of the anterior circulation [6]. Digital subtraction angiography (DSA) remains the gold standard for detection of LVO, but is an invasive procedure that requires time and the availability of neuro-interventionalists [7].
Non-enhanced CT (NECT) of the head is an important tool in acute stroke evaluations that is widely available and primarily used for the detection of intracranial hemorrhage or other contraindications to the use of IV tPA [7-9]. Subtle signs of an acute ischemic stroke may be seen on NECT, such as early ischemic changes to the parenchyma or the visualization of a hyper dense vessel suggestive of a large vessel occlusion [10-12]. Standard NECTs are rarely used for this purpose however, mostly due to their low sensitivities for the detection of LVO. The presence of a hyper dense internal carotid artery (ICA) on standard NECT has a sensitivity of only 30% for the detection of an occlusion of the ICA [13], while a hyper dense middle cerebral artery (MCA) has reported sensitivities of 28% - 45% [13,14]. Though poorly sensitive, it is highly specific and when seen, the presence of a hyper dense MCA has been associated with severe neurological deficits, extensive brain damage, and poor clinical outcomes [15,16]. Similarly the presence of a hyper dense basilar artery on NECT is a strong predictor for basilar artery occlusion and poor functional outcomes [17]. Aside from its prognostic value, the presence of a hyper dense vessel is of further clinical importance in light of previous studies suggesting that a hyper dense MCA predicts a poorer outcome in response to IV tPA [15,18], with more favorable outcomes seen from intra-arterial therapy when compared to IV tPA alone [19].
CT angiography (CTA) is a rapid and non-invasive imaging modality that is highly accurate for the detection of large vessel intracranial stenosis or occlusion [7,20] and may be preferred as the initial diagnostic test when evaluating for LVO. CTA is limited by the need for contrast administration, additional radiation exposure, and a skilled technician and radiologist to administer and interpret the images in an acute setting [21]. It also takes added time to perform after a non-contrast CT, thus delaying treatment times. Patients with contraindications to iodinated contrast (i.e. severe renal failure or significant contrast allergies), or who are evaluated in settings that lack the resources to acutely obtain and interpret CTAs are often precluded from this imaging. A current strategy in patients with severe contrast allergies is the use of appropriate pre-medications techniques. The most common protocols recommend pre-medication thirteen hours before administration of contrast [22], with more acute emergency protocols recommending pre-medication at least one hour before administration of contrast [23]. This is impractical in the acute stroke setting when time is critical. Magnetic Resonance Angiography (MRA) is a non-contrast modality for the detection of LVO though it is also limited by time and availability. Many patients in rural medical settings are ultimately unable to obtain acute vessel imaging. There remains an unmet need for an alternative imaging modality to detect LVO rapidly in the acute stroke situation, and to simplify and streamline the acute pre-treatment evaluation.
Prior studies have reported that thin-slice reconstructions of NECT images can improve detection rates of thrombus in the MCA and be used to accurately quantify thrombotic burden on NECT in acute ischemic stroke [24,25]. The goal of this study is to investigate whether thin image slices of NECT reconstructed with maximum intensity projections (NECT-MIPs) would have a high sensitivity and reliability for the detection of LVO, providing an alternative imaging modality for acute stroke patients unable to obtain CTA.
Materials and Methods
Patient selection
We retrospectively reviewed consecutive patients that were prospectively enrolled in our IRB approved, UCSD Stroke Code Registry between June 2014 and July 2016. All patients with concern for acute ischemic stroke received a NECT and CTA Head and Neck consistent with our institution’s stroke code protocol. Patients with an acute ischemic stroke that had a proximal large vessel occlusion in the ICA/M1, M2 or basilar arteries on CTA were included. Cases were excluded if no CTA was done, if MIPs were not generated, if imaging was done at an outside institution (e.g. transfer cases), if there was a prior known high grade intracranial stenosis/occlusion, dissection, or an occlusion located in an artery other than ICA/M1, M2, or basilar. For each patient that fit the criteria, we included a patient that had a NECT-MIPs generated but did not have an intracranial large vessel occlusion on CTA.
Image acquisition
All patients were scanned on a helical, multi-detector CT scanner with 320 detector rows (Toshiba Aquilion One, Toshiba American Medical Systems). NECT protocol consists of a collimation of 32 x 0.5 mm, at a voltage of 120 kV and a current of 240 mA. Thin 0.5 mm slice images are reconstructed into standard 5mm thick NECT images. NECT-MIPs are generated from these thin slice reconstructions using maximum intensity projections, with the subsequent reconstructed 5 mm thick slices overlapping by 1 mm. CTA protocol consists of a bolus of 75 ml of iodinated contrast at 4 ml/sec, using a collimation of 80 x 0.5 mm, at a voltage of 120 kV and a current of 200 mA, and reconstructed into 2 mm images. Axial and coronal source images of NECT, NECT-MIPs, and CTA were obtained according to the protocols above. Image sets were de-identified, and randomly assigned a unique ID number.
Image analysis
De-identified axial and coronal images of standard NECT and NECT-MIPs were reviewed by seven individual stroke specialists (three experienced Faculty Stroke Specialists, two Vascular Neurology Fellows, and two Advanced Care Practitioners [ACPs]). Each reader evaluated the images independently. No clinical information was provided. CTA images were not provided to avoid bias. Images were reviewed with our standard imaging software (Impax 6.6.1, Agfa Health Care). Readers scored the images based on whether they detected a hyper dense vessel on the NECT-MIPs, and in which location. They then repeated this with standard NECT images. Images were provided in a random order each time so as to reduce bias. Readers were not informed of the number of positive versus negative LVO cases included in the data set for review, or that each positive case had a matched negative case.
Statistical analysis
Sensitivities and specificities were calculated from the results obtained after image analysis, using official radiology CTA read as the gold standard. Sensitivities and specificities for the detection of “All LVO” were calculated based on whether readers detected any large vessel occlusion, regardless of whether the correct location of the LVO was identified. In comparison, ICA/M1, M2 and basilar occlusion sensitivities and specificities were based on correct identification of the large vessel occlusion.
Analyses were calculated for each individual stroke specialist, as well as cumulatively for each of five different sub-groups (All Readers, Faculty Stroke Specialists, Vascular Neurology Fellows, Advanced Care Practitioners, and Vascular Neurology Fellows + Advanced Care Practitioners). Inter-rater agreement for each sub-group was calculated using Fleiss’ Kappa Coefficient in order to account for multiple readers in each sub-group [26].
Results
Of the 12 patients who fit the inclusion and exclusion criteria 7 patients had an ICA/M1, 4 patients had an M2, and 1 patient had a basilar artery occlusion. For each patient that fit the criteria, we included a patient evaluated for an acute stroke who had a NECTMIPs generated but did not have a large vessel occlusion on CTA, for a total of 24 patients.
Sensitivities and specificities for the detection of LVO using NECT-MIPs for each sub-group of reader, and further stratified by location of the large vessel occlusion, are listed in Table 1. The “Faculty Stroke Specialists” sub-group generally had higher sensitivities for the detection of a large vessel occlusion, particularly for the ICA/M1 occlusions where sensitivity reached 95%. The “Vascular Neurology Fellows”, “Advanced Care Practitioners”, and “Vascular Neurology Fellows + Advanced Care Practitioners” sub-groups had moderate to good sensitivity for the detection of a ICA/M1 occlusion, but still poorer than the “Faculty Stroke Specialists” sub-group. Sensitivities dropped considerably for the detection of M2 occlusions in all groups.
All LVO
ICA/M1
M2
Basilar
All Readers
Sen = 0.79
Spe = 0.83
Sen = 0.76
Spe = 0.93
Sen = 0.29
Spe = 0.90
Sen = 0.71
Spe = 0.95
Faculty Stroke Specialists
Sen = 0.81
Spe = 0.86
Sen = 0.95
Spe = 0.92
Sen = 0.42
Spe = 1.00
Sen = 0.67
Spe = 0.96
Vascular Neurology Fellows
Sen = 0.67
Spe = 0.79
Sen = 0.71
Spe = 0.94
Sen = 0.00
Spe = 0.83
Sen = 1.00
Spe = 1.00
Advanced Care Practitioners
Sen = 0.88
Spe = 0.83
Sen = 0.50
Spe = 0.94
Sen = 0.38
Spe = 0.83
Sen = 0.50
Spe = 0.89
Vascular Neurology Fellows + Advanced Care Practitioners
Sen = 0.77
Spe = 0.81
Sen = 0.61
Spe = 0.94
Sen = 0.19
Spe = 0.83
Sen = 0.75
Spe = 0.95
Table 1: Sensitivity and Specificities for the detection of a large vessel occlusion with NECT-MIPs, using CTA as the gold standard, grouped by location and reader.
Inter-rater agreement (Fleiss’ Kappa Coefficient) for each subgroup of reader, and further stratified by location of the large vessel occlusion, are listed in Table 2. Inter-rater agreement was good for the “Faculty Stroke Specialists” sub-group for detection of ICA/ M1 (k=0.75) and M2 occlusions (k=0.79). It was also good for the “Vascular Neurology Fellows” sub-group for the detection of M2 (k=0.83) and basilar artery occlusions (k=1.0).
All LVO
ICA/M1
M2
Basilar
All Readers
K = 0.46
K = 0.57
K = 0.27
K = 0.28
Faculty Stroke Specialists
K = 0.58
K = 0.75
K = 0.79
K = 0.14
Vascular Neurology Fellows
K = 0.72
K = 0.56
K = 0.83
K = 1.00
Advanced Care Practitioners
K = 0.51
K = 0.59
K = 0.49
K = 0.24
Vascular Neurology Fellows + Advanced Care Practitioners
K = 0.48
K = 0.57
K = 0.40
K = 0.27
Table 2: Inter-rater reliability for the detection of a large vessel occlusion grouped by location and reader.
Sensitivities and specificities for the detection of LVO using standard NECT are listed in Table 3. The “Faculty Stroke Specialists” sub-group had a sensitivity of 43% the detection of ICA/M1 occlusions using standard NECT, which is lower than the sensitivity of 95% when NECT-MIPs were used. Similarly, sensitivities and specificities for the detection of ICA/M1 occlusions were lower for all sub-groups of readers when using standard NECT compared to NECT-MIPs.
All LVO
ICA/M1
M2
Basilar
All Readers
Sen = 0.67
Spe = 0.70
Sen = 0.48
Spe = 0.79
Sen = 0.25
Spe = 0.94
Sen = 0.67
Spe = 0.90
Faculty Stroke Specialists
Sen = 0.46
Spe = 0.92
Sen = 0.43
Spe = 0.94
Sen = 0.13
Spe = 0.98
Sen = 0.00
Spe = 0.93
Vascular Neurology Fellows
Sen = 0.88
Spe = 0.67
Sen = 0.57
Spe = 0.68
Sen = 0.38
Spe = 0.90
Sen = 1.00
Spe = 0.89
Advanced Care Practitioners
Sen = 0.62
Spe = 0.50
Sen = 0.43
Spe = 0.76
Sen = 0.25
Spe = 0.93
Sen = 1.00
Spe = 0.87
Vascular Neurology Fellows + Advanced Care Practitioners
Sen = 0.75
Spe = 0.59
Sen = 0.50
Spe = 0.72
Sen = 0.32
Spe = 0.92
Sen = 1.00
Spe = 0.88
Table 3: Sensitivity and Specificities for the detection of a large vessel occlusion with standard NECT, using CTA as the gold standard, grouped by location and reader.
Discussion
Historically, patients are evaluated for LVO acutely via NECT or CTA scans. The yield of detecting LVO using standard NECT is low [13,14]. Even using modern multi-detector row CT scanners, a recent study has reported a detection rate only as high as 40% for middle cerebral artery occlusions using standard NECT [27]. CTAs improve the detection rate of LVOs, but add the risk of contrast exposure and take more time. Time is already limited in the acute stroke pre-treatment time period, and national efforts continue to recommend finding ways to lessen the door to treatment window [28]. Furthermore, many rural medical centers do not have the capabilities to perform CTA acutely.
Our results suggest that MIPs can be used to increase the sensitivity of NECT for the detection of LVO (Figure 1), which may potentially allow for streamlining of the acute pre-treatment time period as patients with LVO detected on NECT-MIPs would thus not need a CTA. This is especially critical for patients who cannot have a CTA due to logistic or clinical reasons. For our cohort of patients, the “Faculty Stroke Specialists” sub-group had a sensitivity of 95% when using NECT-MIPs for the detection of ICA/M1 occlusions. As expected this is much higher than the sensitivity of 43% obtained when the same sub-group used standard NECT. In the “Vascular Neurology Fellows + Advanced Care Practitioners” sub-group, the sensitivity of 61% when using NECT-MIPs, was much lower than that of the “Faculty Stroke Specialists” sub-group. Although speculative, this discrepancy likely reflects a degree of difference in the experience of interpreting imaging.
Figure 1: Images of standard NECT (left) and NECT-MIPs (right) of the same patient with an acute ischemic stroke and left M1 occlusion. The NECT-MIPs (right) shows a much more obvious hyper dense vessel sign extending along the left middle cerebral artery when compared to the standard NECT (left).
In all sub-groups there was a lesser sensitivity for detecting M2 as compared to ICA/M1 occlusions. This was not unexpected given the progressively smaller diameters of these arteries. For the detection of basilar artery occlusions, the “Vascular Neurology Fellows” and “Vascular Neurology Fellows + Advanced Care Practitioners” sub- groups had a higher sensitivity than the “Faculty Stroke Specialists” sub-group but this may be due to chance because of the small group size (basilar artery occlusion n=1).
The good inter-rater agreement for the “Faculty Stroke Specialists” sub-group for the detection of ICA/M1 and M2 occlusions suggests that NECT-MIPs is reliable, when evaluated between experienced readers. For the detection of ICA/M1 occlusions, Kappas were lower in the “Vascular Neurology Fellows” and “Advanced Care Practitioners” sub-groups suggesting that less experienced readers had poorer inter-rater reliability. Perfect inter-rater agreement was noted in the “Vascular Neurology Fellows” sub-group for the detection of basilar artery occlusion, but again this may be due to the small group size.
Our data report does have limitations. In this study we compared MIPs to the surrogate of CTAs rather than traditional cerebral angiograms. Prior studies have shown high sensitivity and specificity when comparing CTA to DSA [7,20]. Further studies corroborating our findings versus a true gold standard of digital subtraction angiography (DSA) should be performed. We also sought to provide a comparison of MIPs to the “real world” scenario where CTA is often the modality of choice in advanced acute stroke decision-making (e.g. who will go on to possible embolectomy care). Another limitation included the relatively few number of patients included in this study and small subsets. Further studies in this regard are underway. Finally, imaging raters in this study were all stroke clinicians. Although the addition of a neuro-radiologist would likely improve the accuracy of diagnosis, our aim was to replicate the acute stroke code scenario where stroke specialists are often interpreting these images for acute medical decision making purposes.
The findings of this study suggest that MIPs may be used to detect LVO when CTA is not possible. This may also decrease contrast exposure and further streamline the acute stroke pre-treatment (door to needle, or door to puncture) time intervals. This finding is particularly important due to the increasing systematic desire to determine presence of occlusions rapidly in the acute stroke code time period to expedite IV t-PA treatment and endovascular therapies. The poor reliability of standard NECT, and the current limitations of CTA (time, contrast exposure, expertise), under-score the need for an alternate modality to detect LVO, that is rapid, non-invasive, and can be used accurately and reliably in acute stroke code patients.
Study Funding
Research reported in this publication was supported by the National Institutes of Health, under award number U10NS086535. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
References
- Berkhemer OA, Fransen PS, Beumer D, van den Berg LA, Lingsma HF, Yoo AJ, et al. A randomized trial of intraarterial treatment for acute ischemic stroke. N Engl J Med. 2015; 372: 11-20.
- Campbell BC, Mitchell PJ, Kleinig TJ, Dewey HM, Churilov L, Yassi N, et al. Endovascular therapy for ischemic stroke with perfusion-imaging selection. N Engl J Med. 2015; 372: 1009-1018.
- Goyal M, Demchuk AM, Menon BK, Eesa M, Rempel JL, Thornton J, et al. Randomized assessment of rapid endovascular treatment of ischemic stroke. N Engl J Med. 2015; 372: 1019-1030.
- Jovin TG, Chamorro A, Cobo E, de Miquel MA, Molina CA, Rovira A, et al. Thrombectomy within 8 hours after symptom onset in ischemic stroke. N Engl J Med. 2015; 372: 2296-2306.
- Saver JL, Goyal M, Bonafe A, Diener HC, Levy EI, Pereira VM, et al. Stentretriever thrombectomy after intravenous t-pa vs. T-pa alone in stroke. N Engl J Med. 2015; 372: 2285-2295.
- Powers WJ, Derdeyn CP, Biller J, Coffey CS, Hoh BL, Jauch EC, et al. 2015 american heart association/american stroke association focused update of the 2013 guidelines for the early management of patients with acute ischemic stroke regarding endovascular treatment: A guideline for healthcare professionals from the American heart association/american stroke association. Stroke. 2015; 46: 3020-3035.
- Jauch EC, Saver JL, Adams HP, Jr., Bruno A, Connors JJ, Demaerschalk BM, et al. Guidelines for the early management of patients with acute ischemic stroke: A guideline for healthcare professionals from the American heart association/American stroke association. Stroke. 2013; 44: 870-947.
- Tissue plasminogen activator for acute ischemic stroke. The national institute of neurological disorders and stroke rt-pa stroke study group. N Engl J Med. 1995; 333: 1581-1587.
- Wahlgren N, Ahmed N, Dávalos A, Ford GA, Grond M, Hacke W, et al. Thrombolysis with alteplase for acute ischaemic stroke in the Safe Implementation of Thrombolysis in Stroke-Monitoring Study (SITS-MOST): an observational study. Lancet. 2007; 369: 275-282.
- Moulin T, Cattin F, Crepin-Leblond T, Tatu L, Chavot D, Piotin M, et al. Early ct signs in acute middle cerebral artery infarction: Predictive value for subsequent infarct locations and outcome. Neurology. 1996; 47: 366-375.
- Von Kummer R, Allen KL, Holle R, Bozzao L, Bastianello S, Manelfe C, et al. Acute stroke: Usefulness of early ct findings before thrombolytic therapy. Radiology. 1997; 205: 327-333.
- Marks MP, Holmgren EB, Fox AJ, Patel S, von Kummer R, Froehlich J. Evaluation of early computed tomographic findings in acute ischemic stroke. Stroke. 1999; 30: 389-392.
- Ozdemir O, Leung A, Bussiere M, Hachinski V, Pelz D. Hyperdense internal carotid artery sign: A ct sign of acute ischemia. Stroke. 2008; 39: 2011-2016.
- Kim EY, Lee SK, Kim DJ, Suh SH, Kim J, Heo JH, et al. Detection of thrombus in acute ischemic stroke: Value of thin-section noncontrast-computed tomography. Stroke. 2005; 36: 2745-2747.
- Tomsick T, Brott T, Barsan W, Broderick J, Haley EC, Spilker J, et al. Prognostic value of the hyperdense middle cerebral artery sign and stroke scale score before ultraearly thrombolytic therapy. AJNR Am J Neuroradiol. 1996; 17: 79-85.
- Manelfe C, Larrue V, von Kummer R, Bozzao L, Ringleb P, Bastianello S, et al. Association of hyperdense middle cerebral artery sign with clinical outcome in patients treated with tissue plasminogen activator. Stroke. 1999; 30: 769-772.
- Goldmakher GV, Camargo EC, Furie KL, Singhal AB, Roccatagliata L, Halpern EF, et al. Hyperdense basilar artery sign on unenhanced ct predicts thrombus and outcome in acute posterior circulation stroke. Stroke. 2009; 40: 134-139.
- von Kummer R, Meyding-Lamadé U, Forsting M, Rosin L, Rieke K, Hacke W, et al. Sensitivity and prognostic value of early CT in occlusion of the middle cerebral artery trunk. AJNR Am J Neuroradiol. 1994; 15: 9-15.
- Mattle HP, Arnold M, Georgiadis D, Baumann C, Nedeltchev K, Benninger D, et al. Comparison of intraarterial and intravenous thrombolysis for ischemic stroke with hyperdense middle cerebral artery sign. Stroke. 2008; 39: 379- 383.
- Bash S, Villablanca JP, Jahan R, Duckwiler G, Tillis M, Kidwell C, et al. Intracranial vascular stenosis and occlusive disease: Evaluation with ct angiography, mr angiography, and digital subtraction angiography. AJNR Am J Neuroradiol. 2005; 26: 1012-1021.
- Merino JG, Warach S. Imaging of acute stroke. Nat Rev Neurol. 2010; 6: 560-571.
- Lasser EC, Berry CC, Talner LB, Santini LC, Lang EK, Gerber FH, et al. Pretreatment with corticosteroids to alleviate reactions to intravenous contrast material. N Engl J Med. 1987; 317: 845-849.
- Greenberger PA, Halwig JM, Patterson R, Wallemark CB. Emergency administration of radiocontrast media in high-risk patients. J Allergy Clin Immunol. 1986; 77: 630-634.
- Riedel CH, Jensen U, Rohr A, Tietke M, Alfke K, Ulmer S, et al. Assessment of thrombus in acute middle cerebral artery occlusion using thin-slice nonenhanced computed tomography reconstructions. Stroke. 2010; 41: 1659-1664.
- Riedel CH, Zoubie J, Ulmer S, Gierthmuehlen J, Jansen O. Thin-slice reconstructions of nonenhanced CT images allow for detection of thrombus in acute stroke. Stroke. 2012; 43: 2319-2323.
- Fleiss J. Measuring nominal scale agreement among many raters. Psychological Bulletin. 1971; 76: 378-382.
- Topcuoglu MA, Arsava EM, Akpinar E. Clot characteristics on computed tomography and response to thrombolysis in acute middle cerebral artery stroke. J Stroke Cerebrovasc Dis. 2015; 24: 1363-1372.
- Fonarow GC, Zhao X, Smith EE, Saver JL, Reeves MJ, Bhatt DL, et al. Doorto- needle times for tissue plasminogen activator administration and clinical outcomes in acute ischemic stroke before and after a quality improvement initiative. JAMA. 2014; 311: 1632-1640.