Wake-Up is a multicenter trial using imaging based criteria for entry, funded by the European Union, to determine if emergency stroke care (and r-tPa) can be extended to those who wake up with a stroke. Galinovic et al. seek to demonstrate in this paper how interrater agreement in FLAIR signal can be improved by using quantitative techniques. The ratio of DWI signal to FLAIR signal (or DWI-FLAIR mismatch) may provide important information about the timing of a stroke to less than 4.5 hours of onset, which is the cutoff for approved r-tpa use. The trial uses for inclusion in addition to the mismatch, the exclusion of hemorrhage, and a DWI lesion <1/3 the MCA territory.
As FLAIR positivity is subjective, the opinions of 2 senior neuroradiologist were taken as the “gold standard” and 6 individuals rated 143 cases. Additional raters processed a subset to compare relative signal intensity (rSI) in the contralateral tissue. An ROC-analysis was used to find the threshold, which corresponded with the “gold standard of FLAIR positivity. Raters on an additional subset of cases prospectively tested this threshold.
The raters had a low rate of agreement between each other (kappa <0.5). The signal threshold (called rSI) did not improve the interrater agreement. The importance of this study is that it demonstrates that aspects of FLAIR positivity cannot be quantified, and are indeed subjective (and dependent on the rater experience). Automated methods such as rSI can be used as a “back up” to the human eye, as the authors suggest, but did not add additional information when used with the subjective analysis. The efforts to standardize imaging biomarkers used in treatment decisions is important, especially to compare results across trials and look forward to extended trial results to a clinical population.