McArthur KS, Johnson PCD, Quinn TJ, Higgins P, Langhorne P, Walters MR. Improving the Efficiency of Stroke Trials: Feasibility and Efficacy ofGroup Adjudication of Functional End Points. Stroke. 2013
MacArthur et. al performed an interesting study to demonstrate that group adjudication of stroke functional outcome using the modified Rankin Score (mRS) had better kappa (interobserver agreement score) than individually scored mRS. Local site investigators performed mRS interviews and submitted their scores to the registry. Blinded coordinating center adjudicators then reviewed recorded video interviews and individually adjudicated. Where there was disagreement, they conferred and then submitted a group score.
Using sample size calculations, they demonstrated this can potentially reducing the trial sample size and increasing trial power by improving the reliability of the of the mRS assessments.
An interesting sub-study they performed was to use translator services for group adjudication of mRS for primary interviews performed in Chinese. The interobserver agreement continued to remain high (kappa=0.9)
This suggests that group adjudication can potentially improve the reliability of mRS functional outcomes and thus, potentially reduce the onerous burden of large sample size in intervention studies that may have modest measurable benefit in functional outcomes.
While I like the idea of have individual and subsequent group adjudications, this does add additional costs in other ways to the clinical trial process with recording video and subsequent transfer to coordinating centers as well as additional adjudication work by the coordinating center. Due to advances in technology, it’s possible that these recordings of subject interviews can be feasible and replicated. In addition, it’s possible that having the mRS interviews be recorded may improve the quality of the mRS interviews being conducted.
Anything that can improve the power of studies and reduce sample size and thus, reduce the cost and duration of clinical studies should be looked upon favorably as barriers to bringing good interventions to market and community use is still high and ever growing.