Estimating the Reliability of Aggregated and Within-Person Centered Scores in Ecological Momentary Assessment

A procedure for estimating the reliability of test scores in the context of ecological momentary assessment (EMA) was proposed to take into account the characteristics of EMA measures. Two commonly used test scores in EMA were considered: the aggregated score (AGGS) and the within-person centered score (WPCS). Conceptually, AGGS and WPCS represent the interindividual differences and the intraindividual differences, respectively. The reliability coefficients for AGGS and WPCS were derived using a multilevel factor model with a serial correlation structure framework. Point estimates and confidence intervals of these coefficients were obtained using Mx (Neale, Boker, Xie, & Maes, 2004). A simulation study showed that the proposed procedure performed well empirically. Diary data from Huang (2009), which recorded daily joy level of 110 undergraduate students for 8 days, was used to illustrate the applicability of the proposed method.