Online peer-assessment quality control: A proactive measure, validation study, and sensitivity analysis

Fu Yun Yu, Yu Hsin Liu, Kristine Liu

Research output: Contribution to journalArticlepeer-review

1 Citation (Scopus)

Abstract

Despite the generally positive learning effects of peer-assessment (PA), there are some undesirable signs and troublesome behavior needed to be addressed (e.g., biased judgments, superficial-level engagement). Anchoring on the concept of individual accountability and assessment validity, while attending to other important elements (including both subjective and objective criteria, individuals’ differentiation ability across the points on the assessment scale, and completion rate), a measure was proposed for PA quality control. This measure consists of four indexes/variables and enables the quantification and appraisal of the peer-assessor's performance. A validation study involving two classes of undergraduate students (N = 103) engaged in the assessment of peers’ oral presentations was conducted to evaluate the effects of the proposed measure. The statistically significant results confirmed the efficacy of the proposed measure to promote better online PA performance. Additionally, to assess its applicability, sensitivity analysis targeting different aspects of our proposed measure (i.e., different plausible weighting for each of the associated indexes/variables, different correlation values between peers’ and the teacher's ratings, and different bound values for the scale) was conducted. All significant results of the 281 test cases further substantiated the applicability and efficacy of our proposed measure under various assessment schemes for PA quality control.

Original languageEnglish
Article number101279
JournalStudies in Educational Evaluation
Volume78
DOIs
Publication statusPublished - 2023 Sept

All Science Journal Classification (ASJC) codes

  • Education

Fingerprint

Dive into the research topics of 'Online peer-assessment quality control: A proactive measure, validation study, and sensitivity analysis'. Together they form a unique fingerprint.

Cite this