Online Student-Generated Questions with Different Levels of Structure: System Development, Instructional Design, and Learning Assessment

Generally, the empirical evidence supporting the learning effects of student-generated questions (SGQ) abound, and numerous online learning systems are available to support SGQ activities. However, existing studies mostly centered on ‘free’ SGQ situations. For those that examined ‘none-free’ SGQ situations (i.e., semi- and structured SGQ, e.g., testlets), all existing studies were contextualized in mathematics. Moreover, regardless of the frequent presence of testlets in today’s testing practice, the potential of student-generated testlets is yet to be realized, nor are online learning systems available to support student-generated testlets. In view of these, the main goals of the 3-year project are two-fold: first, to design and develop a versatile, scaffolded, pro-deep online learning system for student-generated testlets for classroom use; second, to assess its learning effects. To attain this, for the first year, general principles are set up (i.e., flexibility, cognitive supports, and metacognitive supports) to guide the design and development of the focal online system. Then, the usability and learning potential of various online SGQ structures in terms of problem situations (i.e., free SGQ vs. testlets), formats (i.e., text-based vs. graphics-based), and task characteristics (i.e., given condition: none, scenario only, or scenario + item sets; editability) will be examined. For the second year, the comparative learning effects of online student-generated testlets relative to online free SGQ will be investigated so that the presumably superiority of student-generated testlets for inviting the exploration and examination of problem statements and solution ideas, which, in view of cognitive elaboration theory, may lead to deeper understanding, more employment of elaboration strategy, better question-generation performance, and elaborated knowledge construction will be empirically attested. In addition, knowing that both text-based and graphics-based scenarios are frequently seen in testlets, the comparative learning effects will be examined in both formats for external validity establishment. For the third year, the task characteristics of online SGQ structures are targeted. Guided by two sets of competing theorizing—scaffolding theory and cognitive load theory (for given conditions) vs. self-determination theory and motivational theory (for giving degrees of freedom), the learning effects of online student-generated testlets under different given conditions (i.e., none, scenario only, scenario + items set; editability) on academic achievement, task performance, cognitive load, learning motivation, and satisfaction toward the past learning experience will be examined. Similarly, the external validity of the study will be established. To attain these goals, for the first year, following the design and development of the focal online learning system, it will undergo the usability assessment via eighteen 1-1 evaluation studies and a field-trial evaluation study to collect data on perceived learning usefulness, ease of use, and preference toward the nine different SGQ arrangements. For the second and third years, a pretest-posttest quasi-experimental research design will be used. In addition, a set of open-ended questions will be used to gather descriptive data from the participants. A combination of one-group t-tests, an analysis of covariance technique, regression analysis, chi-square tests, content analysis, and a constant comparison method will be used for data analysis.