Enhancing Comprehension of Lecture Content in a Foreign Language as the Medium of Instruction: Comparing Speech-to-Text Recognition With Speech-Enabled Language Translation

Scholars suggest that not every student completely comprehends the content of a lecture in a foreign language as the medium of instruction, especially in the case of those with low language ability. To facilitate comprehension of lectures in a foreign language, learning content was presented to students in multiple modalities; that is, in addition to verbal (speech of the instructor) and visual (lecture slides) content, texts generated by speech-to-text recognition (STR) or speech-enabled language translation (SELT) were shown to the students. The goal was to compare how these two additional content modalities (i.e., STR-texts vs. SELT-texts) facilitate student comprehension of lecture content. Because processing multimodal content requires additional cognitive resources, another goal was to explore whether STR-texts versus SELT-texts impose any cognitive load on the students. To this end, two groups of students were recruited, where they attended two lectures at the intermediate and advanced levels. STR-texts were shown to a control group, and SELT-texts were shown to an experimental group. The posttest results and the cognitive load of the students in both groups after each lecture were compared. Four main findings were obtained: (a) The experimental group outperformed the control group on both posttests. However, when student language ability was considered, the difference was statistically significant for low ability students only; (b) there was not a significant between-group difference in cognitive load; however, if student language ability was considered, a significant between-group difference existed during the more difficult lecture; (c) between-group differences in self-efficacy were statistically insignificant; and (d) associations among some research variables were found. Based on these results, several implications were drawn for the teaching and research community.