Accumulating and visualizing tacit knowledge of teachers on educational assessments

Assessments play an important role in evaluating learners' comprehension of a subject and are created with teachers' implicit domain knowledge. Most elementary and junior high school teachers in Taiwan, and in many other countries as well, compile their assessments using test items taken from item banks provided by textbook publishers or from freely shared online item banks. Although these resources provide assessment writing assistance, few of them provide teachers' tacit knowledge in terms of the importance of both the concepts and their relationships of a subject, which, if captured, made explicit, and made available to teachers, can help construct more effective assessments, thereby improving teaching performance. This study builds a knowledge map-based assessment system to help elementary school teachers compile their assessments and collect their tacit knowledge of specific topics using an implicit knowledge extraction mechanism. The collected knowledge is made explicit as concrete weights of the concepts and their mutual relationships and is then displayed graphically as a knowledge map to guide teachers through assessment compiling. The knowledge extraction mechanism also operates in a real-time mode that analyses an assessment undergoing development and gives its author a real-time, updated knowledge map that reveals the overall distribution of its embedded concepts. Several experiments based on courses on science and technology of nature and life taught in elementary schools in Taiwan were conducted in this research. Eighteen teachers were involved in the experiments; the results clearly indicate the potential of this system, as all of the teachers agreed that the presentation of knowledge maps helped them to comprehend the proportions of concepts they intended to test and to notice concepts they may have ignored, and hence to create better assessments.