Inter-rater agreement on the classification of job titles

C. Y. Li, K. H. Chang, C. K. Feng, S. C. Wu

Research output: Contribution to journalArticlepeer-review

5 Citations (Scopus)

Abstract

Objectives: To assess the inter-rater reliability for the Standard Occupational Classification System (SOCS) of the Republic of China. Methods: Information on the life-time longest held occupation for a total of 145 adults 65 years and over was obtained from a longitudinal study using questionnaires to interview nearly 1600 community elderly on their work histories. Two investigators independently reviewed participants' work histories and categorized each participant's longest-held occupation into a job title designated by a four-digit number. Briefly, the SOCS is based on a 4-digit system. The first digit in the series represents the broadest job category while the subsequent digits are indicative of more specific details. The chance-adjusted agreement between the two raters was assessed by Kappa(K) and its 95% confidence interval(CI). Results: Among the 145 records studied, these two raters agreed on the first digit in the categorization for 110(75.8%) records (K = 0.69, 95% CI = 0.62-0.78). They also agreed on the first two digits for 106 (73.4%) records (K = 0.70, 95% CI = 0.61-0.77), the first three digits for 101 (70.0%) records (K = 0.66 ' 95% CI = 0.58-0.74), and all of the four digits for 98 (68.6%) records (K = 0.64, 95% CI = 0.56-0.72). Causes of disagreements included insufficient job information in the questionnaire, rater's mistakes, and two different SOCS classification codes for similar types of jobs. Conclusions: This study suggests that the reliability of the SOCS was substantial. Further improvement in the reliability can be achieved by training raters with standardized rating procedures and by providing sufficient information on work history to the raters.

Original languageEnglish
Pages (from-to)255-261
Number of pages7
JournalChinese Journal of Public Health
Volume18
Issue number4
Publication statusPublished - 1999 Jan 1

All Science Journal Classification (ASJC) codes

  • Public Health, Environmental and Occupational Health

Fingerprint Dive into the research topics of 'Inter-rater agreement on the classification of job titles'. Together they form a unique fingerprint.

Cite this