AICURE: Pre-training of Cross-Modality Transformer Encoder for ICU Electronic Health Records Prediction

Abstract

Recently there have been increasing researches applying deep learning on electronic health records (EHR) However their methodologies and task definitions are very diverse and most past works depended solely on medical codes for learning patient's health status and making prediction These problems limit applicability of deep learning models on medical domain To solve difficulties above we propose AICURE (a ICU record encoder) an encoder pre-trained on each visit record for learning good visit vector and then fine-tuned on each EHR prediction task Dataset here comprises visit records which contain medical codes clinical notes and patients' demographics We adapt cross-modality learning for combining information from different domains and introduce the concept in natural language processing for learning patient's medical history i e visit record sequence And we design 4 EHR tasks based on actual clinical scenario situations for more proper definition Because our pre-trained AICURE learns good visit vectors it can be applied to many EHR tasks and has competitive performances on these 4 tasks Moreover our model can provide interpretable predictions by visualizing inference procedure In the end we analyze performance and abilities of AICURE by case study

Date of Award	2021
Original language	English
Supervisor	Hung-Yu Kao (Supervisor)