Deep Reinforcement Learning with Adaptive-Halting Policy for Temporal Early Classification

Abstract

Prediction problem is always an important issue in machine learning No matter in manufacturing healthcare or finance it will response the solution when it recognizes opportunities or anomalies The benefit which the prediction problem creates is very considerable Time series prediction utilizes the time characteristics of an event in the past period of time to predict the possibility of the event occurring in the next period of time In real life most of studies on time series prediction mostly use multivariate time series Multivariate time series have many different characteristics at each time point and these characteristics can make the machine better understand the real situation With the development of reinforcement learning the machine tries to make the machine rely on itself and act according to the state of the environment to obtain the maximum expected benefits letting the machine learn how to make decisions in an unknown environment Deep Deterministic Policy Gradient (DDPG) proposed by Deep Mind can effectively analyze continuous signals and actions allowing machines to interact with high-dimensional spatial environments and learn to make the best decisions by themselves First our study uses DDPG as the main research framework and proposes a deep reinforcement learning network framework with adaptive halting policy for early prediction (EarlyDDPG) Second EarlyDDPG analyze multivariate time series and make early predictions Third we allow the machine to learn when to stop training and make a certain level of prediction Our study expects to use less time information to achieve the results of existing methods or even better prediction results Our research used the UCR time series classification database by the University of California Riverside to conduct our experiment We used the wafer dataset the ECG dataset and the GunPoint dataset as the experimental data set All of these datasets are all real datasets According to the experimental results EarlyDDPG has obtained good prediction results in most of cases Seconds the time information required for prediction of EarlyDDPG is at least 40% earlier than the previous literature which mean it can use less information to predicts close or better prediction results

Date of Award	2020
Original language	English
Supervisor	Sheng-Tun Li (Supervisor)