Incorporating Monotonic Prior Knowledge in Support Vector Machines

Abstract

Support vector machine (SVM) is a state-of-the-art artificial neural network based on statistical learning For more than a decade SVM has drawn considerable attention from diverse research communities in data mining thanks to its outstanding performance in solving problems related to classification and function estimation It has been successfully applied to many different fields such as forecasting corporate distress consumer loan evaluation text categorization bioinformatics handwriting recognition and speaker verification The original idea of SVM is to use a linear separating hyperplane to create a classifier For non-linearly separable cases input vectors are mapped to a higher-dimensional feature space and the system will then easily construct the hyperplane which ensures high generalizability for classifying new objects In many data mining applications there is prior domain knowledge concerning the monotonic relations between the response and predictor variables and taking into account monotonicity may be an important model requirement with regard to explaining and justifying decisions Therefore this study firstly proposes a regularized monotonicity constrained SVM (RMC-SVM) that incorporates monotonic nature of the problems being considered In RMC-SVM a quadratic programming problem in the dual space is derived a Tikhonov regularization is utilized to ensure the access to the global solution and an algorithm implemented with a quadratic programming solver is developed Furthermore considering the fact that in many applications each input point may not be exactly labeled as one particular class this study extensively proposes a novel fuzzy SVM model to explore this issue It applies a fuzzy membership to each input point It also utilizes expert knowledge concerning the monotonic relations between the response and predictor variables which is represented in the form of monotonicity constrains The classification problem of a monotonically constrained fuzzy SVM called a regularized monotonic FSVM (RMC-FSVM) is formulated its dual optimization problem is derived and its monotonic property is theoretically analyzed The Tikhonov regularization method is also adopted to ensure that the solution is unique and bounded A new measure the frequency monotonicity rate is proposed to evaluate the ability of the model to retain the monotonicity When applied to some benchmark datasets the proposed RMC-SVM shows statistically significant advantages and promising results over the original SVM As for RMC-FSVM the results of the experiments on real-world and synthetic datasets show that it has a number of advantages with regard to predictive ability and retaining monotonicity over the original FSVM and SVM models when applied to classification problems

Date of Award	2014 Dec 4
Original language	English
Supervisor	Sheng-Tun Li (Supervisor)