Deep Learning for Monotonic Support Vector Machines

Deep learning, which has the ability to extract significant patterns from dataset, becomes increasingly important with regard to the information age that has the characteristics of diversity and large volume of data. Compared with deep neural networks (DNN), support vector machine (SVM) is one of the most representative models of shallow neural networks (SNN), and has been widely used in various fields since it has the ability to process high-dimensional dataset effectively. Furthermore, by considering the prior knowledge of monotonicity, SVM can enhance its generalization capability against noise data. Monotonicity-constrained SVM (MC-SVM) is a model that take prior knowledge of monotonicity into account and this model is proven to outperform conventional SVM. To benefit from the advantage of deep neural network on feature extraction and performance of MC-SVM, we propose a deep learning model called deep MC-SVM (DMC-SVM) that takes MC-SVM as feature extractor to improve the generalization property of the classifier, especially when solving complicated classification problem. We took two real-world datasets, Wisconsin Breast Cancer Diagnostic (WBCD) and PD600 as our training datasets, and the experiment results demonstrated that DMC-SVM could extract more abstract features that make this model more generalized, and for this reason, DMC-SVM has the better ability while dealing with classification problems.