Facial expression are widely used for emotion recognition. Facial expressions may be expressed differently by different people subjectively, inaccurate results are unavoidable. Nevertheless, physiological reactions are non-autonomic nerves in physiology. The physiological reactions and the corresponding signals are hardly to control while emotions are excited. Therefore, an emotion recognition system with consideration of physiological signals is proposed in this paper. A specific designed mood induction experiment is performed to collect physiological signals of subjects. Five biosensors including electrocardiogram, respiration, galvanic skin responses (GSR), blood volume pulse, and pulse are used. Then a Support Vector Regression (SVR) is used to train three regression curves of three emotions (sad, fear, and pleasure). Experimental results show that the proposed method based on SVR emotion recognition has a good performance in accuracy.