A digital circuit design of hyperbolic tangent sigmoid function for neural networks

This paper presents a digital circuit design approach for a commonly used activation function, hyperbolic tangent sigmoid functions, for neural networks. Our design concept for such a nonlinear function is to approximate the function of its first-order derivative by piece-wise linear functions first, then to obtain the estimate of the original function by integrating the approximated function of the first-order derivative by a digital circuit. The average error and maximum error of the proposed approximation approach are in the order of 10^-3 and 10^-2, respectively in the software simulation. The hardware implementation of the proposed method consumes only one multiplication and one addition/subtraction ALU with the aid of resource sharing. The performance of our circuit has been validated by a neural network for a system identification problem in the software simulation.