This paper presents a grey neural network-based forecasting system (GNNFS) in solving the prediction problem. GNNFS adopts a grey model to predict the signal and a neural network (NN) to forecast the prediction error of the grey model. A sequential batch learning (SBL) is developed to adjust the weights of the NN. The proposed GNNFS is applied to a binocular robot, called an Eye-Robot, for human-robot interaction which involved predicting the trajectory of a participant's hand and tracking the hand. By applying the SBL, the GNNFS can gradually learn to predict the trajectory of the hand and track it well. The experimental results show that the GNNFS can carry out the SBL in real-time for vision-guided robot trajectory tracking.