Convolutional neural network has been recently studied and used in many object recognition tasks. In this work, we employ fully convolutional networks (FCNs) to recognize On-Premise Signs (OPS) in real scene. This technology is capable of being utilized in many camera-enabled devices like smart phones to develop practical commercial applications. The fully convolutional network technique is used to train a model to infer whether a street view image contains a specific OPS and where the OPS locates in the input image. Furthermore, to improve the detection performance, data augmentation approaches are applied in our work, and the experiment results show our model outperforms the previous tasks.