While mobile sensors are ubiquitous nowadays, the geographical activities of human beings are feasible to be collected and the geospatial interactions between people can be derived. As we know there is an underlying social network between mobile users, such social relationships are hidden and hold by service providers. Acquiring the social network over mobile users would enable lots of applications, such as friend recommendation and energy-saving mobile DB management. In this paper, we propose to infer the social relationships using the sensor data, which contains the encounter records between individuals, without any knowledge about the real friendships in prior. We propose a two-phase prediction method for the social inference. Experiments conducted on the CRAWDAD data demonstrate the encouraging results with satisfying prediction scores of precision and recall.