There has been an increasing research interest in classification for data streams. Due to the evolving nature of data streams, it is a highly challenging issue to detect the appearance of concept drifts, which will make the current classification model invalid as time passes. So far most stream classification solutions exploit the so-called incremental learning process to continuously track the deviation of prediction accuracy. Unfortunately, to achieve the prompt concept-drifting detection, such strategies usually rely on an infeasible assumption about the availability of data instances with true labels. We in this paper propose a new framework, called Inference of Concept Evolution (abbreviated as ICE), to minimize the need of real-time acquisition of true labels. Specifically, the ICE framework is devised based on the idea of model reuse. The dictionary learning technique is utilized to determine whether the concept drift appears without the need of label acquisition. When the drift happens, the ICE framework will select the best model maintained in the model pool, decreasing the need of model re-training and its costly label acquisition. As demonstrated in our experimental result, the ICE framework can track the best model correctly and efficiently, showing its feasibility in real cases.