The application of image super-resolution technologies in recent years has increased noticeably. The main purpose of image up-scaling is to obtain high-resolution images from low-resolution images, and these up-scaled images should keep satisfactory visual qualities and present natural textures. The most popular image up-scaling algorithms are based on interpolation methods in spatial domain. However, the up-scaled images may produce blurring artifacts. Therefore, using spatial sharpening filters is usually used to make blurred images sharp and clear. The quantity of image sharpening is the key to decide the visual qualities of up-scaled images. In this paper, a method based on self-similarity of images and using simple linear regression to build a reconstruction model for improving visual qualities of up-scaled images adaptively is proposed. The experimental results show that our algorithm provides better subjective visual qualities as well as the peak signal-to-noise ratio (PSNR).