In this paper, we present a real-time image denoising chip. For each pixel of the image under processing, our design uses a mask on it to determine whether it is likely being corrupted by impulse noise or not. After the noise detection, we reconstruct the noisy pixel by considering the possible edges existed in the mask. Particularly, our design removes the noise from corrupted images efficiently and requires no previous training. Extensive simulations demonstrate that the proposed method achieves excellent performance in quantitative evaluation and visual quality. Furthermore, the computational complexity of the proposed method is low and its memory requirement is small. An efficient VLSI architecture for this scheme is developed and it yields a processing rate of about 150 MHz by using TSMC 0.18μm technology.