Color is one of the most important modality to convey information. However, around the world about 200 million people are with color vision deficiency (CVD). Some works have been developed to improve viewing experience for people with CVD, such as simulating colorblind vision, re-coloring images, and using patterns to encode images. In this work, we advocate that combining re-coloring and adding patterns to image might be more helpful to colorblind people and is worth deep research. We propose a framework to combine patterns and re-coloring. We first simulate colorblind vision, determine how to add patterns according to the degree of deformation, and then re-color images overlaid with patterns. In the evaluation, we verify effectiveness of combining adding patterns and re-coloring, and demonstrate content-dependent characteristics through the studies based on different types of images and different types of patterns.