Research On Image-based Non-photorealistic Rendering

Realistic rendering was always the main melody in the development history of computer graphics. It is required to supply scenes close to reality. The criterion of its success is to what degree computer-generated images approaching to those taken by cameras. But some researchers found realistic images were not always the best ways of expression. On the contrary, people sometimes need computer to generate images which are different from photos. For example, in architecture, medical and multimedia area, we don't need accurately rendering. We want to make major points which people care about prominent, or to better present artistic features. So the technique of non-photorealistic rendering (NPR) emerges as the times require. It is expected to generate non-photorealistic images that appear to been drawn by hand. In contrast to traditional computer graphics, which focused on reality, the purpose of NPR is to express artistic styles, to mimic artistic works or to complement the realistic graphics. For NPR, the criterion of success is how close the images produced by computer are to the works drawn by artists. NPR enriches computer graphics and can be a good supplement to realistic graphics.As a combination of computer and art, NPR use computer to mimic artistic style by various mathematics models and algorithms. It emphasizes people's subjective feelings and enjoyment of beauty, not the fidelity to the real world. It means we must simulate people's thinking and reasoning, especially creative thoughts, which are almost impossible to be implemented by algorithms. So what we can do is to only simulate the result of artistic processing.Now, NPR methods can be categorized into object-based methods and image-based methods. The former takes 3D models as input and the latter takes digital images as input. The object-based NPR system can obtain geometry information such as color, texture and shape. The artistic style images are generated by projecting, morphing, or other processing on 3D models. The disadvantage of object-based NPR is the creation or modification to geometry models may be very difficult or even impossible. As to image-based NPR, although there is no 3D information, the cost of input is decreased. Many image processing techniques, such as image filtering, feature enhancing, edge detecting, or image segmenting, etc. can be used to process photos into ones with special visual effect.It is important in NPR that transforming styles between different images, such as processing a photo into various binary styles. This paper emphasizes on the processing technologies of image-based binary stylized, including image-based sketching and stippling generation and image morphing between two binary sketching images.First, we introduced the importance of NPR and the challenges that NPR would confront. We gave out the motive, the significance, the main contents and the innovations of our study. Then, we listed the basic concepts and principles about image-based NPR were listed. We also explored the major implementation methods and the difference between image-based NPR and object-based NPR. We described the current research status of image-based NPR.Next, we studied the existing image-based sketching style generation methods. Because of its special expressive force, sketching images are widely applied in artistic creation, illustrations of scientific literature, industrial arts, and graphic advertisement etc. However, the traditional methods don't work very well in some cases. So, we proposed a new image-based sketching style generation method according to the different effects of various image filters and the statistic information of difference image's gray values. We produced gray feature images by self-difference or self-quotient where the effects of illumination were almost removed. From the histogram of gray feature images, we found that intensity values' distributions were approximate to normal distribution. Mean value reflects the whole brightness of an image and square deviation reflects the gray value's clustering degree. According to the statistical information, feature pixels can be distinguished from others and sketching image can be obtained by preserving the primary features only. Because traditional self-quotient image (SQI) method can't generate sketching image very well in some cases, we improved the existing SQI method of NPR. Since traditional filter may blur boundary of image while anisotropy filter can preserve edge information, we use these two kinds of filter in different phases of our algorithm. Combining with the function of gray value transformation, we can extract feature lines from self-quotient image excellently.Then, we pointed out the difference and relationship between stippling works and halftoning images. Through analysis of traditional stippling generation methods and observation to artistic stippling works, we found that traditional algorithms always focused on generating uniform dots randomly. In fact the dots in artistic works are not always random, which have close relationship with graph shapes. For example, the dots distribution generated by traditional voronoi stippling method was not very reasonable. Features were not prominent and the process of iteration was time consuming. In view of this, we proposed a stippling method based on mathematical morphology. In order to outstand image's characteristic, feature regions were extracted and processed differently from the way to other regions. Dot density in a region was computed by a function which was designed previously. The tone of image was expressed by applying different densities in different regions. Based on the principle of mathematical morphology, we sampled pixels by erosion stippling regions layer upon layer. Thus, the sampled dots were kept uniform, random and in accordance with region's shape. At the same time, stippling speed is faster than before.In order to further study the technique of stippling style generation. We improved Kim's feature-guided stippling method that used offset lines guiding stipples distribution. Instead of dots relaxation, we sampled stipples straightly on the offset lines, so that the speed was increased greatly. In traditional methods, tone was always expressed by various dots densities, thus the process of iteration or density function was needed. Here we replaced round dots with black regions which have various sizes and shapes. They were generated by a threshold matrix with 256 degrees. Using various dots as substitutions for sampled pixels on offset lines, we can obtain stippling image more quickly. Also, the generated images have better visual effect. Using this method we can produce stippling images closer to hand works more quickly and more reasonably. Finally, we analyzed the existing image morphing techniques. It was found that researchers pay much attention to generating transition sequences with prescribed features correspondence. But the process of feature prescription preformed by user interactively was always very tedious. Distortion may appear if users were not very professional or the prescription was not very accurate. We proposed a skeleton based morphing method between sketching images. It focused on morphing between images with little corresponding features. In this case, morphing would be more flexible. Skeletons of original image and destination image were extracted respectively and were represented with sets of curves. These curves were all one pixel width. They were composed of skeleton pixels with linear relationship. The number of curves in original set should be equal to that in destination set, so some long curves were equal divided. We computed centroid of every curve and their relative position in the set. The correspondence between original set and target set was determined automatically by our algorithm. Skeleton transition was done by interpolation between original curves and their corresponding target curves. In order to generate sketching effect, we used line width of every skeleton pixel in two images as radius. By rendering filled circles with various radiuses instead of skeleton points, we obtain binary mid-sequences with sketching style.Focusing on image-based NPR techniques, we further studied the basic principles and implementation methods. We discussed the techniques of sketching style generation, stippling rendering and sketching image morphing. Some traditional methods were improved, some new models and rendering methods were proposed. Experiments show that these new methods can be used to mimic hand works more conveniently. Of course, there has a certain gap between images generated by computer and hand works created by artists. It need us work harder to narrow this gap.