An Algorithmic Exploration In Chinese Calligraphy And Painting

There is a long trace of efforts by talented artists to try unremittingly to transform the computer into a novel art creation tool, even at times when computers were outrageously expensive, clumsy and had very limited power. But until recently the limited amount of computing power available to any ordinary person had meant more hindrances than opportunities to the innovative artists who fancied using the computer to do art. This thesis study is formulated in the light of recent technical advances that are turning what has been largely a dream into the reality of the computer having a serious role in art. Our study concentrates on two traditional Chinese art forms—calligraphy and painting. This is because despite the fact that the very long Chinese history has nurtured an extremely rich culture and system of Chinese arts of calligraphy and painting, which are increasingly becoming more popular around the globe, very limited serious research efforts have been dedicated to the challenges of Chinese painting and calligraphy in computer science circles. Many of the research problems are quite unique and not present in Western arts. This motivates us to explore a computational approach to Chinese calligraphy and painting.In our research, we try to bridge the beauty and intelligence involved in the two traditional forms of Chinese visual arts with the algorithmic possibilities brought about by advanced modern computing techniques. There are three main components in our study: 1) automatic generation of Chinese calligraphy, 2) a stroke-based intelligent Chinese painting decomposition and animation method, and 3) an interactive approach for electronically creating Chinese calligraphy and painting using a physically-based virtual hairy paint brush model and an expressive digital pigment behavior simulation model. In the following, we give the highlights of each.In the first part of our research, intelligent Chinese calligraphy generation, we address the demanding task of developing intelligent systems with creativity abilities that can perform design tasks automatically. The main challenge is how to model human beings' creativity mathematically and mimic such creativity computationally. At the heart of this research, we adopt the synthesis reasoning model as the underlying reasoning mechanism to simulate human beings' creative thinking when they handle design tasks. We adopt this model because it is a mental simulation device grounded in cognitive sciences, and there is proof by prior studies that the model is suited for reproducing human's innovative thinking in the imagery thinking domain. We carefully tailor the standard form of the general synthesis reasoning model according to our application context and propose a concrete analogous reasoning algorithm based on the general synthesis reasoning model for automatic generation of Chinese calligraphy. We also look into knowledge representation and acquisition issues in our automatic calligraphy generation problem as without the support of a comprehensive and efficient knowledge representation and reference mechanism, no reasoning model will be capable enough to handle real-world problems. We propose a hierarchical and parametric schema to represent Chinese calligraphic writings in the form of static images. Our design facilitates efficient knowledge representation and reference by observing repetitive image patterns and structures exhibited in the calligraphic writings. Based on the synthesis reasoning model and our calligraphy knowledge representation and acquisition schema, once a computer has learned a few calligraphic samples through acquiring the parametric and hierarchical representation of these sample calligraphic writings, it can intelligently and automatically generate Chinese calligraphic writings in novel writing styles. When knowledge is combined, the space in which computer can create visually interesting Chinese calligraphic artworks is immense. Given such a large space of potential solutions, the computer can generate Chinese calligraphic writings with writing styles that we have never seen before. In addition to generating novel artistic Chinese calligraphic writings for calligraphy amateurs, for those advanced calligraphic fans and practicing calligraphists, the computer generated artworks, writing styles, and visual effects can be a useful inspiration. Our system therefore functions as an intelligent computer-aided design system that can stimulate calligraphy art masters to explore new calligraphy art creation possibilities and opportunities which might otherwise be inaccessible due to limitations on the creativity of human brains such as habitual thinking and the lack of art creation motives or inspirations.Our study on automatic generation of Chinese calligraphic artwork spans a broad research context in which our work can be considered a case in point aiming at revealing the possibilities of simulating human being's imagery thinking in creative problem solving. Our purpose is to emulate the human thinking process for the design and development of advanced intelligent computer aided design systems or tools. With these systems or tools, people can conduct challenging design tasks that call for a high degree of creativity and imagination which could be challenging even for the most gifted designers and original thinkers. We present the detailed procedure of designing and developing an intelligent computer aided design system based on the synthesis reasoning model. It demonstrates that the notion of using the computer to assist human designers on their non-trivial design tasks is technically feasible. This also implies the feasibility of developing similar intelligent computer aided design tools in other domain specific applications. In the light of such a vision, we propose a generic methodology for constructing intelligent systems using the synthesis reasoning model based on the design and implementation experiences of our intelligent calligraphy generation system as well as several other systems for solving real-world problems. We describe an extended version of the synthesis reasoning model and its associated software development methodology to serve as a reference for the design and development of synthesis reasoning based intelligent CAD systems for arbitrary domain specific applications.In the second part of our research, we propose a technique to animate a Chinese style painting given its image. What is particularly interesting is that the animation is done at the brush-stroke level. To do that, we first extract descriptions of the brush strokes that hypothetically produced the painting. The key to the extraction process is the use of a brush stroke library which aids region segmentation. The brush stroke library is obtained by digitizing single brush strokes drawn by an experienced artist. The steps in our extraction technique are first to segment the input image, then to find the best set of brush strokes that fit the regions, and finally to refine these strokes to account for the local appearance. We model a single brush stroke using its skeleton and contour, and we characterize texture variation within each stroke by sampling perpendicularly along its skeleton. Our brush stroke model plays a critical role in allowing the painting's appearance to be captured and subsequently rendered with good fidelity. We also introduced a brush stroke overlap separation algorithm which allows full appearance of strokes to be extracted despite the presence of overlaps. Once these brush descriptions have been obtained, the painting can be animated at the brush stroke level. A key contribution of our work is the automatic recovery of separate, vectorized brush strokes. This is a tremendous time saver compared to manual segmentation especially when the painting has hundreds of brush strokes. In addition, proper automatic color separation in the overlap regions is not trivial and is not a feature in common image editing tools such as Photoshop. The animation is significantly easier once the segmentation is done. We present several animations of real paintings using our technique. In our study, we focus on Chinese paintings with relatively sparse strokes. The animation is produced using a graphical application we developed. Experimental results show that our method of decomposition is capable of producing high-quality reconstructions of paintings. The quality of the sample animations also serves to illustrate the effectiveness of our decomposition approach for producing stroke-based animation of Chinese paintings.The third part of our research is on the design and development of a novel interactive software system for Chinese calligraphy and painting. We aim at user friendliness and user expressiveness in the design. Many researchers in computer graphics and human-computer interaction circles for a long time have attempted to design such a system. Providing a digital environment for paper-less artwork creation is not only challenging in terms of algorithm design, but requires also significant system research skills, and much effort in constructing a functioning system satisfying user needs. This pursuit has promising market values and could lead to new software tools that may reshape the current digital art software industry. This thesis describes the following two main pieces of our work: 1) a novel algorithmic framework for interactive digital painting and calligraphy based on a new virtual hairy brush model, including an advanced design to optimize the overall performance, and 2) a novel digital pigment behavior simulation model to serve as a foundation for the creation of expressive digital painting and calligraphy.For our algorithmic framework supporting interactive digital painting and calligraphy, we propose a novel virtual hairy brush model for which solid modeling and parametric design techniques are employed. Our model meets all the criteria for a good digital paintbrush. We use only four attributes to capture the essential features of the brush, and a suitably powerful modeling metaphor for its behavior. The virtual hairy brush model's geometry, dynamic motions, and pigment changes are all dealt with in a single model. A single model simplifies the synchronization between the various system modules, thus giving rise to a more stable system, and lower costs. By a careful tradeoff between the complexity of the model and computation efficiency, elaborate simulation of the virtual hairy brush model's deformation and its recovery for interactive painterly rendering is made possible. We also propose a novel paper-ink model to complement the brush's model, and a machine intelligence module to empower the user to easily create beautiful calligraphy and painting. We have successfully developed a prototype system with which end users can interactively produce high-quality digital paintings and calligraphic artwork. Despite the complexity of the modeling behind the scene, the high-level user interface has a simple and friendly design. The final results created by our virtual hairy brush model can achieve visually very compelling results.We also propose a novel generic pigment model suitable for digital painting in a wide range of genres including traditional Chinese painting and water-based painting. The model embodies a simulation of the pigment-water solution and its interaction with the brush and the paper at the level of pigment particles; such a level of details is needed for achieving highly aesthetic effects by the artist. The simulation covers pigment diffusion in the brush, sorption processes at the paper surface, and aspects of pigment particle deposition on the paper. We follow rules and formulations from quantitative studies of adsorption and diffusion processes in surface chemistry and the textile industry. The result is a pigment model that spans a continuum from the very wet to the very dry brush effects, and hence is capable of simulating many intricate effects in digital painting. To experiment with the proposed model, we have embedded the model in our virtual hairy brush based digital painting and calligraphy prototype system. The combined system exhibits interactive speeds on a modest PC platform with improved system performance in terms of the system's artistic expressiveness, as reflected in the extended range of visual effects users can create using our system.In summary, this thesis studies an interesting problem in computer science—the application of computing to Chinese calligraphy and painting. Our approach represents a multi-disciplinary treatment involving intelligent computer aided design, artificial intelligence, computer graphics and interactive techniques, and human-computer interaction. We address many unique difficulties and challenges of using the computer to produce Chinese painting and calligraphy. This thesis presents both our research results and the lessons and engineering experiences obtained during our research process. As a whole, this thesis offers a fresh view on Chinese calligraphy and painting from a computational perspective.