Research On Robot Portrait Drawing Technology Based On Image/natural Language Fusion

In this paper,face images are taken as the main research object.We transform the face images into portrait sketch with artist drawing style.Then,the obtained portraits will be processed and calculated to obtain the coordinate sequences for robot drawing.The details are as follows:First,we studied the development of semantic segmentation and chose Bisenet as the basic network of our segmentation method.A series of improvements are made to it in view of its shallow network and rough up-sampling mode.The main improvements are: Adding residual module into Spatial Path to increase the depth;Reconstructing Context Path to make full use of feature maps of different scales;A softer up-sampling method is used to improve the segmentation accuracy at the edge of images.The improved Bisenet network is used to segment different parts of human faces.Second,we studied the existing methods of style transfer.We propose and implement an APDrawing GAN based method to transform the real face images into portrait sketches with artist drawing style.The whole face image and the segmented eyes,nose,mouth and other parts are input into the improved APD to get the portrait style image.In order to make the lines of the generated portraits cleaner and conciser,we add a sketch simplification network at the top of APDrawing GAN as the second stage of the style transfer method in this paper.Through the sketch simplification network,we can delete a large number of trivial and disorderly lines in the portraits generated by APDrawing GAN in the first stage,so as to make the lines near eyes,nose and other parts of the portraits more concise and clean.the portrait sketch obtained by this method is more suitable for robot drawing.Third,we studied the existing methods of generating images from text description,designs and implements a text to image model based on stack GAN,and extends the model in a series of ways.Including: Dividing the generation task into more sub-stages to reduce the learning pressure of per-stage network;Gradually adding new tasks to the existing networks by gradually increasing the network in each stage;Independent condition enhancement method is used to improve the robustness to text changes.However,due to the lack of data and poor data quality,the face images generated based on language description can only roughly show the face contour,and there is much room for further research.Forth,in order to make the robot arm imitate human's habit to draw a portrait sketch,this paper improves the Octagonal Search method,and designs a search method that makes the robot arm draw more "anthropomorphic".In this way,the coordinate sequences obtained by searching the portrait can be given to the robot arm to draw,which can better simulate the habit of human artists in drawing the portrait.Finally,based on the researches above,a portrait rendering system is implemented,which verifies the effectiveness of the above methods.