Fine-Scale Face Fitting and Texture Fusion With Inverse Renderer

You are here

Top Reasons to Join SPS Today!

1. IEEE Signal Processing Magazine
2. Signal Processing Digital Library*
3. Inside Signal Processing Newsletter
4. SPS Resource Center
5. Career advancement & recognition
6. Discounts on conferences and publications
7. Professional networking
8. Communities for students, young professionals, and women
9. Volunteer opportunities
10. Coming soon! PDH/CEU credits
Click here to learn more.

Fine-Scale Face Fitting and Texture Fusion With Inverse Renderer

Yang Liu; Yangyu Fan; Zhe Guo; Anam Zaman; Shiya Liu

3D face reconstruction from a single image still suffers from low accuracy and inability to recover textures in invisible regions. In this paper, we propose a method for generating a 3D portrait with complete texture. The coarse face-and-head model and texture parameters are obtained using 3D Morphable Model fitting. We design an image-geometric inverse renderer that acquires normal, albedo, and light to jointly reconstruct the facial details. Then, we use a texture fusion network to extract the valid texture from rendered faces employing different viewpoints. Specifically, this fused texture recovers the invisible region of the input face, which illustrates the realistic surface of our 3D geometric model. Our approach faithfully reconstructs the original face details, including the profiles and the head region. Extensive experiments are performed to demonstrate that our method outperforms state-of-the-art techniques in various challenging scenarios.


3DAVATAR creation has great theoretical significance and practical value in the fields of computer graphics and virtual reality. We have witnessed great success of high-quality face model generation in the last few years, and this technology plays an important role in entertainment and multimedia communications. However, the 3D avatar creation still suffers from expensive artificial cost and complex equipment for capturing. We aim to generate a complete photorealistic human portrait model from a single “in-the-wild” face image acquired conveniently.


IEEE SPS Educational Resources

IEEE SPS Resource Center

IEEE SPS YouTube Channel