Fine-Scale Face Fitting and Texture Fusion With Inverse Renderer

You are here

Top Reasons to Join SPS Today!

1. IEEE Signal Processing Magazine
2. Signal Processing Digital Library*
3. Inside Signal Processing Newsletter
4. SPS Resource Center
5. Career advancement & recognition
6. Discounts on conferences and publications
7. Professional networking
8. Communities for students, young professionals, and women
9. Volunteer opportunities
10. Coming soon! PDH/CEU credits
Click here to learn more.

Fine-Scale Face Fitting and Texture Fusion With Inverse Renderer

Yang Liu; Yangyu Fan; Zhe Guo; Anam Zaman; Shiya Liu

3D face reconstruction from a single image still suffers from low accuracy and inability to recover textures in invisible regions. In this paper, we propose a method for generating a 3D portrait with complete texture. The coarse face-and-head model and texture parameters are obtained using 3D Morphable Model fitting. We design an image-geometric inverse renderer that acquires normal, albedo, and light to jointly reconstruct the facial details. Then, we use a texture fusion network to extract the valid texture from rendered faces employing different viewpoints. Specifically, this fused texture recovers the invisible region of the input face, which illustrates the realistic surface of our 3D geometric model. Our approach faithfully reconstructs the original face details, including the profiles and the head region. Extensive experiments are performed to demonstrate that our method outperforms state-of-the-art techniques in various challenging scenarios.


3DAVATAR creation has great theoretical significance and practical value in the fields of computer graphics and virtual reality. We have witnessed great success of high-quality face model generation in the last few years, and this technology plays an important role in entertainment and multimedia communications. However, the 3D avatar creation still suffers from expensive artificial cost and complex equipment for capturing. We aim to generate a complete photorealistic human portrait model from a single “in-the-wild” face image acquired conveniently.

SPS on Twitter

  • DEADLINE EXTENDED: The 2023 IEEE International Workshop on Machine Learning for Signal Processing is now accepting…
  • ONE MONTH OUT! We are celebrating the inaugural SPS Day on 2 June, honoring the date the Society was established in…
  • The new SPS Scholarship Program welcomes applications from students interested in pursuing signal processing educat…
  • CALL FOR PAPERS: The IEEE Journal of Selected Topics in Signal Processing is now seeking submissions for a Special…
  • Test your knowledge of signal processing history with our April trivia! Our 75th anniversary celebration continues:…

IEEE SPS Educational Resources

IEEE SPS Resource Center

IEEE SPS YouTube Channel