Open Access Open Access  Restricted Access Subscription Access

FaceForge: Personalized Facial Enhancement with Class-guided Noise Reduction

Jagadish P, Nethravathi B, Sandhyarani ., Chirag C, Bhagyashre ., Gokhul C S

Abstract


Personalizing digital faces is a fundamental task in various fields such as virtual try-on, digital avatars, and face image editing. However, achieving realistic and editable face personalization remains a challenge due to the inherent noise and variability of face images. This study presents FaceForge, a new approach to editable face personalization using class-driven noise suppression regulation. FaceForge uses a diffusion-based generative model trained with class-specific information to enable high-quality facial manipulation while preserving identity and expression details. The proposed method effectively suppresses noise and artifacts in face images, allowing users to personalize faces with unprecedented fidelity and control. Experimental results demonstrate the superior performance of FaceForge compared to state-of-the-art methods in terms of image quality, editability, and robustness to input variations. In addition, we demonstrate the practical implications of FaceForge in real-world applications such as virtual testing systems and the creation of digital avatars. Overall, FaceForge represents a significant advance in the field of facial personalization, offering users powerful tool to easily create personalized and distinctive digital faces


Full Text:

PDF

References


Alina Kuznetsova, Hassan Rom, Neil Alldrin, Jasper Uijlings, Ivan Krasin, Jordi Pont-Tuset, Shahab Kamali, Stefan Popov, Matteo Malloci, Alexander Kolesnikov, et al. The open images dataset v4: Unified image classification, object detection, and visual relationship detection at scale. IJCV, 2020.

Xi Chen, Xiao Wang, Soravit Changpinyo, AJ Piergiovanni, Piotr Padlewski, Daniel Salz, Sebastian Goodman, Adam Grycner, Basil Mustafa, Lucas Beyer, Alexander Kolesnikov, Joan Puigcerver, Nan Ding, Keran Rong, Hassan Akbari, Gaurav Mishra, Linting Xue, Ashish V Thapliyal, James Bradbury, Weicheng Kuo, Mojtaba Seyedhosseini, Chao Jia, Burcu Karagol Ayan, Carlos Riquelme Ruiz, Andreas Peter Steiner, Anelia Angelova, Xiaohua Zhai, Neil Houlsby, and Radu Soricut. PaLI: A jointly-scaled multilingual languageimage model. In ICLR, 2023.

Zhuowei Chen, Shancheng Fang, Wei Liu, Qian He, Mengqi Huang, Yongdong Zhang, and Zhendong Mao. Dreamidentity: Improved editability for efficient face-identity preserved image generation. arXiv preprint arXiv:2307.00300, 2023.

Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Arcface: Additive Angular Margin Loss for Deep Face Recognition. In CVPR, 2019.

Yu Deng, Jiaolong Yang, Sicheng Xu, Dong Chen, Yunde Jia, and Xin Tong. Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Set. In CVPR Workshop, 2019.

Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. In NeurIPS, 2021.

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. In ICLR, 2020.

Rinon Gal, Or Patashnik, Haggai Maron, Gal Chechik, and Daniel Cohen-Or. Stylegan-nada: Clip-guided domain adaptation of image generators. arXiv preprint arXiv:2108.00946, 2021.

Jing Shi, Wei Xiong, Zhe Lin, and Hyun Joon Jung. Instant booth: Personalized text-to-image generation without test time finetuning. arXiv preprint arXiv:2304.03411, 2023.

Rinon Gal, Moab Arar, Yuval Atzmon, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or. Designing an encoder for fast personalization of text-to-image models. arXiv preprint arXiv:2302.12228, 2023.


Refbacks

  • There are currently no refbacks.