no code implementations • 5 May 2024 • Honghua Chen, Chen Change Loy, Xingang Pan
Despite the emergence of successful NeRF inpainting methods built upon explicit RGB and depth 2D inpainting supervisions, these methods are inherently constrained by the capabilities of their underlying 2D inpainters.
no code implementations • 19 Mar 2024 • Yongwei Chen, Tengfei Wang, Tong Wu, Xingang Pan, Kui Jia, Ziwei Liu
Though promising results have been achieved in single object generation, these methods often struggle to model complex 3D assets that inherently contain multiple objects.
no code implementations • 18 Mar 2024 • Yushi Lan, Fangzhou Hong, Shuai Yang, Shangchen Zhou, Xuyi Meng, Bo Dai, Xingang Pan, Chen Change Loy
The latent is decoded by a transformer-based decoder into a high-capacity 3D neural field.
no code implementations • 12 Dec 2023 • Kaiwen Zhang, Yifan Zhou, Xudong Xu, Xingang Pan, Bo Dai
Our key idea is to capture the semantics of the two images by fitting two LoRAs to them respectively, and interpolate between both the LoRA parameters and the latent noises to ensure a smooth semantic transition, where correspondence automatically emerges without the need for annotation.
no code implementations • 18 Aug 2023 • Xudong Xu, Zhaoyang Lyu, Xingang Pan, Bo Dai
In this work, we propose Material-Aware Text-to-3D via LAtent BRDF auto-EncodeR (\textbf{MATLABER}) that leverages a novel latent BRDF auto-encoder for material generation.
no code implementations • 1 Jun 2023 • Mohit Mendiratta, Xingang Pan, Mohamed Elgharib, Kartik Teotia, Mallikarjun B R, Ayush Tewari, Vladislav Golyanik, Adam Kortylewski, Christian Theobalt
Our method edits the full head in a canonical space, and then propagates these edits to remaining time steps via a pretrained deformation network.
5 code implementations • 18 May 2023 • Xingang Pan, Ayush Tewari, Thomas Leimkühler, Lingjie Liu, Abhimitra Meka, Christian Theobalt
Synthesizing visual content that meets users' needs often requires flexible and precise controllability of the pose, shape, expression, and layout of the generated objects.
no code implementations • 31 Mar 2023 • Mallikarjun B R, Xingang Pan, Mohamed Elgharib, Christian Theobalt
Advances in 3D-aware generative models have pushed the boundary of image synthesis with explicit camera control.
no code implementations • 25 Mar 2023 • Kartik Teotia, Mallikarjun B R, Xingang Pan, Hyeongwoo Kim, Pablo Garrido, Mohamed Elgharib, Christian Theobalt
This paper presents a novel approach to building highly photorealistic digital head avatars.
no code implementations • ICCV 2023 • Yuanbo Xiangli, Linning Xu, Xingang Pan, Nanxuan Zhao, Bo Dai, Dahua Lin
Traditional modeling pipelines keep an asset library storing unique object templates, which is both versatile and memory efficient in practice.
no code implementations • CVPR 2023 • Linning Xu, Yuanbo Xiangli, Sida Peng, Xingang Pan, Nanxuan Zhao, Christian Theobalt, Bo Dai, Dahua Lin
An alternative solution is to use a feature grid representation, which is computationally efficient and can naturally scale to a large scene with increased grid resolutions.
no code implementations • ICCV 2023 • Chao Wang, Ana Serrano, Xingang Pan, Bin Chen, Hans-Peter Seidel, Christian Theobalt, Karol Myszkowski, Thomas Leimkuehler
Most in-the-wild images are stored in Low Dynamic Range (LDR) form, serving as a partial observation of the High Dynamic Range (HDR) visual world.
no code implementations • 31 Oct 2022 • Mallikarjun BR, Ayush Tewari, Xingang Pan, Mohamed Elgharib, Christian Theobalt
We start with a global generative model (GAN) and learn to decompose it into different semantic parts using supervision from 2D segmentation masks.
1 code implementation • 26 Aug 2022 • Tong Wu, Jiaqi Wang, Xingang Pan, Xudong Xu, Christian Theobalt, Ziwei Liu, Dahua Lin
Previous methods based on neural volume rendering mostly train a fully implicit model with MLPs, which typically require hours of training for a single scene.
no code implementations • 18 Jun 2022 • Xingang Pan, Ayush Tewari, Lingjie Liu, Christian Theobalt
2D images are observations of the 3D physical world depicted with the geometry, material, and illumination components.
no code implementations • CVPR 2022 • Ayush Tewari, Mallikarjun B R, Xingang Pan, Ohad Fried, Maneesh Agrawala, Christian Theobalt
Our model can disentangle the geometry and appearance variations in the scene, i. e., we can independently sample from the geometry and appearance spaces of the generative model.
no code implementations • 10 Dec 2021 • Yuanbo Xiangli, Linning Xu, Xingang Pan, Nanxuan Zhao, Anyi Rao, Christian Theobalt, Bo Dai, Dahua Lin
The wide span of viewing positions within these scenes yields multi-scale renderings with very different levels of detail, which poses great challenges to neural radiance field and biases it towards compromised results.
1 code implementation • NeurIPS 2021 • Xudong Xu, Xingang Pan, Dahua Lin, Bo Dai
In this paper, we propose Generative Occupancy Fields (GOF), a novel model based on generative radiance fields that can learn compact object surfaces without impeding its training convergence.
1 code implementation • NeurIPS 2021 • Xingang Pan, Xudong Xu, Chen Change Loy, Christian Theobalt, Bo Dai
Motivated by the observation that a 3D object should look realistic from multiple viewpoints, these methods introduce a multi-view constraint as regularization to learn valid 3D radiance fields from 2D images.
1 code implementation • ICCV 2021 • Yuming Jiang, Ziqi Huang, Xingang Pan, Chen Change Loy, Ziwei Liu
In this work, we propose Talk-to-Edit, an interactive facial editing framework that performs fine-grained attribute manipulation through dialog between the user and the system.
Ranked #1 on Fine-Grained Facial Editing on CelebA-Dialog
1 code implementation • ICLR 2021 • Xingang Pan, Bo Dai, Ziwei Liu, Chen Change Loy, Ping Luo
Through our investigation, we found that such a pre-trained GAN indeed contains rich 3D knowledge and thus can be used to recover 3D shape from a single 2D image in an unsupervised manner.
2 code implementations • CVPR 2020 • Xiaohang Zhan, Xingang Pan, Bo Dai, Ziwei Liu, Dahua Lin, Chen Change Loy
This is achieved via Partial Completion Network (PCNet)-mask (M) and -content (C), that learn to recover fractions of object masks and contents, respectively, in a self-supervised manner.
1 code implementation • ECCV 2020 • Xingang Pan, Xiaohang Zhan, Bo Dai, Dahua Lin, Chen Change Loy, Ping Luo
Learning a good image prior is a long-term goal for image restoration and manipulation.
1 code implementation • ICML 2020 • Wenqi Shao, Shitao Tang, Xingang Pan, Ping Tan, Xiaogang Wang, Ping Luo
Unlike prior arts that simply removed the inhibited channels, we propose to "wake them up" during training by designing a novel neural building block, termed Channel Equilibrium (CE) block, which enables channels at the same layer to contribute equally to the learned representation.
no code implementations • 25 Sep 2019 • Wenqi Shao, Shitao Tang, Xingang Pan, Ping Tan, Xiaogang Wang, Ping Luo
However, over-sparse CNNs have many collapsed channels (i. e. many channels with undesired zero values), impeding their learning ability.
no code implementations • CVPR 2020 • Ziwei Liu, Zhongqi Miao, Xingang Pan, Xiaohang Zhan, Dahua Lin, Stella X. Yu, Boqing Gong
A typical domain adaptation approach is to adapt models trained on the annotated data in a source domain (e. g., sunny weather) for achieving high performance on the test data in a target domain (e. g., rainy weather).
1 code implementation • ICCV 2019 • Xingang Pan, Xiaohang Zhan, Jianping Shi, Xiaoou Tang, Ping Luo
Unlike existing works that design normalization techniques for specific tasks, we propose Switchable Whitening (SW), which provides a general form unifying different whitening methods as well as standardization methods.
Ranked #6 on Robust Object Detection on DWD
1 code implementation • CVPR 2019 • Xiaohang Zhan, Xingang Pan, Ziwei Liu, Dahua Lin, Chen Change Loy
Instead of explicitly modeling the motion probabilities, we design the pretext task as a conditional motion propagation problem.
24 code implementations • ECCV 2018 • Xingang Pan, Ping Luo, Jianping Shi, Xiaoou Tang
IBN-Net carefully integrates Instance Normalization (IN) and Batch Normalization (BN) as building blocks, and can be wrapped into many advanced deep networks to improve their performances.
Ranked #3 on All-day Semantic Segmentation on All-day CityScapes
8 code implementations • 17 Dec 2017 • Xingang Pan, Jianping Shi, Ping Luo, Xiaogang Wang, Xiaoou Tang
Although CNN has shown strong capability to extract semantics from raw pixels, its capacity to capture spatial relationships of pixels across rows and columns of an image is not fully explored.
Ranked #50 on Lane Detection on CULane