no code implementations • 26 Mar 2024 • Sherwin Bahmani, Xian Liu, Yifan Wang, Ivan Skorokhodov, Victor Rong, Ziwei Liu, Xihui Liu, Jeong Joon Park, Sergey Tulyakov, Gordon Wetzstein, Andrea Tagliasacchi, David B. Lindell
We learn local deformations that conform to the global trajectory using supervision from a text-to-video model.
no code implementations • 22 Feb 2024 • Willi Menapace, Aliaksandr Siarohin, Ivan Skorokhodov, Ekaterina Deyneka, Tsai-Shien Chen, Anil Kag, Yuwei Fang, Aleksei Stoliar, Elisa Ricci, Jian Ren, Sergey Tulyakov
Since video content is highly redundant, we argue that naively bringing advances of image models to the video generation domain reduces motion fidelity, visual quality and impairs scalability.
Ranked #2 on Text-to-Video Generation on UCF-101
no code implementations • 1 Feb 2024 • Guocheng Qian, Junli Cao, Aliaksandr Siarohin, Yash Kant, Chaoyang Wang, Michael Vasilkovsky, Hsin-Ying Lee, Yuwei Fang, Ivan Skorokhodov, Peiye Zhuang, Igor Gilitschenski, Jian Ren, Bernard Ghanem, Kfir Aberman, Sergey Tulyakov
We introduce Amortized Text-to-Mesh (AToM), a feed-forward text-to-mesh framework optimized across multiple text prompts simultaneously.
1 code implementation • 29 Nov 2023 • Sherwin Bahmani, Ivan Skorokhodov, Victor Rong, Gordon Wetzstein, Leonidas Guibas, Peter Wonka, Sergey Tulyakov, Jeong Joon Park, Andrea Tagliasacchi, David B. Lindell
Recent breakthroughs in text-to-4D generation rely on pre-trained text-to-image and text-to-video models to generate dynamic 3D scenes.
no code implementations • 12 Oct 2023 • Xian Liu, Jian Ren, Aliaksandr Siarohin, Ivan Skorokhodov, Yanyu Li, Dahua Lin, Xihui Liu, Ziwei Liu, Sergey Tulyakov
Our model enforces the joint learning of image appearance, spatial relationship, and geometry in a unified network, where each branch in the model complements to each other with both structural awareness and textural richness.
1 code implementation • ICCV 2023 • Wenxuan Zhang, Paul Janson, Kai Yi, Ivan Skorokhodov, Mohamed Elhoseiny
The GRW loss augments the training by continually encouraging the model to generate realistic and characterized samples to represent the unseen space.
1 code implementation • 30 Jun 2023 • Guocheng Qian, Jinjie Mai, Abdullah Hamdi, Jian Ren, Aliaksandr Siarohin, Bing Li, Hsin-Ying Lee, Ivan Skorokhodov, Peter Wonka, Sergey Tulyakov, Bernard Ghanem
We present Magic123, a two-stage coarse-to-fine approach for high-quality, textured 3D meshes generation from a single unposed image in the wild using both2D and 3D priors.
no code implementations • 29 May 2023 • Yue Fan, Ivan Skorokhodov, Oleg Voynov, Savva Ignatyev, Evgeny Burnaev, Peter Wonka, Yiqun Wang
We develop a method that recovers the surface, materials, and illumination of a scene from its posed multi-view images.
1 code implementation • CVPR 2023 • Yiqun Wang, Ivan Skorokhodov, Peter Wonka
The first component is to borrow the tri-plane representation from EG3D and represent signed distance fields as a mixture of tri-planes and MLPs instead of representing it with MLPs only.
no code implementations • ICCV 2023 • Ahmed Abdelreheem, Ivan Skorokhodov, Maks Ovsjanikov, Peter Wonka
We explore the task of zero-shot semantic segmentation of 3D shapes by using large-scale off-the-shelf 2D image recognition models.
no code implementations • 2 Mar 2023 • Ivan Skorokhodov, Aliaksandr Siarohin, Yinghao Xu, Jian Ren, Hsin-Ying Lee, Peter Wonka, Sergey Tulyakov
Existing 3D-from-2D generators are typically designed for well-curated single-category datasets, where all the objects have (approximately) the same scale, 3D location, and orientation, and the camera always points to the center of the scene.
no code implementations • CVPR 2023 • Aliaksandr Siarohin, Willi Menapace, Ivan Skorokhodov, Kyle Olszewski, Jian Ren, Hsin-Ying Lee, Menglei Chai, Sergey Tulyakov
We propose a novel approach for unsupervised 3D animation of non-rigid deformable objects.
no code implementations • CVPR 2023 • Yinghao Xu, Menglei Chai, Zifan Shi, Sida Peng, Ivan Skorokhodov, Aliaksandr Siarohin, Ceyuan Yang, Yujun Shen, Hsin-Ying Lee, Bolei Zhou, Sergey Tulyakov
Existing 3D-aware image synthesis approaches mainly focus on generating a single canonical object and show limited capacity in composing a complex scene containing a variety of objects.
1 code implementation • 21 Jun 2022 • Ivan Skorokhodov, Sergey Tulyakov, Yiqun Wang, Peter Wonka
In this work, we show that it is possible to obtain a high-resolution 3D generator with SotA image quality by following a completely different route of simply training the model patch-wise.
1 code implementation • 15 Jun 2022 • Yiqun Wang, Ivan Skorokhodov, Peter Wonka
We develop HF-NeuS, a novel method to improve the quality of surface reconstruction in neural rendering.
1 code implementation • CVPR 2022 • Ivan Skorokhodov, Sergey Tulyakov, Mohamed Elhoseiny
We build our model on top of StyleGAN2 and it is just ${\approx}5\%$ more expensive to train at the same resolution while achieving almost the same image quality.
no code implementations • 29 Sep 2021 • Kilichbek Haydarov, Aashiq Muhamed, Jovana Lazarevic, Ivan Skorokhodov, Mohamed Elhoseiny
To the best of our knowledge, our work is the first one which explores text-controllable continuous image generation.
1 code implementation • 20 Apr 2021 • Divyansh Jha, Kai Yi, Ivan Skorokhodov, Mohamed Elhoseiny
By generating representations of unseen classes based on their semantic descriptions, e. g., attributes or text, generative ZSL attempts to differentiate unseen from seen categories.
1 code implementation • ICCV 2021 • Ivan Skorokhodov, Grigorii Sotnikov, Mohamed Elhoseiny
In this work, we develop a method to generate infinite high-resolution images with diverse and complex content.
Ranked #1 on Infinite Image Generation on LHQ
no code implementations • ICLR 2021 • Ivan Skorokhodov, Mohamed Elhoseiny
Normalization techniques have proved to be a crucial ingredient of successful training in a traditional supervised learning regime.
1 code implementation • 24 Dec 2020 • Ivan Skorokhodov
In this work, we propose an approach to perform non-uniform image interpolation based on a Gaussian Mixture Model.
1 code implementation • CVPR 2021 • Ivan Skorokhodov, Savva Ignatyev, Mohamed Elhoseiny
In most existing learning systems, images are typically viewed as 2D pixel arrays.
Ranked #12 on Image Generation on LSUN Churches 256 x 256
3 code implementations • 19 Jun 2020 • Ivan Skorokhodov, Mohamed Elhoseiny
Normalization techniques have proved to be a crucial ingredient of successful training in a traditional supervised learning regime.
1 code implementation • 9 Oct 2019 • Ivan Skorokhodov, Mikhail Burtsev
We present multi-point optimization: an optimization technique that allows to train several models simultaneously without the need to keep the parameters of each one individually.