no code implementations • 22 May 2024 • Rui Xu, Jiepeng Wang, Hao Pan, Yang Liu, Xin Tong, Shiqing Xin, Changhe Tu, Taku Komura, Wenping Wang
We show that the space spanned by the combination of dimensions and attributes is insufficiently sampled by existing training scheme of diffusion generative models, causing degraded test time performance.
no code implementations • 19 May 2024 • Yinghao Huang, Leo Ho, Dafei Qin, Mingyi Shi, Taku Komura
We address the problem of accurate capture and expressive modelling of interactive behaviors happening between two persons in daily scenarios.
no code implementations • 24 Apr 2024 • Rui Xu, Longdu Liu, Ningna Wang, Shuangmin Chen, Shiqing Xin, Xiaohu Guo, Zichun Zhong, Taku Komura, Wenping Wang, Changhe Tu
In mesh simplification, common requirements like accuracy, triangle quality, and feature alignment are often considered as a trade-off.
1 code implementation • 23 Apr 2024 • Rui Chen, Mingyi Shi, Shaoli Huang, Ping Tan, Taku Komura, Xuelin Chen
We present a novel character control framework that effectively utilizes motion diffusion probabilistic models to generate high-quality and diverse character animations, responding in real-time to a variety of dynamic user-supplied control signals.
no code implementations • 23 Jan 2024 • Zimeng Wang, Zhiyang Dou, Rui Xu, Cheng Lin, YuAn Liu, Xiaoxiao Long, Shiqing Xin, Taku Komura, Xiaoming Yuan, Wenping Wang
We introduce Coverage Axis++, a novel and efficient approach to 3D shape skeletonization.
no code implementations • 2 Jan 2024 • Guying Lin, Lei Yang, YuAn Liu, Congyi Zhang, Junhui Hou, Xiaogang Jin, Taku Komura, John Keyser, Wenping Wang
Sampling against this intrinsic frequency following the Nyquist-Sannon sampling theorem allows us to determine an appropriate training sampling rate.
no code implementations • 7 Dec 2023 • Weilin Wan, Yiming Huang, Shutong Wu, Taku Komura, Wenping Wang, Dinesh Jayaraman, Lingjie Liu
In this study, we introduce a learning-based method for generating high-quality human motion sequences from text descriptions (e. g., ``A person walks forward").
no code implementations • 4 Dec 2023 • Wenyang Zhou, Zhiyang Dou, Zeyu Cao, Zhouyingcheng Liao, Jingbo Wang, Wenjia Wang, YuAn Liu, Taku Komura, Wenping Wang, Lingjie Liu
We introduce Efficient Motion Diffusion Model (EMDM) for fast and high-quality human motion generation.
no code implementations • 29 Nov 2023 • Yilin Wen, Hao Pan, Takehiko Ohkawa, Lei Yang, Jia Pan, Yoichi Sato, Taku Komura, Wenping Wang
We present a novel framework that concurrently tackles hand action recognition and 3D future hand motion prediction.
no code implementations • 29 Nov 2023 • Jiepeng Wang, Hao Pan, Yang Liu, Xin Tong, Taku Komura, Wenping Wang
Such a localized rewriting process enables probabilistic modeling of ambiguous structures and robust generalization across object categories.
no code implementations • 28 Nov 2023 • Weilin Wan, Zhiyang Dou, Taku Komura, Wenping Wang, Dinesh Jayaraman, Lingjie Liu
Controllable human motion synthesis is essential for applications in AR/VR, gaming, movies, and embodied AI.
no code implementations • 28 Nov 2023 • Zhengming Yu, Zhiyang Dou, Xiaoxiao Long, Cheng Lin, Zekun Li, YuAn Liu, Norman Müller, Taku Komura, Marc Habermann, Christian Theobalt, Xin Li, Wenping Wang
The experiments demonstrate the superior performance of Surf-D in shape generation across multiple modalities as conditions.
no code implementations • 20 Sep 2023 • Zhiyang Dou, Xuelin Chen, Qingnan Fan, Taku Komura, Wenping Wang
We present C$\cdot$ASE, an efficient and effective framework that learns conditional Adversarial Skill Embeddings for physics-based characters.
2 code implementations • 7 Sep 2023 • YuAn Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, Wenping Wang
In this paper, we present a novel diffusion model called that generates multiview-consistent images from a single-view image.
no code implementations • 7 Sep 2023 • Kunkun Pang, Dafei Qin, Yingruo Fan, Julian Habekost, Takaaki Shiratori, Junichi Yamagishi, Taku Komura
Learning the mapping between speech and 3D full-body gestures is difficult due to the stochastic nature of the problem and the lack of a rich cross-modal dataset that is needed for training.
1 code implementation • 24 Aug 2023 • Paul Starke, Sebastian Starke, Taku Komura, Frank Steinicke
This paper introduces a novel data-driven motion in-betweening system to reach target poses of characters by making use of phases variables learned by a Periodic Autoencoder.
1 code implementation • 27 May 2023 • YuAn Liu, Peng Wang, Cheng Lin, Xiaoxiao Long, Jiepeng Wang, Lingjie Liu, Taku Komura, Wenping Wang
We present a neural rendering-based method called NeRO for reconstructing the geometry and the BRDF of reflective objects from multiview images captured in an unknown environment.
1 code implementation • 15 May 2023 • Dafei Qin, Jun Saito, Noam Aigerman, Thibault Groueix, Taku Komura
We propose an end-to-end deep-learning approach for automatic rigging and retargeting of 3D models of human faces in the wild.
1 code implementation • 28 Mar 2023 • Peng Wang, YuAn Liu, Zhaoxi Chen, Lingjie Liu, Ziwei Liu, Taku Komura, Christian Theobalt, Wenping Wang
Based on our analysis, we further propose a novel space-warping method called perspective warping, which allows us to handle arbitrary trajectories in the grid-based NeRF framework.
1 code implementation • ICCV 2023 • Wenjia Wang, Yongtao Ge, Haiyi Mei, Zhongang Cai, Qingping Sun, Yanjun Wang, Chunhua Shen, Lei Yang, Taku Komura
As it is hard to calibrate single-view RGB images in the wild, existing 3D human mesh reconstruction (3DHMR) methods either use a constant large focal length or estimate one based on the background environment context, which can not tackle the problem of the torso, limb, hand or face distortion caused by perspective camera projection when the camera is close to the human body.
Ranked #6 on 3D Human Pose Estimation on 3DPW
no code implementations • 11 Mar 2023 • Jiawei Huang, Akito Iizuka, Hajime Tanaka, Taku Komura, Yoshifumi Kitamura
The variance reduction speed of physically-based rendering is heavily affected by the adopted importance sampling technique.
no code implementations • ICCV 2023 • Mingyi Shi, Sebastian Starke, Yuting Ye, Taku Komura, Jungdam Won
We present a novel motion prior, called PhaseMP, modeling a probability distribution on pose transitions conditioned by a frequency domain feature extracted from a periodic autoencoder.
no code implementations • CVPR 2023 • Peng Wang, YuAn Liu, Zhaoxi Chen, Lingjie Liu, Ziwei Liu, Taku Komura, Christian Theobalt, Wenping Wang
Existing fast grid-based NeRF training frameworks, like Instant-NGP, Plenoxels, DVGO, or TensoRF, are mainly designed for bounded scenes and rely on space warping to handle unbounded scenes.
no code implementations • CVPR 2023 • Xiaoxiao Long, Cheng Lin, Lingjie Liu, YuAn Liu, Peng Wang, Christian Theobalt, Taku Komura, Wenping Wang
In this paper, we propose to represent surfaces as the Unsigned Distance Function (UDF) and develop a new volume rendering scheme to learn the neural UDF representation.
no code implementations • ICCV 2023 • Zhiyang Dou, Qingxuan Wu, Cheng Lin, Zeyu Cao, Qiangqiang Wu, Weilin Wan, Taku Komura, Wenping Wang
We further demonstrate the generalizability of our method on hand mesh recovery.
1 code implementation • CVPR 2023 • Yilin Wen, Hao Pan, Lei Yang, Jia Pan, Taku Komura, Wenping Wang
Understanding dynamic hand motions and actions from egocentric RGB videos is a fundamental yet challenging task due to self-occlusion and ambiguity.
no code implementations • 10 Jul 2022 • Peng Wang, YuAn Liu, Guying Lin, Jiatao Gu, Lingjie Liu, Taku Komura, Wenping Wang
ProLiF encodes a 4D light field, which allows rendering a large batch of rays in one training step for image- or patch-level losses.
no code implementations • 27 Jun 2022 • Jiepeng Wang, Peng Wang, Xiaoxiao Long, Christian Theobalt, Taku Komura, Lingjie Liu, Wenping Wang
The key idea of NeuRIS is to integrate estimated normal of indoor scenes as a prior in a neural rendering framework for reconstructing large texture-less shapes and, importantly, to do this in an adaptive manner to also enable the reconstruction of irregular shapes with fine details.
no code implementations • 25 Jun 2022 • Weilin Wan, Lei Yang, Lingjie Liu, Zhuoying Zhang, Ruixing Jia, Yi-King Choi, Jia Pan, Christian Theobalt, Taku Komura, Wenping Wang
We also observe that an object's intrinsic physical properties are useful for the object motion prediction, and thus design a set of object dynamic descriptors to encode such intrinsic properties.
1 code implementation • 12 Jun 2022 • Xiaoxiao Long, Cheng Lin, Peng Wang, Taku Komura, Wenping Wang
We introduce SparseNeuS, a novel neural rendering based method for the task of surface reconstruction from multi-view images.
1 code implementation • 22 Apr 2022 • YuAn Liu, Yilin Wen, Sida Peng, Cheng Lin, Xiaoxiao Long, Taku Komura, Wenping Wang
In this paper, we present a generalizable model-free 6-DoF object pose estimator called Gen6D.
1 code implementation • 12 Jan 2022 • Ian Mason, Sebastian Starke, Taku Komura
In this work we present a style modelling system that uses an animation synthesis network to model motion content based on local motion phases.
1 code implementation • CVPR 2022 • Yingruo Fan, Zhaojiang Lin, Jun Saito, Wenping Wang, Taku Komura
Speech-driven 3D facial animation is challenging due to the complex geometry of human faces and the limited availability of 3D audio-visual data.
Ranked #1 on 3D Face Animation on VOCASET
no code implementations • 4 Dec 2021 • Yingruo Fan, Zhaojiang Lin, Jun Saito, Wenping Wang, Taku Komura
The existing datasets are collected to cover as many different phonemes as possible instead of sentences, thus limiting the capability of the audio-based model to learn more diverse contexts.
1 code implementation • 27 Jul 2021 • Yilin Wen, Xiangyu Li, Hao Pan, Lei Yang, Zheng Wang, Taku Komura, Wenping Wang
Scalable 6D pose estimation for rigid objects from RGB images aims at handling multiple objects and generalizing to novel objects.
6 code implementations • NeurIPS 2021 • Peng Wang, Lingjie Liu, YuAn Liu, Christian Theobalt, Taku Komura, Wenping Wang
In NeuS, we propose to represent a surface as the zero-level set of a signed distance function (SDF) and develop a new volume rendering method to train a neural SDF representation.
no code implementations • 22 Jun 2020 • Mingyi Shi, Kfir Aberman, Andreas Aristidou, Taku Komura, Dani Lischinski, Daniel Cohen-Or, Baoquan Chen
We introduce MotioNet, a deep neural network that directly reconstructs the motion of a 3D human skeleton from monocular video. While previous methods rely on either rigging or inverse kinematics (IK) to associate a consistent skeleton with temporally coherent joint rotations, our method is the first data-driven approach that directly outputs a kinematic skeleton, which is a complete, commonly used, motion representation.
1 code implementation • 10/06 2020 • Sebastian Dorothee Starke, Yiwei Zhao, Taku Komura, Kazi A. Zaman
Training a bipedal character to play basketball and interact with objects, or a quadruped character to move in various locomotion modes, are difficult tasks due to the fast and complex contacts happening during the motion.
no code implementations • 7 Feb 2020 • Chuanyu Yang, Kai Yuan, Wolfgang Merkt, Taku Komura, Sethu Vijayakumar, Zhibin Li
This paper presents a hierarchical framework for Deep Reinforcement Learning that acquires motor skills for a variety of push recovery and balancing behaviors, i. e., ankle, hip, foot tilting, and stepping strategies.
no code implementations • 13 Dec 2019 • Pengpeng Hu, Edmond SL Ho, Nauman Aslam, Taku Komura, Hubert PH Shum
However, the virtual clothing fit evaluation is still under-researched.
no code implementations • 16 Apr 2018 • Pengpeng Hu, Duan Li, Ge Wu, Taku Komura, Dongliang Zhang, Yueqi Zhong
A personalized mannequin is essential for apparel customization using CAD technologies.
no code implementations • 6 Nov 2017 • Pengpeng Hu, Taku Komura, Duan Li, Ge Wu, Yueqi Zhong
Purpose The purpose of this paper is to present a novel framework of reconstructing the 3D textile model with synthesized texture.
no code implementations • 9 May 2017 • Pengpeng Hu, Taku Komura, Daniel Holden, Yueqi Zhong
In this paper, we propose a novel scanning-based solution for modeling and animating characters wearing multiple layers of clothes.