From LLMs to Actions: Latent Codes as Bridges in Hierarchical Robot Control

no code implementations • 8 May 2024 • Yide Shentu, Philipp Wu, Aravind Rajeswaran, Pieter Abbeel

This enables LLMs to flexibly communicate goals in the task plan without being entirely constrained by language limitations.

Autoregressive Uncertainty Modeling for 3D Bounding Box Prediction

no code implementations • 13 Oct 2022 • Yuxuan Liu, Nikhil Mishra, Maximilian Sieb, Yide Shentu, Pieter Abbeel, Xi Chen

3D bounding boxes are a widespread intermediate representation in many computer vision applications.

Learning Instance Segmentation by Interaction

1 code implementation • 21 Jun 2018 • Deepak Pathak, Yide Shentu, Dian Chen, Pulkit Agrawal, Trevor Darrell, Sergey Levine, Jitendra Malik

The agent uses its current segmentation model to infer pixels that constitute objects and refines the segmentation model by interacting with these pixels.

Instance Segmentation Segmentation +1

Zero-Shot Visual Imitation

1 code implementation • ICLR 2018 • Deepak Pathak, Parsa Mahmoudieh, Guanghao Luo, Pulkit Agrawal, Dian Chen, Yide Shentu, Evan Shelhamer, Jitendra Malik, Alexei A. Efros, Trevor Darrell

In our framework, the role of the expert is only to communicate the goals (i. e., what to imitate) during inference.

Imitation Learning