no code implementations • 28 May 2024 • Botao He, Ze Wang, Yuan Zhou, Jingxi Chen, Chahat Deep Singh, Haojia Li, Yuman Gao, Shaojie Shen, Kaiwei Wang, Yanjun Cao, Chao Xu, Yiannis Aloimonos, Fei Gao, Cornelia Fermuller
These event cameras' output is dependent on both motion and texture.
1 code implementation • 2 Apr 2024 • Dehao Yuan, Cornelia Fermüller, Tahseen Rabbani, Furong Huang, Yiannis Aloimonos
We propose VecKM, a local point cloud geometry encoder that is descriptive and efficient to compute.
no code implementations • 20 Mar 2024 • Jingxi Chen, Brandon Y. Feng, Haoming Cai, Mingyang Xie, Christopher Metzler, Cornelia Fermuller, Yiannis Aloimonos
Through extensive experimentation, we demonstrate the capability of our approach to synthesize high-quality videos that effectively ``rewind'' time, showcasing the potential of combining event camera technology with generative models.
1 code implementation • 14 Mar 2024 • Jiayi Wu, Xiaomin Lin, Shahriar Negahdaripour, Cornelia Fermüller, Yiannis Aloimonos
By creating realistic synthetic images that mimic the complexities of the water surface, we provide fine-grained training data for our network (MARVIS) to discern between real and virtual images effectively.
no code implementations • 4 Mar 2024 • Snehesh Shrestha, Yantian Zha, Saketh Banagiri, Ge Gao, Yiannis Aloimonos, Cornelia Fermuller
NatSGD serves as a foundational resource at the intersection of machine learning and HRI research, and we demonstrate its effectiveness in training robots to understand tasks through multimodal human commands, emphasizing the significance of jointly considering speech and gestures.
no code implementations • 29 Nov 2023 • Eadom Dessalene, Michael Maynord, Cornelia Fermüller, Yiannis Aloimonos
We apply LEAP over a majority (87\%) of the training set of the EPIC Kitchens dataset, and release the resulting action programs as a publicly available dataset here (https://drive. google. com/drive/folders/1Cpkw_TI1IIxXdzor0pOXG3rWJWuKU5Ex? usp=drive_link).
1 code implementation • 31 Oct 2023 • Dehao Yuan, Furong Huang, Cornelia Fermüller, Yiannis Aloimonos
In addition, the encoding is decodable, which enables neural networks to regress continuous objects by regressing their encodings.
no code implementations • 12 Oct 2023 • Amir-Hossein Shahidzadeh, Seong Jong Yoo, Pavan Mantripragada, Chahat Deep Singh, Cornelia Fermüller, Yiannis Aloimonos
Tactile exploration plays a crucial role in understanding object structures for fundamental robotics tasks such as grasping and manipulation.
no code implementations • 16 Aug 2023 • Yianni Karabatis, Xiaomin Lin, Nitin J. Sanket, Michail G. Lagoudakis, Yiannis Aloimonos
When access to real, human-labeled data is limited, a combination of mostly synthetic data and a small amount of real data can enhance olive detection.
1 code implementation • 15 Aug 2023 • Akshaj Gaur, Cheng Liu, Xiaomin Lin, Nare Karapetyan, Yiannis Aloimonos
With a number of marine populations in rapid decline, collecting and analyzing data about marine populations has become increasingly important to develop effective conservation policies for a wide range of marine animals, including whales.
no code implementations • CVPR 2023 • Eadom Dessalene, Michael Maynord, Cornelia Fermuller, Yiannis Aloimonos
In this paper we introduce a rule-based, compositional, and hierarchical modeling of action using Therbligs as our atoms.
no code implementations • 3 Oct 2022 • Chahat Deep Singh, Riya Kumari, Cornelia Fermüller, Nitin J. Sanket, Yiannis Aloimonos
In the era of deep learning, data is the critical determining factor in the performance of neural network models.
no code implementations • 16 Sep 2022 • Xiaomin Lin, Nitin J. Sanket, Nare Karapetyan, Yiannis Aloimonos
However, systems for accurate oyster detection require large datasets obtaining which is an expensive and labor-intensive task in underwater environments.
no code implementations • 31 May 2022 • Peter Sutor, Dehao Yuan, Douglas Summers-Stay, Cornelia Fermuller, Yiannis Aloimonos
This process can be performed iteratively and even on single neural networks by instead making a consensus of multiple classification hypervectors.
no code implementations • 6 May 2022 • Levi Burner, Anton Mitrokhin, Cornelia Fermüller, Yiannis Aloimonos
Depth and segmentation are provided at 60 Hz for the event cameras and 30 Hz for the classical camera.
no code implementations • 24 Mar 2022 • Snehesh Shrestha, Cornelia Fermüller, Tianyu Huang, Pyone Thant Win, Adam Zukerman, Chethan M. Parameshwara, Yiannis Aloimonos
Pose Estimation techniques rely on visual cues available through observations represented in the form of pixels.
no code implementations • CVPR 2022 • Chethan M. Parameshwara, Gokul Hari, Cornelia Fermüller, Nitin J. Sanket, Yiannis Aloimonos
In this paper, we introduce a network NFlowNet, for normal flow estimation which is used to enforce robust and direct constraints.
no code implementations • 14 Mar 2022 • Levi Burner, Nitin J. Sanket, Cornelia Fermüller, Yiannis Aloimonos
Distance estimation from vision is fundamental for a myriad of robotic applications such as navigation, manipulation, and planning.
no code implementations • 22 Sep 2021 • Chahat Deep Singh, Nitin J. Sanket, Chethan M. Parameshwara, Cornelia Fermüller, Yiannis Aloimonos
In this paper, we present the first framework to segment unknown objects in a cluttered scene by repeatedly 'nudging' at the objects and moving them to obtain additional motion cues at every step using only a monochrome monocular camera.
no code implementations • 29 Jun 2021 • Nitin J. Sanket, Chahat Deep Singh, Chethan M. Parameshwara, Cornelia Fermüller, Guido C. H. E. de Croon, Yiannis Aloimonos
Our network can detect propellers at a rate of 85. 1% even when 60% of the propeller is occluded and can run at upto 35Hz on a 2W power budget.
no code implementations • 13 May 2021 • Chethan M. Parameshwara, Simin Li, Cornelia Fermüller, Nitin J. Sanket, Matthew S. Evanusa, Yiannis Aloimonos
Spiking Neural Networks (SNN) are the so-called third generation of neural networks which attempt to more closely match the functioning of the biological brain.
no code implementations • 1 Feb 2021 • Eadom Dessalene, Chinmaya Devaraj, Michael Maynord, Cornelia Fermuller, Yiannis Aloimonos
Human actions involving hand manipulations are structured according to the making and breaking of hand-object contact, and human visual understanding of action is reliant on anticipation of contact as is demonstrated by pioneering work in cognitive science.
1 code implementation • 5 Nov 2020 • Nitin J. Sanket, Chahat Deep Singh, Varun Asthana, Cornelia Fermüller, Yiannis Aloimonos
To our knowledge, this is the first work that applies the concept of morphable design to achieve a variable baseline stereo vision system on a quadrotor.
1 code implementation • 2 Nov 2020 • Kanishka Ganguly, Behzad Sadrfaridpour, Krishna Bhavithavya Kidambi, Cornelia Fermüller, Yiannis Aloimonos
Several end-effector designs for robust manipulation have been proposed but they mostly work when provided with prior information about the objects or equipped with external sensors for estimating object shape or size.
Robotics
no code implementations • 27 Oct 2020 • Matthew Evanusa, Snehesh Shrestha, Michelle Girvan, Cornelia Fermüller, Yiannis Aloimonos
In many real-world applications, fully-differentiable RNNs such as LSTMs and GRUs have been widely deployed to solve time series learning tasks.
no code implementations • 13 Oct 2020 • Matthew Evanusa, Cornelia Fermüller, Yiannis Aloimonos
Deep Reservoir Computing has emerged as a new paradigm for deep learning, which is based around the reservoir computing principle of maintaining random pools of neurons combined with hierarchical deep learning.
no code implementations • 1 Sep 2020 • Matthew Evanusa, Cornelia Fermuller, Yiannis Aloimonos
Here we show that a large, deep layered SNN with dynamical, chaotic activity mimicking the mammalian cortex with biologically-inspired learning rules, such as STDP, is capable of encoding information from temporal data.
1 code implementation • 11 Jun 2020 • Nitin J. Sanket, Chahat Deep Singh, Cornelia Fermüller, Yiannis Aloimonos
Odometry on aerial robots has to be of low latency and high robustness whilst also respecting the Size, Weight, Area and Power (SWAP) constraints as demanded by the size of the robot.
1 code implementation • 11 Jun 2020 • Chethan M. Parameshwara, Nitin J. Sanket, Chahat Deep Singh, Cornelia Fermüller, Yiannis Aloimonos
Segmentation of moving objects in dynamic scenes is a key process in scene understanding for navigation tasks.
no code implementations • 5 Jun 2020 • Eadom Dessalene, Michael Maynord, Chinmaya Devaraj, Cornelia Fermuller, Yiannis Aloimonos
We introduce Egocentric Object Manipulation Graphs (Ego-OMG) - a novel representation for activity modeling and anticipation of near future actions integrating three components: 1) semantic temporal structure of activities, 2) short-term dynamics, and 3) representations for appearance.
no code implementations • 25 Jan 2020 • John Kanu, Eadom Dessalene, Xiaomin Lin, Cornelia Fermuller, Yiannis Aloimonos
While traditional methods for instruction-following typically assume prior linguistic and perceptual knowledge, many recent works in reinforcement learning (RL) have proposed learning policies end-to-end, typically by training neural networks to map joint representations of observations and instructions directly to actions.
2 code implementations • 7 Jun 2019 • Nitin J. Sanket, Chethan M. Parameshwara, Chahat Deep Singh, Ashwin V. Kuruttukulam, Cornelia Fermüller, Davide Scaramuzza, Yiannis Aloimonos
To our knowledge, this is the first deep learning -- based solution to the problem of dynamic obstacle avoidance using event cameras on a quadrotor.
5 code implementations • ICLR 2020 • Chengxi Ye, Matthew Evanusa, Hua He, Anton Mitrokhin, Tom Goldstein, James A. Yorke, Cornelia Fermüller, Yiannis Aloimonos
Convolution is a central operation in Convolutional Neural Networks (CNNs), which applies a kernel to overlapping regions shifted across the image.
no code implementations • 18 Mar 2019 • Anton Mitrokhin, Chengxi Ye, Cornelia Fermuller, Yiannis Aloimonos, Tobi Delbruck
In addition to camera egomotion and a dense depth map, the network estimates pixel-wise independently moving object segmentation and computes per-object 3D translational velocities for moving objects.
no code implementations • 16 Nov 2018 • Konstantinos Zampogiannis, Cornelia Fermuller, Yiannis Aloimonos
In this paper, we introduce a non-rigid registration pipeline for pairs of unorganized point clouds that may be topologically different.
no code implementations • 23 Sep 2018 • Chengxi Ye, Anton Mitrokhin, Cornelia Fermüller, James A. Yorke, Yiannis Aloimonos
In this work we present a lightweight, unsupervised learning pipeline for \textit{dense} depth, optical flow and egomotion estimation from sparse event output of the Dynamic Vision Sensor (DVS).
no code implementations • 13 Jul 2018 • Konstantinos Zampogiannis, Kanishka Ganguly, Cornelia Fermuller, Yiannis Aloimonos
When we physically interact with our environment using our hands, we touch objects and force them to move: contact and motion are defining properties of manipulation.
no code implementations • 2 Jul 2018 • Chengxi Ye, Chinmaya Devaraj, Michael Maynord, Cornelia Fermüller, Yiannis Aloimonos
We introduce Evenly Cascaded convolutional Network (ECN), a neural network taking inspiration from the cascade algorithm of wavelet analysis.
1 code implementation • 1 Jul 2018 • Konstantinos Zampogiannis, Cornelia Fermuller, Yiannis Aloimonos
We introduce cilantro, an open-source C++ library for geometric and general-purpose point cloud data processing.
no code implementations • 28 Jun 2018 • Peter Sutor Jr., Douglas Summers-Stay, Yiannis Aloimonos
Semantic vectors are learned from data to express semantic relationships between elements of information, for the purpose of solving and informing downstream tasks.
no code implementations • 17 May 2018 • Francisco Barranco, Cornelia Fermüller, Yiannis Aloimonos, Eduardo Ros
Conventional image motion based structure from motion methods first compute optical flow, then solve for the 3D motion parameters based on the epipolar constraint, and finally recover the 3D geometry of the scene.
no code implementations • 12 Mar 2018 • Anton Mitrokhin, Cornelia Fermuller, Chethan Parameshwara, Yiannis Aloimonos
Event-based vision sensors, such as the Dynamic Vision Sensor (DVS), are ideally suited for real-time motion analysis.
1 code implementation • 28 Feb 2018 • Huai-Jen Liang, Nitin J. Sanket, Cornelia Fermüller, Yiannis Aloimonos
We merge the successes of these two communities and present a way to incorporate semantic information in the form of visual saliency to Direct Sparse Odometry - a highly successful direct sparse VO algorithm.
1 code implementation • 14 Feb 2018 • Nitin J. Sanket, Chahat Deep Singh, Kanishka Ganguly, Cornelia Fermüller, Yiannis Aloimonos
We use this philosophy to design a minimalist sensori-motor framework for a quadrotor to fly though unknown gaps without a 3D reconstruction of the scene using only a monocular camera and onboard sensing.
Robotics
no code implementations • ICLR 2018 • Sanjukta Krishnagopal, Yiannis Aloimonos, Michelle Girvan
Thus, as opposed to training the entire high dimensional reservoir state, the reservoir only needs to train on these unique relationships, allowing the reservoir to perform well with very few training examples.
no code implementations • 2 Aug 2017 • Chengxi Ye, Yezhou Yang, Cornelia Fermuller, Yiannis Aloimonos
We conclude this paper with the construction of a novel contractive neural network.
no code implementations • 17 Nov 2016 • Somak Aditya, Yezhou Yang, Chitta Baral, Yiannis Aloimonos
We compile a dataset of over 3k riddles where each riddle consists of 4 images and a groundtruth answer.
1 code implementation • 9 May 2016 • Chengxi Ye, Chen Zhao, Yezhou Yang, Cornelia Fermuller, Yiannis Aloimonos
LightNet is a lightweight, versatile and purely Matlab-based deep learning framework.
no code implementations • 8 Mar 2016 • Ruzena Bajcsy, Yiannis Aloimonos, John K. Tsotsos
Despite the recent successes in robotics, artificial intelligence and computer vision, a complete artificial agent necessarily must include active perception.
no code implementations • 29 Jan 2016 • Chengxi Ye, Yezhou Yang, Cornelia Fermuller, Yiannis Aloimonos
For robots that have the capability to interact with the physical environment through their end effectors, understanding the surrounding scenes is not merely a task of image classification or object recognition.
no code implementations • 10 Dec 2015 • Yezhou Yang, Yi Li, Cornelia Fermuller, Yiannis Aloimonos
In this paper we consider the problem of continuously discovering image contents by actively asking image based questions and subsequently answering the questions being asked.
no code implementations • IJCNLP 2015 • Yezhou Yang, Yiannis Aloimonos, Cornelia Fermuller, Eren Erdal Aksoy
In this paper we present a formal computational framework for modeling manipulation actions.
no code implementations • ICCV 2015 • Ching L. Teo, Cornelia Fermuller, Yiannis Aloimonos
Symmetry, as one of the key components of Gestalt theory, provides an important mid-level cue that serves as input to higher visual processes such as segmentation.
no code implementations • ICCV 2015 • Francisco Barranco, Ching L. Teo, Cornelia Fermuller, Yiannis Aloimonos
The bio-inspired, asynchronous event-based dynamic vision sensor records temporal changes in the luminance of the scene at high temporal resolution.
no code implementations • 10 Nov 2015 • Somak Aditya, Yezhou Yang, Chitta Baral, Cornelia Fermuller, Yiannis Aloimonos
Specifically, commonsense reasoning is applied on (a) detections obtained from existing perception methods on given images, (b) a "commonsense" knowledge base constructed using natural language processing of image annotations and (c) lexical ontological knowledge from resources such as WordNet.
no code implementations • CVPR 2015 • Ching Teo, Cornelia Fermuller, Yiannis Aloimonos
To this end, we use several different local cues: shape, spectral properties of boundary patches, and semi-global grouping cues that are indicative of perceived depth.
no code implementations • CVPR 2015 • Yezhou Yang, Cornelia Fermuller, Yi Li, Yiannis Aloimonos
The grasp type provides crucial information about human action.
no code implementations • CVPR 2013 • Yezhou Yang, Cornelia Fermuller, Yiannis Aloimonos
There is a small set of fundamental primitive action consequences that provides a systematic high-level classification of manipulation actions.