Joint Hand-Object Pose Estimation

HOnnotate - A method for 3D Annotation of Hand and Objects Poses

We propose a method for annotating images of a hand manipulating an object with the 3D poses of both the hand and the object, together with a dataset created using this method. There is a current lack of annotated real images for this problem, as estimating the 3D poses is challenging, mostly because of the mutual occlusions between the hand... [Read More]

Joint Hand-Object Pose Estimation

Generalized Feedback Loop for Joint Hand-Object Pose Estimation

We propose an approach to estimating the 3D pose of a hand, possibly handling an object, given a depth image. We show that we can correct the mistakes made by a Convolutional Neural Network trained to predict an estimate of the 3D pose by using a feedback loop. The components of this feedback loop are also Deep Networks, optimized using... [Read More]

PhD Thesis

3D Hand Pose Estimation from Images for Interactive Applications

Hands are the most important body part for humans to interact with and manipulate their environment. Hence, 3D hand pose estimation is an important cornerstone of many Human-Computer Interaction (HCI), Virtual Reality (VR), and Augmented Reality (AR) applications, such as robotic control or virtual object interaction. Despite the plentiful applications and importance for the industry, hand pose estimation is still... [Read More]

Hand Segmentation

HandSeg: An Automatically Labeled Dataset for Hand Segmentation from Depth Images

We propose an automatic method for generating high-quality annotations for depth-based hand segmentation, and introduce a large-scale hand segmentation dataset. Existing datasets are typically limited to a single hand. By exploiting the visual cues given by an RGBD sensor and a pair of colored gloves, we automatically generate dense annotations for two hand segmentation. This lowers the cost/complexity of creating... [Read More]

Domain Transfer for 3D Pose Estimation

Domain Transfer for 3D Pose Estimation from Color Images without Manual Annotations

We introduce a novel learning method for 3D pose estimation from color images. While acquiring annotations for color images is a difficult task, our approach circumvents this problem by learning a mapping from paired color and depth images captured with an RGB-D camera. We jointly learn the pose from synthetic depth images that are easy to generate, and learn to... [Read More]

Deep Heatmaps for 3D Object Pose Estimation

Making Deep Heatmaps Robust to Partial Occlusions for 3D Object Pose Estimation

We introduce a novel method for robust and accurate 3D object pose estimation from a single color image under large occlusions. Following recent approaches, we first predict the 2D projections of 3D points related to the target object and then compute the 3D pose from these correspondences using a geometric method. Unfortunately, as the results of our experiments show, predicting... [Read More]

Transfer Learning for 3D Pose Estimation

Feature Mapping for Learning Fast and Accurate 3D Pose Inference from Synthetic Images

We propose a simple and efficient method for exploiting synthetic images when training a Deep Network to predict a 3D pose from an image. The ability of using synthetic images for training a Deep Network is extremely valuable as it is easy to create a virtually infinite training set made of such images, while capturing and annotating real images can... [Read More]

Hand-Object Interaction

Efficient Physics-Based Implementation for Realistic Hand-Object Interaction in Virtual Reality

We propose an efficient physics-based method for dexterous ‘real hand’-‘virtual object’ interaction in Virtual Reality environments. Our method is based on the Coulomb friction model, and we show how to efficiently implement it in a commodity VR engine for real-time performance. This model enables very convincing simulations of many types of actions such as pushing, pulling, grasping, or even dexterous... [Read More]

Deep Prior++ for Hand Pose Estimation

DeepPrior++: Improving Fast and Accurate 3D Hand Pose Estimation

DeepPrior is a simple approach based on Deep Learning that predicts the joint 3D locations of a hand given a depth map. Since its publication early 2015, it has been outperformed by several impressive works. Here we show that with simple improvements: adding ResNet layers, data augmentation, and better initial hand localization, we achieve better or similar performance than more... [Read More]

Training Data for Hand Pose Estimation

Efficiently Creating 3D Training Data for Fine Hand Pose Estimation

While many recent hand pose estimation methods critically rely on a training set of labelled frames, the creation of such a dataset is a challenging task that has been overlooked so far. As a result, existing datasets are limited to a few sequences and individuals, with limited accuracy, and this prevents these methods from delivering their full potential. We propose... [Read More]

Deep Prior for Hand Pose Estimation

Hands Deep in Deep Learning for Hand Pose Estimation

We introduce and evaluate several architectures for Convolutional Neural Networks to predict the 3D joint locations of a hand given a depth map. We first show that a prior on the 3D pose can be easily introduced and significantly improves the accuracy and reliability of the predictions. We also show how to use context efficiently to deal with ambiguities between... [Read More]

Master's Thesis

Embeddings for Random Ferns Classification

Efficient multi-class machine learning methods are a key component in many computer vision applications. In this area the Random Forest classifier is considered state-of-the-art as it provides an efficient way to learn an inherently multi-class classifier, that naturally handles high dimensional, multi-modal data. Recent research focused on optimizing this classifier for different applications, by learning appropriate node split criteria. In... [Read More]

Insulator Fault Detection

Visual Recognition and Fault Detection for Power Line Insulators

The inspection of high voltage power lines is an important task in order to prevent failure of the transmission system. In this work, we present a novel approach to detect insulators in aerial images and to analyze them automatically for possible faults. Our detection algorithm is based on discriminative training of local gradient-based descriptors and a subsequent voting scheme for... [Read More]