Project Description

At ZenRobotics, I led a pivotal project as the tech lead that achieved recognition with its mid-stage results presented at the IROS 2016 conference (https://arxiv.org/abs/1609.01044). This project tackled the complex challenge of autonomously sorting densely cluttered piles of unknown objects using a robotic system, highly relevant to the field of robotic waste sorting.

Our innovative approach extended active learning methods to grasping, enabling the system to learn the task autonomously. Instead of predicting grasp success, it forecasted the object classes to be picked and placed on a conveyor, simplifying subsequent segmentation and identification. Remarkably, we avoided analyzing the cluttered working area directly; instead, we allowed the controller to learn a mapping from an RGBD image near the grasp to a predicted result.

The grasp selection comprised two stages: an initial hardcoded stage generated potential grasps, and a second stage used a learned criterion to make the final grasp selection. In experiments, the system rapidly acquired the ability to make precise pickups, predict object classes in advance, and efficiently sort objects by color from dense clutter. We harnessed Python, TensorFlow, scikit-learn, and OpenCV in this groundbreaking project.