Rich Babies, Poor Robots: Towards Rich Sensing, Continuous Data and Multiple Environments

Abstract: In recent years, we have seen a shift in different fields of AI such as computer vision, robotics. From task-driven supervised learning, we are now starting to see a shift towards more human-like learning. Self-supervised learning, embodied AI, multimodal learning are few subfields which have emerged from this shift. Yet I will argue the shift is half-hearted in nature and there is a huge situational gap between babies (human learners) and current robots. Our babies learn continuously from multiple environments using five different senses using both active and passive data. On the other hand, our AI algorithms still primarily use vision (best case), learn from fixed datasets or pre-defined environments, and use either passive or active data. In this talk, I will argue how to bridge this gap. First, I will talk about how to bring tactile sensing into the mainstream. More specifically, I will introduce our magnetic sensing skin called ReSkin. Next, I will talk about how our current setups lack the lifelong learning aspect. More specifically, I will introduce our recent efforts in developing continuous versions of self-supervision and curiosity/exploration. Finally, if time remains, I will talk about how to use passive and active data together to learn actions.

Bio: Abhinav Gupta is an Associate Professor at the Robotics Institute, Carnegie Mellon University. His research focuses on scaling up learning by building self-supervised, lifelong and interactive learning systems. Specifically, he is interested in how self-supervised systems can effectively use data to learn visual representation, common sense and representation for actions in robots. Abhinav is a recipient of several awards including IAPR 2020 JK Aggarwal Prize, PAMI 2016 Young Researcher Award, ONR Young Investigator Award, Sloan Research Fellowship, Okawa Foundation Grant, Bosch Young Faculty Fellowship, YPO Fellowship,  IJCAI Early Career Spotlight, ICRA Best Student Paper award, and the ECCV Best Paper Runner-up Award. His research has also been featured in Newsweek, BBC, Wall Street Journal, Wired and Slashdot.