Title: Multi-modal and Cross-modal Robot Visual-Tactile Perception
Abstract: In our daily life, both vision and touch are indispensable sensing methods for us to interact with the surrounding environment, and the two often assist each other to complete some tasks, such as grasping objects. Similarly, visual and tactile perception can also assist each other as the main channel for robots to understand the surrounding environment. Inspired by this, we developed a new type of touch sensor named GelTip, which is shaped like a human finger. It uses a vision camera at the bottom to capture the contact between the sensor surface elastomers and the contacting object, and extracts information about the contacting object based on the obtained tactile image. We have also designed a series of learning algorithms for extracting information from both visual and tactile images. Through the integration of vision and touch (both multi- and cross-modal), it can effectively improve the robot's object perception ability, so that the robot can effectively improve its capability of object recognition and localisation. Recently, we have also developed a simulation model of high-resolution tactile sensors that can be applied in the widely used ROS simulation environments and enables the Sim2Real learning with tactile sensing. Our research can be used to improve the dexterous manipulation of robots, suitable for warehouse robotics, inspection, medical robotics and other fields.
Short bio: Dr. Shan Luo leads the advanced robotics laboratory smARTlab at the Department of Computer Science, the University of Liverpool. His main research interests are in visuo-tactile robotics, including the development of robot visuo-tactile sensors, object recognition and localisation using vision and touch, and multimodal perception. Dr. Luo received his Ph.D. in Robotics from King's College London in 2016. He visited the MIT Computer and Artificial Intelligence Laboratory (CSAIL) in 2016. After graduating from his Ph.D., he worked as a Postdoctoral Research Fellow at the University of Leeds and Harvard University, and joined the University of Liverpool in 2018. He has more than 40 publications in high-impact robotics journals and international conferences (including Autonomous Robots, ICRA, IROS, ICML. AAMAS, etc.). He will join King’s College London as a Senior Lecturer in the fall of 2021.