Universität Bonn: Autonomous Intelligent Systems Group   Autonomous Intelligent Systems: NimbRo Explorer

CENTAURO Project

Centauro robot  CENTAURO Logo

We coordinated the H2020 Research and Innovation Action CENTAURO – Robust Mobility and Dexterous Manipulation in Disaster Response by Fullbody Telepresence in a Centaur-like Robot.

Summary

Disaster scenarios, like the Fukushima nuclear accident, clearly showed that the capabilities disaster response robots were not sufficient for providing the needed support to rescue workers. The CENTAURO project aimed at development of a human-robot symbiotic system where a human operator is telepresent with its whole body in a Centaur-like robot, which is capable of robust locomotion and dexterous manipulation in the rough terrain and austere conditions characteristic of disasters. The CENTAURO robot consists of a four-legged basis and an anthropomorphic upper body and is be driven by lightweight, compliant actuators. It is able to navigate in affected man-made environments, including the inside of buildings and stairs, which are cluttered with debris and partially collapsed. The Centauro system is capable of using unmodified human tools for solving complex bimanual manipulation tasks, such as connecting a hose or opening a valve, in order to relieve the situation. A human operator can control the robot intuitively using a full-body telepresence suit that provides visual, auditory, and upper body haptic feedback. Rich sensors  provide the necessary situation awareness. Robot percepts and suggested actions are displayed to the operator with augmented reality techniques. For routine manipulation and navigation tasks, autonomous robot skills have been developed. This allows for taking the operator partially out of the control loop, which is necessary to cope with communication latencies and bandwidth limitations and to reduce the operator workload. A series of increasingly complex tests with corresponding evaluation criteria was devised from end-user requirements to systematically benchmark the capabilities of the developed disaster response system.

Publications

Tobias Klamt, Diego Rodriguez, Lorenzo Baccelliere, Xi Chen, Domenico Chiaradia, Torben Cichon, Massimiliano Gabardi, Paolo Guria, Karl Holmquist, Malgorzata Kamedula, Hakan Karaoguz, Navvab Kashiri, Arturo Laurenzi, Christian Lenz, Daniele Leonardis, Enrico Mingo, Luca Muratore, Dmytro Pavlichenko, Francesco Porcini, Zeyu Ren, Fabian Schilling, Max Schwarz, Massimiliano Solazzi, Michael Felsberg, Antonio Frisoli, Michael Gustmann, Patric Jensfelt, Klas Nordberg, Juergen Rossmann, Uwe Suess, Nikos Tsagarakis, and Sven Behnke:
Flexible Disaster Response of Tomorrow - Final Presentation and Evaluation of the CENTAURO System
IEEE Robotics and Automation Magazine, Special Issue on Humanoid Robot Applications in Real World Scenarios, December 2019.
[Video]

Dmytro Pavlichenko, Diego Rodriguez, Christian Lenz, Max Schwarz, and Sven Behnke:
Autonomous Bimanual Functional Regrasping of Novel Object Class Instances
In Proceedings of IEEE-RAS International Conference on Humanoid Robots (Humanoids), Toronto, Canada, October 2019.
[Video]

Tobias Klamt, Max Schwarz, Christian Lenz, Lorenzo Baccelliere, Domenico Buongiorno, Torben Cichon, Antonio DiGuardo, David Droeschel, Massimiliano Gabardi, Malgorzata Kamedula, Navvab Kashiri, Arturo Laurenzi, Daniele Leonardis, Luca Muratore, Dmytro Pavlichenko, Arul S. Periyasamy, Diego Rodriguez, Massimiliano Solazzi, Antonio Frisoli, Michael Gustmann, Jürgen Roßmann, Uwe Süss, Nikos G. Tsagarakis, and Sven Behnke:
Remote mobile manipulation with the Centauro robot: Full‐body telepresence and autonomous operator assistance
Journal of Field Robotics (JFR), Wiley, 2019.
[Video]

Jürgen Beyerer, Frank Kirchner, Alin Albu-Schäffer, Tamim Asfour, Sven Behnke, Andreas Birk, Wolfram Burgard, Thomas Deserno, Jeronimo Dzaack, Thomas Egloffstein, Michael Gustmann, Andreas Nüchter, Patrick Rosen, Hauke Speth, Sirko Straube, Igor Tchouchenkov, Martin Zimmermann:
Retten, schützen, erkunden: Lernende Systeme in lebensfeindlichen Umgebungen - Potenziale, Herausforderungen und Gestaltungsoptionen
Plattform Lernende Systeme, Bericht der AG Lebensfeindliche Umgebungen, Juni 2019.

Daniel Schleich,Tobias Klamt, and Sven Behnke:
Value Iteration Networks on Multiple Levels of Abstraction
In Proceedings of Robotics: Science and Systems (RSS), Freiburg, Germany, June 2019.
[Video]

Tobias Klamt and Sven Behnke:
Towards Learning Abstract Representations for Locomotion Planning in High-dimensional State Spaces
In Proceedings of IEEE International Conference on Robotics and Automation (ICRA), Montreal, Canada, May 2019.
[Video]

Dmytro Pavlichenko, Diego Rodriguez, Max Schwarz, Christian Lenz, Arul Selvam Periyasamy, and Sven Behnke:
Autonomous Dual-Arm Manipulation of Familiar Objects
In Proceedings of IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids), Beijing, China, November 2018.

Diego Rodriguez, Antonio Di Guardo, Antonio Frisoli, and Sven Behnke:
Learning Postural Synergies for Categorical Grasping through Shape Space Registration
In Proceedings of IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids), Beijing, China, November 2018.
[Software]

Arul Selvam Periyasamy, Max Schwarz, and Sven Behnke:
Robust 6D Object Pose Estimation in Cluttered Scenes using Semantic Segmentation and Pose Regression Networks
In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain, October 2018.
[Video]

Tobias Klamt, Diego Rodriguez, Max Schwarz, Christian Lenz, Dmytro Pavlichenko, David Droeschel, and Sven Behnke:
Supervised Autonomous Locomotion and Manipulation for Disaster Response with a Centaur-like Robot
In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain, October 2018.
[Video]

Diego Rodriguez and Sven Behnke:
Transferring Category-based Functional Grasping Skills by Latent Space Non-Rigid Registration
IEEE Robotics and Automation Letters (RA-L), Special Issue on Learning and Control for Autonomous Manipulation Systems: the Role of Dimensionality Reduction, 3(3):2662-2669, April 2018.
[Video]

Diego Rodriguez, Corbin Cogswell, Seongyong Koo, and Sven Behnke:
Transferring Grasping Skills to Novel Instances by Latent Space Non-Rigid Registration
In Proceedings of IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia, May 2018.
[Video]

Tobias Klamt and Sven Behnke:
Planning Hybrid Driving-Stepping Locomotion on Multiple Levels of Abstraction
In Proceedings of IEEE International Conference on Robotics and Automation (ICRA), pp. 1695-1702, Brisbane, Australia, May 2018.
[Video]

Tobias Klamt and Sven Behnke:
Anytime Hybrid Driving-Stepping Locomotion Planning
In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, Canada, September 2017.
Finalist for Best Paper Award on Safety Security and Rescue Robotics.
[Video]

Dmytro Pavlichenko and Sven Behnke:
Efficient Stochastic Multicriteria Arm Trajectory Optimization
In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, Canada, September 2017.
[Video]
 

DLR SpaceBot Cup 2015

13.10.2016

The article by Max Schwarz, Marius Beul, David Droeschel, Sebastian Schüller, Arul Selvam Periyasamy, Christian Lenz, Michael Schreiber, and Sven Behnke:
Supervised Autonomy for Exploration and Mobile Manipulation in Rough Terrain with a Centaur-like Robot
has been published for the journal Frontiers in Robotics and AI, section Humanoid Robotics, vol. 3, article 57, 2016.
DOI: 10.3389/frobt.2016.00057

29.7.2016

The paper by Max Schwarz, Tobias Rodehutskors, David Droeschel, Marius Beul, Michael Schreiber, Nikita Araslanov, Ivan Ivanov, Christian Lenz, Jan Razlaw, Sebastian Schüller, David Schwarz, Angeliki Topalidou-Kyniazopoulou, and Sven Behnke:
NimbRo Rescue: Solving Disaster-Response Tasks through Mobile Manipulation Robot Momaro
has been accepted for publication in the Journal of Field Robotics (JFR), Wiley.
It appeared in volume 34, issue 2, pages 400–425, March 2017.

12.5.2016

The paper by Max Schwarz, Sebastian Schüller, Christian Lenz, David Droeschel, and Sven Behnke:
Supervised Autonomy for Exploration and Mobile Manipulation in Rough Terrain
has been presentated at the 14th International Conference on Intelligent Autonomous Systems (IAS), Shanghai, China, July 2016.

15.1.2016

The paper by Max Schwarz, Tobias Rodehutskors, Michael Schreiber, and Sven Behnke:
Hybrid Driving-Stepping Locomotion with the Wheeled-legged Robot Momaro
has been presentated at the IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden, May 2016. [Video]

16.11.2015

Press release of University of Bonn: Roboter überzeugt beim DLR SpaceBot Camp

13.11.2015

Our team NimbRo Explorer solved through the mobile manipulation robot Momaro all tasks at the DLR SpaceBot Camp: taking a soil sample, finding and grasping two objects, transporting them to a basis object, assembly (placing the soil sample at a scale and inserting a battery pack) and operating a switch.

DLR SpaceBot Camp: Momaro taking a soil sample DLR SpaceBot Camp: Momaro taking a soil sample DLR SpaceBot Camp: Momaro grasping the mug DLR SpaceBot Camp: Momaro grasping the battery DLR SpaceBot Camp: Momaro driving uphill DLR SpaceBot Camp: Momaro placing mug DLR SpaceBot Camp: Assembled basis object Team NimbRo Explorer @ DLR SpaceBot Camp Momaro removing debris after solving the DLR SpaceBot Camp tasks

12.11.2015

Today is a testing day. Our team has two scheduled test slots.

Momaro 3D map of the arena with robot trajectory

11.11.2015

Our team NimbRo Explorer participates with the mobile manipulation robot Momaro at the DLR SpaceBot Camp 2015 in Hürth.Ten teams from leading German robotics labs will demonstrate semi-autonomous mobile manipulation in a moon-like environment. Based on a coarse map, the robot must explore the arena, find two objects, take a soil sample, transport the objects to a base and assemble them there. Communication to the operator crew is possible only sometimes and has 4 seconds latency.

Team NimbRo Explorer 10 participating robots

30.10.2015

A team from DLR made a teaser video.

10.9.2015

Our team NimbRo Explorer qualified as first of the ten qualification event participants for the DLR SpaceBot Cup. Through the mobile manipulation robot Momaro, our operators solved all three qualification tasks in the first attempt. The DLR SpaceBot Cup qualification event took place in Bonn. 

Momaro in the SpaceBot Cup qualification arena Momaro grasping battery pack Momaro grasping mug Mobile manipulation robot Momaro Momaro placing mug Qualification team NimbRo Explorer

3.7.2015

Representatives of DLR and the network provider visited our lab. The DLR communication network was tested sucessfully. The Momaro robot was demonstrated.

6.6.2015

Our new mobile manipulation robot Momaro was tested at the DARPA Robotics Challenge in Pomona, CA. Our team NimbRo Rescue was ranked as the best European team, comming in 4th in the overall ranking. Though the Momaro robot, the operators solved seven of the eight possible tasks in only 34 minutes: driving a car, climbing out of the car, opening a door, turning a valve, cutting a hole in a wall, traversing debris, and a surprise task, which was operating an electrical switch. More information. 

Momaro robot with field crew Operator station Car driving Egress car Door opening Valve turning Wall cutting Surprise task: operating a switch Traversing debris Greeting the audience Team NimbRo Rescue Mobile Manipulation Robot Momaro

15.4.2015

Representatives of DLR visited our lab for the review meeting "Design".

4.3.2015

The article by Jörg Stückler, Max Schwarz, Mark Schadler, Angeliki Topalidou-Kyniazopoulou, and Sven Behnke:
NimbRo Explorer: Semi-Autonomous Exploration and Mobile Manipulation in Rough Terrain
has been accepted for the Journal of Field Robotics. It appeared in vol. 33(4):411-430, Wiley, 2016.

Nimbro Explorer robot, local height map, 3D map

8.12.2014

Our Team NimbRo Explorer is supported by the German Aerospace Center to participate at the DLR SpaceBot Cup 2015.

DLR SpaceBot Cup 2015

Tasks in this challenge are semi-autononomous mobile manipulation in rough terrain. The Kickoff meeting took place in Bonn. 

18.3.2014

Max Schwarz and Sven Behnke:
Local Navigation in Rough Terrain using Omnidirectional Height
In Proceedings of Joint 45th International Symposium on Robotics (ISR) and 8th German Conference on Robotics (ROBOTIK), Munich, June 2014.

Omnidirectional Terrain Perception for Local Navigation    

10.3.2014

Mark Schadler, Jörg Stückler, and Sven Behnke:
Rough Terrain 3D Mapping and Navigation using a Continuously Rotating 2D Laser Scanner
German Journal on Artificial Intelligence (KI) 28(2):93-99, Springer, 2014.

DLR SpaceBot Cup 2013

18.12.2013 

Video of our Explorer robot demonstrating exploration and mobile manipulation at the DLR SpaceBot Cup.

22.11.2013

Map of the SpaceBot Cup arena created from 3D laser scans taken after the end of the competition.

SpaceBot Cup arena map

20.11.2013

An informative article on the competition by dw.de: Weltraum-Roboter im Wettstreit

13.11.2013

Press release of University of Bonn on the results of our team: Roboter demonstrieren mobile Manipulation

12.11.2013

In the afternoon, all teams had the opportunity to simultanously operate their robots in the arena. Our Explorer robot demonstrated autonomous grasping and storage of the battery pack and autonomous removal of the battery pack from its holder. It also removed the cup from the holder and placed it on the scale. The jury decided to award no trophies to any of the teams. No competition scores have been communicated to the teams so far.

Grasping the battery Grasping the battery Storing the battery Regrasping the batteryTrying to insert the battery Removing the cup from the holder Placing cup on the scale Cup on the scale Team NimbRo DLR SpaceBot Cup Robots

11.11.2013

Our Explorer robot had its run in the unknown arena today. It created a 3D map of the terrain and navigated to one of the objects that where placed in the arena. It grasped this water-filled cup autonomously and placed it in the holding mechanism without spilling any water. After the official run ended, the robot navitated to the other side of the arena and climbed a hill.

Approaching cup Grasping cup Grasping cup Storing cup Climbing hill 

10.11.2013

We tested in a small area outside the arena. We also tested the ground station and the network provided by DLR.

NimbRo Explorer Robot NimbRo network debugging Testing  Storing cup after autonomously grasping it  Operator station Programming team

9.11.2013

First evening in the SuperCross hall, Rheinbreitbach.
The arena contains a variety of surfaces and obstacles.

DLR SpaceBot Cup Arena

7.11.2013 

University of Bonn released information for the media.

6.11.2013

A first photo of our Explorer robot.

Explorer robot with Jörg Stückler (left) and Max Schwarz (right)

30.10.2013

DLR published a video introducing our team.

26.10.2013

The paper Mark Schadler, Jörg Stückler and Sven Behnke:
Multi-Resolution Surfel Mapping and Real-Time Pose Tracking using a Continuously Rotating 2D Laser Scanner
has been presented at the 11th IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), Linköping, Sweden, October 2013.

3D map of a garage

Further information on our entry NimbRo Centauro to the DLR SpaceBot Cup 2013 is available here.

     University of Bonn, Institute for Computer Science, Departments: I, II, III, IV, V, VI   Impress    Data Privacy Statement