Universität Bonn: Autonomous Intelligent Systems Group   Autonomous Intelligent Systems: NimbRo @Home

RoboCup 2013, Eindhoven


Our team NimbRo won the @Home league of the service robots at RoboCup German Open 2014.

In the final, our robot Cosero demonstrated the use of tools. It opened a bottle and took a saussage from the grill. The robot also demonstrated bimanual grasping and transport of a tray. Our team reached a perfect normalized score of 100, followed byTech United Eindhoven (59 points) and ToBI (Bielefeld University, 50 points).

FInal: Cosero opens a bottle Fiinal: Handing over a bottle Final: Grasping saussage Final_ Cosero placing saussage Final: Cosero grasping tray Final: Cosero transporting tray Team NimbRo@Home @ RoboCup German Open


Our team NimbRo participates at the RoboCup German Open competition in Magdeburg in the @Home League. After Stages I and II, we are leading the competition with 7919 points, followed by Tech United Eindhoven (3611 points) and ToBI (Bielefeld University, 3069 points).

Bottle Opening Grasping Hand Over Understanding complex speech commands Emergency Test Bluetooth object localization


Our team NimbRo won the @Home League competition at RoboCup 2013!

In the final, our cognitive service robot Cosero demonstrated tool use. He used a pair of tongs to cook a sausage on the grill and a bottle opener to open a beer that he served to a person. This demonstration convinced the high-profile jury which awarded the highest number of points. Together with the results of Stage 1 and 2, the final normalized score was 99 points for NimbRo, followed by Wright Eagle (China, 86 points) and TU Eindhoven (Netherlands, 73 points).

RoboCup 2013 NimbRo@Home Final: Tool use to make saussages RoboCup 2013: NimbRo@Home Final: Tool use to make saussages RoboCup 2013: NimbRo@Home Final: Bottle opening RoboCup 2013: NimbRo@Home Final: Bottle opening RoboCup 2013: NimbRo@Home Final: Placing tool RoboCup 2013: NimbRo@Home Final: Handing over bottle


Our robot Cosero performed two tests. In the Restaurant test, it followed a guide and learned a model of an unknown environment. It also used speech recognition to take orders. In the Demo Challenge the motto was health care. Our team demonstrated a handheld teleoperation interface. The robot also moved a chair to its location. Cosero also attempted to open a bottle with a tool.

RoboCup 2013: NimbRo@Home: Restaurant test RoboCup 2013: NimbRo@Home: Restaurant test RoboCup 2013 Handheld Teleoperation Interface RoboCup 2013: NimbRo@Home Demo Challenge RoboCup 2013 NimbRo Bottel Opening RoboCup 2013 NimbRo Grasping Chair


Today, our robot Cosero scored in all three tests. In the Emergency Test, it guided a person who did not the way to the exit of the arena. In the Clean-up test, it found two objects and put one of them on its place. In the EGPSR test, it understood a complex speech command and fetched an object. 

RoboCup 2013: NimbRo Emergency Test Clean-up test EGPSR test EGPSR test RoboCup 2013: @Home League


The highlight of today's Open Challenge was the demonstration of grasp transfer. Our robot Cosero registered a model of a known watering can with an unknown, different watering can. The non-rigid registration allowed to transfer grasp points and the tooltip position to the new watering can. Our robot grasped the can successfully with both hands, used it to water a plant, and placed the can back on a table.

RoboCup 2013: Programming Cosero Follow-Me test Open Challenge: Grasp Transfer Open Challenge: Cosero with watering can Open Challenge: Object placement


Our team NimbRo@Home participates at the @Home League competition of RoboCup 2013 in Eindhoven with the robots Dynamaid and Cosero.
On the first day, the robots went through inspection and registration.

RoboCup 2013, NimbRo@Home, Cosero RoboCup 2013, NimbRo@Home, Dynamaid RoboCup 2013, NimbRo@Home repairs RoboCup 2013 NimbRo@Home Robot Inspection RoboCup 2013 NimbRo@Home Robot Inspection

RoboCup German Open 2013, Magdeburg


Our team NimbRo@Home won RoboCup German Open 2013.

In the final, our robots Dynamaid and Cosero cooperated. Dynamaid took an order, transmitted it over WiFi to Cosero who delivered a drink.
Cosero also pushed away a chair and Dynamaids delivered drinks and snacks to the jury.
Our performance received the highest score from bot the execs and the external experts. Together with our lead from Stages 1 and 2, we reached the maximal normalized score of 1, followed by SmartBots@Ulm (score .67) and team Homer (Univ. of Koblenz, score .61).

NimbRo@Home Final at RoboCup German Open 2013 NimbRo@Home Final at RoboCup German Open 2013NimbRo@Home Final at RoboCup German Open 2013 Team NimbRo@Home at RoboCup German Open 2013 RoboCup 2013 @Home Award Ceremony: Winner team NimbRo


In the Demo Challenge, Cosero was teleoperated by an immobile user who could command the robot navigation and manipulation on three levels of autonomy using a tablet. Cosero fetched items and handed them over to the user.
In the Extended General Purpose Service Robot test, Cosero had to understand several complex speech commands and to solve the requested tasks.
In the Restaurant test, Cosero followed a human guide and build a map of an unknown environment with marked object positions to which it navigated on spoken request.
After Stage 2, our team NimbRo is leading with 6297 points, followed by SmartBots@Ulm (3242 points) and ToBi (3198 points).

Demo challenge Extended General Purpose Service Robot test Restaurant test


In the Clean-up test, our robot Cosero showed that it can recognize object which were distributed in the arena. It grasped a drink and medicine from the fridge and delivered it to their respective locations. While placing the beer on the couch table, it spilled it, however. Afterwards, it found two more drinkson the dinner table and attempted to store them when the test timed out.
In the Emergency Situation test, Cosero guided a person to the exit. Our robot served as a butler in the Open Challenge, delivering drinks to waving persons. After Stage 1, NimbRo is leading with 4247 points, followed by SmartBots@Ulm (2792 points) and TOBI (2748 points).

Clean Up test Clean Up test Emergency Situation test Open Challenge


Our team NimbRo participates at the @Home League competiton of RoboCup German Open 2013 in Magdeburg.
On the first competition day, our robots Dynamaid and Cosero registered for the competiton by handing over the registration form.This RIPS test was evaluated by the team leaders and the Technical Commitee. Our robotsreceived the top score of 925 points.
A mechanical problem in Cosero's torso prevented to participate in the Follow Me test.
The Cocktail Party test went very well. Cosero took orders, fetched and delivered drinks.
After the three tests, our team is leading (1775 points) followed by SmartBots@Ulm (1561 points) and TOBI (Univ. of Bielefeld, 1471 points).

 Registration (RIPS) Cocktail Party

RoboCup 2012, Mexico City


Our team won the @Home League competition.

In the final, our robot Cosero demonstrated the approaching, bi-manual grasping and moving of a chair to a target pose. It also approached and grasped a watering can with both hands and watered a plant. After this demonstration, our robot Dynamaid fetched a drink and delivered it to the jury. In the meanwhile, Cosero approached a transport box, from which it grasped an object using grasp planning. This demonstration convinced the high-profile jury, which awarded the highest number of points in both categories (league-internal jury: scientific contribution, relevance, presentation and performance; external jury: originality, usability, difficulty and success). Together with the lead after Stage 2, our team received 100 normalized points, followed by eR@sers (Japan, 74 points) and ToBi (Univ. of Bielefeld, 64 points).

RoboCup 2012 @Home Final: Cosero grasping a chair RoboCup 2012 @Home Final: Cosero grasping a watering can RoboCup 2012 @Home final: Cosero watering a plant RoboCup 2012 @Home final: Dynamaid fetching a drink RoboCup 2012 @Home Final: Dynamaid and the Jury RoboCup 2012 @Home final: Cosero grasping an object out of a box RoboCup 2012: team NimbRo@Home at award ceremony


In the General Purpose Service Robot test, our robot Cosero had to recognize complex speech commands. It correctly recognized two commands, which gave the most points in this test.
In the Demo Challenge, an immobile person used a handheld PC to teleoperate the robot. The person sent the robot to fetch a drink. The robot recognized that the requested drink was not available and the user could select another drink in the transmitted camera image. After the robot delivered the drink, it recognized a pointing gesture and navigated to the referenced object in order to pick it up from the ground. 
In the Restaurant test, our robot Cosero way guided through a previously unknown bar. The guide showed the robots where the shelves with drinks and the individual tables where. Our robot built a map of this environment and took an order. Afterwards, it navigated to the food shelf to search for requested snacks.
After Stage 2, NimbRo is leading with 6938 points, followed by WrightEagle (4677 points) and eR@sers (Japan, 3547 points).

General Purpose Service Robot test Demo Challenge: Teleoperation Interface Demo Challenge: Object Handover Demo Challenge: Pointing Gesture Recognition Restaurant Challenge 


In the Clean Up test, our robot Cosero had to find objects that were distributed in the apartment, recognize them, and bring them to their place. Our robot detected three objects, from which two were correctly recognized as unknown objects. It grasped all three objects and deposited them in the trash bin.
In the Open Challenge, Cosero demonstrated that it could recognize a waving person. It took over an empty cup from this person and threw it into the trash bin. Afterwards, it approached a watering can and watered a plant. After finishing all test of  Stage 1, NimbRo is leading (5071 points), followed by WrightEagle (3398 points) and ToBi (2627 points).

RoboCup 2012: Clean Up test RoboCup 2012: Clean Up testRoboCup 2012: Clean Up test RoboCup 2012: Open Challenge RoboCup 2012: Open Challenge RoboCup 2012: Open Challenge


Our robot Cosero performed the new Follow-Me test. It learned the face of a previously unknown person, followed it although its line-of-sight to the person was blocked temporarily, and went with the person into a small elevator. At another floor, the robot went out of the elevator again.
In the Who is Who test, Cosero performed very well. He learned the faces of three persons, took an order, fetched three drinks, and delivered two of them, before the time run out. After the third test, NimbRo is leading with 2399 points, followed by WrightEagle (China, 2098 points) and ToBi (Bielefeld, 1283 points).

RoboCup 2012: Follow Me test RoboCup 2012 Follow Me Ellevator Follow Me: After leaving the ellevator Who is Who: Cosero fetches the drinks Who is Who: Cosero delivering a drink


Our team participates at RoboCup 2012 in Mexico City. On the second setup-day, the robots had to register for the competition while Jörg presented our team poster to the other team leaders.

RoboCup 2012: Michael and Max assembling Dynamaid RoboCup 2012: Manus testing Follow-Me Jörg and Ishrat working on object recognition  RoboCup 2012 Cosero at the RIPS test

RoboCup German Open 2012, Magdeburg


Our team won the RoboCup German Open @Home competition. 

In the final, our robot Cosero showed the pushing of a chair and the watering of a plant. For both tasks, it used real-time registration of 3D models with the current RGB-D camera images. Our robot Dynamaid delivered some snacks to the jury. Jury members could raise their hands to get a snack. This performance was awarded the highest number of points by both parts (exec and external) of the jury. 
The team b-it-bots (Hochschule Rhein-Sieg,Sankt Augustin) reached the second place in the @Home competition. Team Golem from Mexico came in third.

Cosero in the Final, pushing a chair Cosero in the Final, with a watering-can Cosero in the Final, watering a plant Dynamaid in the Final, delivering snacks to the jury  Team NimbRo@Home at RoboCup German Open 2012


In the Demo Challenge, our robot Cosero recognized colored and white laundry shown to it, recognized pointing gestures, and picked up laundry  from the floor. It also put away two kitchen items into the sink. 
Cosero performed the Enhanced Who Is Who test almost perfectly. It learned names and faces of three persons, understood a drink order, fetched three drinks and delivered two of them to the correct person.
The Shopping Mall test also went very well. Here, Cosero learned the map of an unknown environment by following a guide, took a shopping order and collected two of the three requested objects.
Consequently, after Stages 1 and 2, our team is leading with 8550 points, followed by b-it-bots (4781 points) and SmartBots (4291 points).

Cosero in the Demo Challange, picking up an object from the floor Cosero in the Demo Challange, putting a bowl into the sink Cosero during the Enhanced Who Is Who test, searching for a person to deliver a drink Cosero in the Shopping Mall test grasping one of the requested objects


In the Who Is Who test, our root Cosero found four persons and recognized three of them. 
In the General Purpose Service Robot test, Cosero understood three complex speech commands and found the requested objects.
The last test today was the Open Challenge. Here, Cosero was guided by a human by taking its hand, carried a table together with the guide, and pushed a chair to the table. After the second day, we were leading the competition with 5300 points, followed by SmartBots (Ulm, 3491 points) and b-it-bots (Sankt Augustin, 3431 points).
Cosero approaching a person in the Who is Who test Cosero at the General Purpose Service Robot test understands a complex speech command Cosero at Open Challange pushing a chair


Our team participates at the RoboCup German Open 2012 competition in Magdeburg.
On the first day, our robots had to register themselves and to perform the Follow Me and Go Get It tests. 
In Follow Me, our robot Cosero learned the face of a previously unknown person and followed him trough a parcour. The robot was not disturbed by another person walking in between the two, recognized its leader again after he disappeared, and recognized a stopping gesture.
In Go Get It, Cosero found the requested object in the kitchen, but could not reach it, because it was outside the workspace of its arms.
After the first day, our team is leading with 2340 points, followed by SmartBots (Ulm, 2004 points) and Golem (Mexico, 1604 points).

 Dynamaid and Cosero registering for the competition Cosero learns the face of a prevously unknown person in the Follow Me test Cosero grasping a cup during the Go Get It test

RoboCup 2011, Istanbul


At the 15th RoboCup International Symposium, the paper Jörg Stückler and Sven Behnke: "Compliant Task-Space Control with Back-Drivable Servo Actuators" received the Best Paper Award.


We won the @Home competition.

In the final, we used both robots. Cosero carried a table, together with a person who  guided the robot by pulling and pushing the table. Cosero also turned on the cooking plate, fetched some egg dow and poured it into a pan to make an omelet. In the meanwhile, Dynamaid got a bottle of orange juice from the fridge. This performance was awarded the highest score from all three parts of the jury. Together with the top score from Stage II, this resulted in an optimal 300 points.

RoboCup@Hoime 2011 Final: Cosero carrying a table together with a person RoboCup@Home 2011 Final: Cosero making omelet RoboCup@Home 2011 Final: Dynamaid RoboCup@Home 2011 Final: Omelet


The General Purpose Service Robot test went very well. Our robot Cosero understood complex speech commands, fetched the requested object, and was not confused by a command to fetch an object that was not at the specified location. In the Demo Challenge, Cosero cleaned the appartment. A human could show the robot where white and colored loundry was to be put and the robot collected some clothes. It also cleaned up other objects and put them at their place. This was awarded with the highest number of points of all remaining 12 teams. After Stage II, our team is leading with some distance, followed by WrightEagle from China.


In the General Purpose Service Robot test, Cosero partially understood a complex command and went to the correct place. The Go-Get-It test went well. Cosero found a known object and delivered it. In the Shopping Mall test, Cosero had severe difficulties with speech recognition. Hence, it could not take the order which items should be shopped and received only partial points.

General Purpose Service Robot test Go-Get-It test Cosero in the supermarket General Purpose Service Robot Test 2 


Cosero demonstrated in the Follow-Me test that he can follow a previously unknown person, recognize gestures and distinguish this person from another person. Due to the noisy environment, Cosero had some difficulties to understand names in the Who-is-Who test. Nevertheless, our robot detected and recognized four persons correctly. In the Open Challenge, Cosero prepared a breakfast. It opened a bottle, poured milk in a bowl and fetched a spoon. This performance was rewarded with the highes number of points of all teams.

Cosero is introduced to a person in the Follow-Me test Who-is-Who test Open Challenge


Our team NimbRo participates at RoboCup 2011, which takes place at the Expo Center in Istanbul.
In the evening, our @Home robots Dynamaid and Cosero registered themselves for the competition in the Robot Inspection and Poster Session test. They reached the highest number of points of all 19 teams in this test.

Domestic service robot Cosero learns the objects Robots Dynamaid and Cosero during the RIPS test

RoboCup German Open 2011, Magdeburg


Our Robots Cosero and Dynamaid won the @Home league!

In the final, our robots prepared a breakfast. Cosero opened a milk bottle, poured out the milk over the cereal bowl, and threw away the bottle into the trash bin. In the meanwhile, Dynamaid got the orange juice from the fridge. Cosero also could fetch a spoon, which had been placed in arbitrary position on a table by a jury member. Finally, the robot recognized a pointing gesture of a jury member showing the exit door. SmartBots (Ulm) came in second, b-it-bots (Sankt Augustin) reached third place.

Cosero pouring out a bottle of milk. Dynamaid at the fridge Cosero grasping a spoon


Our robot Cosero performed the Enhanded Who is Who test, the Shopping Mall test, and the Demo Challenge. Im Shopping Mall test, our robot learned the map of a previously unknown environment, was introduced to objects and delivered two of the three requested objects. In the Demo Challenge, Cosero recognized objects, interpreted pointing gestures and tidied up the appartment by moving objects into shelves. 

Cosero grasps an object during the Shopping Mall test Cosero interprets a pointing gesture during the Demo Challange


Cosero got the most points in the Who is Who test by finding four persons. It got the second most points in the Open Challenge. Dynamaid was the only robot to score in the General Purpose Robot Test II by understanding an underspecified command, asking for the missing parts, and working on the task.

Cosero finds persons during the Who is Who test Cognitive Service Robot Cosero at German Open 2011


We participate at RoboCup German Open in Magdeburg in the @Home league with the service robots Cosero and Dynamaid.
The robots had to show their skills in the Stage 1 tests Robot Inspection and Poster Session, Follow Me und Go Get It!. 

RoboCup 2010, Singapore

In the @Home League final, Dynamaid recognized an object showing gesture and the object shown to her. She also went to the fridge, opened the fridge door, and grasped a can of beer. Afterwards, she closed the fridge again and delivered the drink. Finally, our robot went to a shelf to get some snack. [Video]
In the final ranking, the Japanese team Er@ser was still ahead of NimbRo. NimbRo won the second place in the @Home League

Dynamaid did very well in the Shopping Mall test. This test was conducted in a real toy store. Our robot learned a map of the environment, remembered the location of multiple objects and then went to collect a requested object. [Video]
In the Enhanced-Who-is-Who test, our robot learned the appearance and the names of three persons. One of these persons was recognized later when ordering drinks. In the Demo Challenge, Dynamaid successfully recognized pointing gestures, recognized objects shown to her, and picked-up objects from the floor that she deposited in a thresh bin. After completing all Stage II, tests NimbRo had the second most points in the overall ranking, next only to to the Japanese team Er@sers. 

Dynamaid performed very well in the Open Challenge. She recognized gestures, got something to eat and to drink for a guest and picked-up an object from the floor. After Stage I, NimbRo is second in the overall ranking, next only to team Er@sers from Japan. In the General-Purpose-Service-Robot test, Dynamaid recognized complex commands and asked for clarification in underspecified commands.

Dynamaid opens the fridge in the final Dynamaid closing the fridge after getting the beer Oliver switching on the microphone Dynamaid fetches Pringles in the Supermarket test Dynamaid taking orders during the Enhanced-Who-is-Who test

Dynamaid did the FollowMe test perfectly. During the test, she successfully recognized stop gestures. After the GoGetIt test, NimbRo still has the highest total number of points.  

Our team participates at the RoboCup 2010 competition in Singapore in the @Home League with the robot Dynamaid. In the first test, Dynamaid registered herself successfully for the competition while the team poster was presented. NimbRo reached the highest number of points for this test.

Dynamaid handing over the registration form  Robot assembly Dynapaid recognizes Stop gesture Dynamaid recognizes a pointing gesture

ICRA Mobile Manipulation Challenge, Anchorage, Alaska


Our robot Dynamaid participated in the ICRA Mobile Manipulation Challenge.
It fetched an object requested by the user by means of a pointing gesture.
The user could also request a drink by showing it to the robot.
Finally, Dynamaid collected an object from the table and moved it towards a trash bin. 

User requests an object using a pointing gesture Dynamaid grasps an object The user requests a drink by showing it to Dynamaid Dynamaid grasps an object Handing over the fetched object to the user  Dynamaid throws away an object.

RoboCup German Open 2010, Magdeburg


Our robots Robotinho and Dynamaid participated at German Open 2010.
Robotinho performed the registration. Dynamaid detected and reconized all pesons in the Who-is-Who test and interpreted pointing gestures correctly in the Open Challenge. Robotinho and Dynamaid cooperated in the Demo Challenge.
In the final, Robotinho greeted a guest and guided him to a table. There, the guest could select something to eat using a pointing gesture. Also, the guest ordered a drink by showing it to Dynamaid. Overall, our team reached the second place in the @Home league.  

Dynamaid in the Enhanced-Who-is-Who Test Robotinho and Dynamaid German Open 2010 @Home Award Ceremony

RoboCup 2009, Graz


Our robots Robotinho and Dynamaid participated for the first time at the RoboCup@Home international competitions.
They reached the highest score in the Introduce test. Dynamaid successfully performed the Follow-me and the Who-is-Who test. Both robots reached the second highest score in the Open Challenge, where Robotinho gave a home tour to a guest while Dynamaid delivered a drink. 

In Stage II, Dynamaid did the Walk&Talk test, the Supermarket test and the Party-bot test very well. Both robots were used in the Demo Challenge. After Stage II, our team had the second most points, almost on par with the leading team. 

In the final, Dynamaid detected persons and delivered drinks to them. 

Overall, our team reached the third place in the @Home competition. We won also the innovation award for "Innovative robot body design, Empathic behaviors, and Robot-Robot Cooperation".

RoboCup 2009 @Home Final: Dynamaid Grasping RoboCup 2009 @Home: Dynamaid Hand-over RoboCup 2009: Robotinho RoboCup 2009: Team NimbRo

      The Video is also available in WMV-Format.

RoboCup German Open 2009, Hannover

NimbRo participated for the first time in the @Home league at RoboCup German Open 2009 during Hannover Fair.

In Stage I, we used Robotinho for the introduce task. It explained itself and Dynamaid and interacted with a human in a natural way. For the follow me task, we used Dynamaid. She was able to quickly follow an unknown human though the arena, outside, and back into the arena. She also could be controlled by voice commands. Dynamaid also did the fetch and carry task very well. She took spoken orders from the human user and delivered reliably the requested object. 

In Stage II, Dynamaid did the walk and talk task perfectly. A human showed her places in the apartment that she could visit afterwards as requested by spoken commands. In the demo challenge, the users could order different drinks that she fetched quickly and reliably from different places in the apartment.

In the final, Robotinho gave a tour though the apartment while Dynamaid fetched a drink for a guest [video].

Overall, the NimbRo@Home team reached the second place, only a few points behind b-it-bots.

RoboCup German Open 2009 Final: Robotinho gives a home tour  RoboCup German Open 2009 Final: Dynamaid offers a drink  RoboCup German Open 2009: Dynamaid fetches a drink


   Prof. Dr. Sven Behnke
   Rheinische Friedrich-Wilhelms-Universität Bonn
   Institut für Informatik VI
   Friedrich-Ebert-Allee 144
   53113 Bonn 

Tel:  +49 (0) 228 73-4422
Fax: +49 (0) 228 73-4425 

Email: behnke _at_ cs.uni-bonn.de

Universität Bonn, Institute for Computer Science, Departments: I, II, III, IV, V, VI