DARPA has just announced the winning teams of their virtual robotics challenge. Each team had to develop software to enable a very advanced humanoid robot to perform disaster-response tasks such as 1) walking to a standard utility vehicle and entering it, 2) driving the vehicle using steering wheel and pedals, 3) walking on straight, uneven and obstacle-laden surfaces, 4) picking up a fire hose, connecting it and opening the valve. The winning teams will be provided with actual physical versions of the simulated robots and move on to the next level of the Darpa Robotics Challenge.
DARPA used the open source simulation environment Gazebo to create a virtual environment where the software of more than 100 competing teams could be tested. They have created a complete virtual world with a street, houses, obstacles etc. and developed a model of the robot which could then be placed into the virtual world and controlled by the team’s software. This approach has many advantages: Different scenarios can be tested, no physical damage can occur when the software does not behave as expected, the teams can use the simulation to optimize their algorithms etc.
A similar approach could be used to test driverless cars before they are released to the public. Official simulation environments would be needed with standard interfaces for the driverless car software. This would have to include standardized models which describe the behaviour of the car’s sensors (and translate the state of the simulation environment into sensor readings).
A standard simulation environment for driverless cars would enable a testing body to verify that a software-based driverless car behaves as expected in many difficult cases. It would allow quantitative claims about the performance of driverless car software and could be used to compare different driverless car systems. Driverless car companies could use such test environments to prove that their software has been properly and carefully developed.
Of course test cases would have to be challenging and representative of the actual scenarios which a driverless car will encounter. This requires significant effort. It is likely over time the number of test scenarios which a driverless car software will have to pass will increase. In the event of driverless car accidents, the testing agency could rapidly analyze the causal chain for the accident and then build additional simulation test cases which the updated driverless car software would have to pass.
As we have argued in our white paper three years ago, it is extremely important to develop a public common simulation platform for driverless cars now!