CARLA 0.9.0: Multi-client multi-agent support

Posted by @nsubiron on July 30, 2018

We are very proud to announce CARLA 0.9.0!

In this release we have completely redesigned the networking architecture and built from scratch a new Python API that completely changes the way CARLA is used. Every vehicle can now be controlled, cameras can be placed anywhere, and any number of client applications can connect simultaneously to the same instance of the simulator. But beware, this is a development release. The amount of changes we are doing is enormous, so we wanted to make available some of the latest features for our brave users that want to start experimenting with the new CARLA API. Many things that were there will be temporarily unavailable and many new issues will appear, but we believe all these changes are worth it and put us straight on our road map for creating a great autonomous driving simulator for everyone!

Get CARLA 0.9.0

Disclaimer said, let’s talk a bit about the cool features that this release brings.

Multi-client

The server-client communication has been redesign to allow any number of clients connecting and making changes to the simulation concurrently. It is finally possible to have a multi-player simulation with different AIs and humans driving around in the same city.

We have switched our networking protocol to an RPC model, using the great rpclib library. The new model uses two TCP ports, by default 2000 and 2001, the first one used to control the simulation with rpclib, the second one does the streaming of the sensor data. Both client and server work asynchronously, and although many optimizations are yet to be made, the latency is already pretty low.

The multi-client support opens the door to very interesting scenarios and integrations with third-party software like traffic control tools and pedestrian behaviour modelling libraries. It’s great for scalability as each of the clients can be run on a different node.

Python API

The final goal of the new API is to allow users to control every aspect of the simulation. Our view is to have everything available from Python; loading maps, adding sensors, adding static obstacles, controlling vehicles and pedestrians, and what not. The 0.9.0 API is just a step forward on that direction, but there are a bunch of cool things that you can already do with it.

But first we need to introduce a few core concepts:

  • Actor: Actor is anything that plays a role in the simulation and can be moved around, examples of actors are vehicles, pedestrians, and sensors.
  • Blueprint: Before spawning an actor you need to specify its attributes, and that’s what blueprints are for. We provide a blueprint library with the definitions of all the actors available for you to choose.
  • World: The world represents the currently loaded map and contains the functions for converting a blueprint into a living actor moving around in the simulation.

All the actors can be created and destroyed at any point during the simulation, and all of them can be controlled or moved around. In this release only vehicles and cameras are available but in coming releases we’ll add pedestrians and traffic lights as well.

Let’s take a quick peek at how it looks, the following code adds a red Mustang to the simulation and enables its autopilot so the car can drive freely around the city on its own

from carla import *

client = Client('localhost', 2000)

world = client.get_world()
blueprint = world.get_blueprint_library().find('ford.mustang')
blueprint.set_attribute('color', '255,0,0')

transform = Transform(Location(x=230, y=195, z=40), Rotation(yaw=180))

mustang = world.spawn_actor(blueprint, transform)
mustang.set_autopilot()

Now let’s attach a camera to this vehicle so we can take a look at what’s going on

blueprint = world.get_blueprint_library().find('sensor.camera')
blueprint.set_attribute('post_processing', 'SceneFinal')
camera = world.spawn_actor(
    blueprint,
    Transform(Location(x=0.5, z=1.8)),
    attach_to=vehicle)

camera.listen(lambda image: do_something(image))

the callback function passed to the listen method will be called every time a new image is received. This callback is executed asynchronously, this considerably speeds up the parsing of images. Now it is possible to do things like saving images to disk keeping up with the speed of the simulator.

Actors can also be teleported and moved around the city, for that we have exposed the set_location and set_transform methods

location = vehicle.get_location()
location.x += 10.0
vehicle.set_location(location)

and in the special case of vehicles, they can be controlled too in the usual way

vehicle.apply_control(VehicleControl(throttle=1.0, steer=-1.0))

For the full list of methods available take a look at the Python API Reference.

We’ll revisiting and improving this API in the coming releases. If you find any issues or think of cool improvements that can be added, please don’t hesitate to share it with the community at our GitHub or Discord chat.

Things missing

Many features that are present in previous releases are gone in this one. Things like Lidar, pedestrians, bikes, synchronous mode, or quality settings. This is due to the fact that we have switched to a new API and many things haven’t been ported yet. We’ll be adding them soon in the coming releases. As said before, this is a work in progress, we want to share this progress with the community, after all we think this can be already useful for a lot of people.

Full list of changes

  • Upgraded to Unreal Engine 4.19
  • Redesign of the networking architecture
    • Allows any number of clients to connect simultaneously
    • Now is possible to add and remove at any time any vehicle or camera
    • Now is possible to control any vehicle or camera
    • Now is possible to place cameras anywhere
    • Reduced to two ports instead of three
    • First port uses an RPC protocol based on rpclib
    • Second port is for the streaming of the sensor data
  • Redesign of the Python API
    • Actors and sensors are now exposed in the API and can be independently controlled
    • The Python module is built in C++, with significant performance gain in some operations
    • Many functionality haven’t been ported yet, so expect a lot of things missing
  • Redesign of the build system to accommodate the changes in dependencies
    • Everything can be done now with the Makefile
    • For the moment only Linux is supported, sorry
  • Massive clean up of all unused assets
  • Some aesthetic fixes to the vehicles