CARLA 0.9.12 Release

Large Maps, Physics improvements, Optical Flow Camera, structural improvements and additions to pedestrians and vehicles.

Posted by @corkyw10 on August 02, 2021

Get CARLA 0.9.12

The long-anticipated 0.9.12 Release of CARLA is finally here!! And it was worth the wait! Let’s have a peek at what the newest release brings.

Starting with the most exciting feature of this release, Large Maps! Users can now perform simulations on vast maps that can span upwards of 10,000km². You’ll be able to try it out straight away in the demo map of Town 11 that measures 400km². Use the updated Traffic Manager to populate the map with hundreds of cars that can drive at high speeds!

Next, we have lots of physics improvements, including full physics determinism in synchronous mode, improved wheel physics, custom vehicle wheel settings, and vehicle control telemetry.

We have some updates to the CARLA structure with an upgrade to Unreal Engine 4.26, deprecation of the OpenGL graphics API, new options to install the Python client library with pip, a rebrand of our classic script to populate scenes, and a new default town.

The blueprint library is expanding rapidly. We have a whole new generation of pedestrians and vehicles with improved meshes and details. There’s a lot of variety in our new pedestrians, who have a range of clothing, body shapes, and skin colors.

The sensor library has expanded too, with the addition of the Optical Flow Camera, allowing you to capture the motion of pixels between frames.

OpenStreetMaps functionality has been extended to allow users to automatically generate traffic lights at junctions and control how and where those traffic lights are generated. This feature uses both OpenStreetMap information and CARLA-generated information when the former isn’t available.

CARLA has integrated with Ray/RLlib to enable you to use CARLA as an environment in Ray for reinforcement learning experiments. You’ll be able to use this environment for training and inference purposes.

We present our beta integration with Chrono, a highly realistic multi-physics simulation engine. Our integration will allow you to delegate the simulation of advanced physics to Chrono, providing a more accurate simulation experience.

Finally, our CarSim integration is no longer in beta stages and is fully supported after making some fixes and updates to the functionality.

We hope you enjoy the new 0.9.12 release of CARLA. Read on below to find out more!

Large Maps

The large map feature is the most exciting of all in CARLA 0.9.12. Users can now perform simulations on a vast scale, with maps scaling over 10,000km², complete with realistic traffic simulation. This CARLA release comes with a demo large map called Town 11, measuring 400km², ready for users to explore straight away!

Large maps are constructed from a set of tiles of a given size (maximum 2km x 2km). Tiles are then streamed in and out of the simulation according to their distance from the ego vehicle (tagged with “hero”). The streaming distance is configurable through the Python API.

large_map

The ego vehicle not only controls the streaming of the tiles but also the state of other actors in the simulation. In large maps, actors can be active or dormant. It is important to note that dormant actors exist only within the context of large maps. Actors in the simulation become dormant when they are located outside of the area of influence of the ego vehicle and wake up when the ego vehicle is near them again. This distance is called the actor_active_distance and is configurable via the Python API. Traffic Manager actors are special because they can be respawned near the ego vehicle once they become dormant, or they can be set to hybrid mode, so they continue to teleport around the map while dormant.

Traffic manager has been adapted to populate the extensive road systems involved in large maps and has been upgraded to perform at high speeds, reacting realistically to curves, junctions, and other traffic while maintaining performance. Read more in the documentation.

Physics Determinism and Improvements

There have been several improvements in physics performance in the 0.9.12 release.

First up, collisions are fully deterministic when certain conditions are met. To achieve full determinism, the simulation should be synchronous from the beginning of every simulation, with fixed delta seconds defined. Commands should be issued in batch to avoid loss during heavy computation load. Read more about the requirements in the documentation.

determinism_crash

Several parameters have been exposed through the Python API that allow users fine-grained control over how physics affects vehicle wheels. These parameters include:

  • long_stiff_value: Tire longitudinal stiffness per unit gravitational acceleration.
  • lat_stiff_max_load: Maximum normalized tire load at which the tire can deliver no more lateral stiffness no matter how much extra load is applied to the tire.
  • lat_stiff_value: Maximum stiffness per unit of lateral slip.

Each of our vehicles now has custom values for these parameters, but you can tweak them in your code by following the example in the documentation!

For users simulating physics through another engine, we have exposed the set_wheel_steer_direction method on the Python API. This method controls the visual angle of a vehicle’s wheels, without affecting physics. Each wheel is targeted individually. To use this method, you must have physics disabled on the vehicle first:

vehicle.set_simulate_physics(False)
vehicle.set_wheel_steer_direction(carla.VehicleWheelLocation.FR_Wheel, 40.0)

A new telemetry view has been introduced to allow users to view physics and control metrics for a given actor in the server and simulation windows. This can be enabled simply through the Python API, or you can try it out in manual_control.py by pressing T. Read more in the documentation!

vehicle_telemetry

Finally, general simulation performance has been improved for actors whose physics has been disabled. This is most notable in the performance of hybrid mode in Traffic Manager.

Unreal Engine 4.26

We have upgraded to Unreal Engine 4.26.2, bringing some important changes to the graphics API and rendering pipeline. Vulkan is now the only supported Graphics API; support for OpenGL has been dropped completely.

The upgrade to the Unreal Engine version has made off-screen rendering easier than ever with the Vulkan API. To run CARLA in off-screen mode, simply use the -RenderOffScreen flag when you start a CARLA server:

./CarlaUE4.sh -RenderOffScreen

0.9.12 uses a fork of Unreal Engine 4.26 with custom changes specific to the CARLA simulator. Take this into account when building from source as this requires downloading our specific Unreal Engine repository!

Python Client Library

There are now several options for installing the Python client library. The options available depend on whether you are building from source or using a packaged version of CARLA. The options include:

A) .egg file - Available for source build and packaged version

.egg files are a continuation of the option available in previous releases. Nothing has changed with this option. The .egg file does not need to be installed. All of CARLA’s example scripts automatically look for this file when importing CARLA.

If you previously installed a CARLA .whl, the .whl will take precedence over an .egg file.

B) .whl file - Available for the source build and packaged version

The .whl file should be installed using pip or pip3, depending on your OS.

If you are using the source build version of CARLA, this .whl file cannot be distributed as it is built specifically for your OS during the build process.

C) Downloadable Python package - package version only

The CARLA client library can be downloaded from PyPi. This library is compatible with Python versions 2.7, 3.6, 3.7, and 3.8.

The PyPi download is suitable for use with CARLA packages only. Since the PyPi download only contains the client library, it is most useful in situations where you will be communicating with a remote CARLA server where you do not require downloading a full CARLA package.

This change gives a lot of flexibility to users, but also adds some more complexity to the installation process. Make sure you read the installation guides carefully before you install CARLA!

Optical Flow Ground Truth

The Optical Flow camera is a new sensor that allows users to collect ground truth by capturing motion perceived from the point of view of the camera. Every pixel recorded by the Optical Flow camera encodes the velocity of that point projected to the image plane. You can find the sensor reference material and additional attributes here.

optical_flow

New and Improved Vehicles

The CARLA garage has been busy with five new vehicles added to the CARLA blueprint library!

vehicles

Introducing from left to right:

  • Firetruck
  • Mini Cooper 2021
  • Nissan Patrol 2021
  • Ambulance
  • Mercedes Sprinter

As well as completely new vehicles, we have generally improved the vehicle materials and meshes. Dirt has been added to vehicle exteriors to increase realism. Mesh geometry has been increased to improve appearance.

Finally, some of our vehicle names have changed in the blueprint library. These changes include:

  • vehicle.bmw.isetta -> vehicle.micro.microlino
  • vehicle.dodge_charger.police -> vehicle.dodge.charger_police
  • vehicle.mercedes-benz.coupe -> vehicle.mercedes.coupe
  • vehicle.mini.cooperst -> vehicle.mini.cooper_s
  • vehicle.lincoln.mkz2017 -> vehicle.lincoln.mkz_2017
  • vehicle.mustang.mustang -> vehicle.ford.mustang
  • vehicle.lincoln2020.mkz2020 -> vehicle.lincoln.mkz_2020
  • vehicle.mercedesccc.mercedesccc -> vehicle.mercedes.coupe_2020
  • vehicle.charger2020.charger2020 -> vehicle.dodge.charger_2020
  • vehicle.chargercop2020.chargercop2020 -> vehicle.dodge.charger_police_2020
  • vehicle.vespa.vespa -> vehicle.vespa.zx125

New and Improved Pedestrians

We welcome a diverse new set of pedestrians to the CARLA family. A new generation (Gen 2) of pedestrians has been added to the blueprint library. The new pedestrians have a variety of body shapes and skin colors. Pedestrians all over the board have been improved with new shaders for skin, eyes, and clothing that add microdetail information. Meshes have been improved with a higher polygon count and better topology.

pedestrians

Actors in the blueprint library now have an additional generation attribute to distinguish between old and new models when you spawn them. You can also set some flags in generate_traffic.py and manual_control.py that defines which generations you would like to spawn:

# In generate_traffic.py
# Generate first generation vehicles
--generationv 1

# Generate second generation walkers
--generationw 2

# In manual_control.py
# Generate all generations
--generation All

OpenStreetMaps

Users can now enable the automatic generation of traffic lights on junctions in OpenStreetMap maps through the Python API. Doing so is easy and only requires setting the generate_traffic_lights parameter to True in your code:

# Define the settings.
settings = carla.Osm2OdrSettings()
# Enable traffic light generation from OSM data
settings.generate_traffic_lights = True

Most of the traffic light data will be extracted from OpenStreetMap metadata, but in some areas this information is sparse or unavailable. To work with this limitation, you can enable all junctions to be controlled with traffic lights:

settings.all_junctions_with_traffic_lights = True

To give you even more freedom, you can also disable certain kinds of roads from generating traffic lights, for example, motorway links:

settings.set_traffic_light_excluded_way_types(["motorway_link"])

A full list of the types of roads you can filter by is found in the Python API.

RLlib Integration

CARLA has integrated with the Ray/RLlib library. Ray is an open-source framework that provides a simple, universal API for building distributed applications. Ray is packaged with RLlib, a scalable reinforcement learning library, and Tune, a scalable hyperparameter tuning library. This integration allows for the easy use of CARLA as an environment in Ray for reinforcement learning training and inference purposes. It opens up a whole world of potential for you to explore and build upon your AD agents.

In the documentation, we provide a full guide that shows how users can create and use CARLA as an environment of Ray and use that environment for training and inference purposes. There is an example experiment and you will find instructions that allow you to use the experiment code both locally and in the cloud using AWS.

Chrono Integration (Beta)

CARLA has integrated with Project Chrono, an open-source, multi-physics simulation engine that provides highly realistic vehicle dynamics using a template-based approach. This integration allows users to utilize Chrono templates to simulate vehicle dynamics while navigating a map.

Chrono is ready to use on CARLA packages but the source build needs to be compiled with the flag ARGS="--chrono". You can find out more in the documentation.

It is important to note that physics will revert to CARLA control if a Chrono vehicle encounters a collision.

CarSim Fully Integrated

CarSim is a simulation tool from Mechanical Simulation that performs highly accurate simulation of the dynamic behavior of passenger vehicles and light-duty trucks. Using a 3D multi-body dynamics model, it reproduces the physics of a vehicle in response to different controls (steering, throttle, braking, gear shifting) from a driver and/or automatic agent.

The CarSim integration enables co-simulation between CarSim and CARLA, allowing users to control dozens of physics parameters, including suspension system and tires via CarSim and simulate them in CARLA.

After some fixes to the general functionality and adding options such as being able to choose your own vehicle, this integration is no longer in the beta stages and is now fully supported!

spawn_npc.py Rename

The infamous spawn_npc.py script has been renamed as generate_traffic.py. Along with the renaming, there is an additional change that users should be aware of. The script now works in synchronous mode by default. This is to ensure that Traffic Manager performs at maximum quality and deterministically. If you require the script to work in asynchronous mode, you can add the -async flag when you run it.

New Default Town

When you run your first simulation on 0.9.12 you may notice something different. Our new default map is the beautiful HD Town 10! Enjoy the spectacular views from our most complex and high-quality map!

town_ten

Contributors

This space is dedicated to all of those whose contributions were merged in any of the project’s GitHub repositories during the development of CARLA 0.9.12.

For the Optical Flow Camera, we thank the team at RPG Lab at the University of Zurich!

  • Shashank Singh
  • Florian Mahlknecht
  • Mathias Gehrig
  • Javier Hidalgo
  • Davide Scaramuzza

And to our GitHub contributors, a big thank you!

Changelog

  • DEPRECATED: OpenGL graphics API is deprecated. UE4.26 uses Vulkan only.
  • Changed the resolution of the cached map in Traffic Manager from 0.1 to 5 meters
  • CARLA now is built with Visual Studio 2019 in Windows
  • Upgrading to Unreal Engine 4.26
  • Added generation attribute to vehicles and pedestrians
  • Enabling the no_delay option to RPC and stream sockets
  • The special nomenclature to define roads (ROAD_ROAD), sidewalks (ROAD_SIDEWALK)… can be at any position of the asset name
  • Improved performance benchmark script: sync, map and sensor selection, …
  • Improved performance, destroyed PhysX state for vehicles when physics are disable
  • Improved parallelism of raycast sensors in system with large number of cores
  • Added ‘visualize_multiple_sensors’ example
  • Added ‘check_lidar_bb’ util script
  • Added optional flag to client.replay_file() replay_sensors to enable or disable the replaying the sensors
  • Improved manual_control: now cameras are set in relation with car size
  • Client required files are sent from the server to a local cache (OpenDRIVE, Traffic Manager…)
  • Added CHRONO library for vehicle dynamics simulation (https://projectchrono.org/)
    • Supported JSON vehicle definition
    • Unsupported collision dynamics
  • Added Lincoln 2020 vehicle dimensions for CarSim integration
  • Added performance benchmarking section to documentation
  • Added API functions to traffic light actor get_effect_waypoints(), get_light_boxes() and get_opendrive_id()
  • Added API functions to world get_traffic_lights_from_waypoint(), get_traffic_lights_in_junction and get_traffic_light_from_opendrive_id
  • Added API parameters to WorldSettings: tile_stream_distance and actor_active_distance
  • Added API parameters and functions to Osm2OdrSettings: proj_string, center_map, generate_traffic_lights, all_junctions_with_traffic_lights, set_osm_way_types, and set_traffic_light_excluded_way_types
  • Added API function to enable car telemetry
  • CARLA is compatible with the last RoadRunner nomenclature for road assets
  • Extended make import process for applying road painter materials (carla art tool)
  • Added Large Map feature to CARLA, alowing to import large maps divided in square tiles (at most 2kmx2km per tile). Only section of a Large Map can be loaded at a time which introduces a sleep state for actors that are far away from the ego vehicle
  • Added creation of custom JSON file for applying decals to imported roads
  • Added ApplyVehiclePhysicsControl to commands
  • Added flush in the sublevel loading to increase carla’s determinism in Opt maps
  • Exposed tire parameters for longitudinal and lateral stiffness in the PhysicsControl.
  • When setting a global plan at the LocalPlanner, it is now optional to stop the automatic fill of the waypoint buffer
  • Improved agent’s vehicle detection to also take into account the actor bounding boxes
  • Added Optical Flow camera
  • API extensions:
    • Added set_wheel_steer_direction() function to change the bone angle of each wheel of a vehicle
    • Added get_wheel_steer_angle() function to get the steer angle of a vehicle wheel
    • Added scattering_intensity , mie_scattering_scale , rayleigh_scattering_scale to PythonAPI for changing weather attributes
  • Improved the python agents API. Old behaviors have been improved and new ones have also been added, improving the functionalities of the agents. Several bugs have also been fixed

Fixes

  • Fixed import sumo_integration module from other scripts
  • Fixed bug causing the RoadOptions at the BehaviorAgent to not work as intended
  • Fixed a bug when importing a FBX map with some _ in the FBX name
  • Fix bug in carla.Transform.get_up_vector()
  • Fix bug in lidar channel point count
  • Fix imu: some weird cases were given nan values
  • Fix bugs in apply_physics_control and friction trigger

← Previous Post Next Post →