CARLA 0.9.13 Release

Get CARLA 0.9.13

The 0.9.13 release of CARLA is finally here, so read on to see how the new improvements will revolutionize your workflow.

CARLA now features ground truth instance segmentation! A new camera sensor can be used to output images with the semantic tag and a unique ID for each world object. Now the ground truth ID is available to differentiate objects of the same type in the camera field of view. This functionality can be used for many different purposes such as training and evaluating networks to differentiate and track overlapping objects.

Runtime texture streaming is now possible through the CARLA API. This new functionality enables the user to change the textures of every object in the scene during runtime without requiring the Unreal Editor. Produce high quality training data to improve generalization, avoid overfitting pitfalls and challenge neural networks with adversarial attacks through continuously updating textures.

CARLA’s API now provides new functions to retrieve ground truth pedestrian 3D bone positions and hence evaluate the pedestrian’s 3D pose, facilitating the validation and training of pose estimation models and also adding custom gestures to standard animations through the API.

CARLA 0.9.13 presents new assets to enrich your simulations with a selection of additional pedestrians and vehicles. 4 new pedestrians are introduced including 2 adults (each with 3 variants) and 2 children to add extra variety to the humans occupying the pavements and roads in your simulations. 2 new vehicles have also been added to the CARLA garage, the Volkswagen T2 2021 and a new Ford Crown Taxi. Generation 2 (GEN-2) cars now include articulated doors that can interact with the environment through collisions. Articulating doors are especially useful for training and testing autonomous vehicles in car parks or on tight city streets.

This release comes with major improvements to the Traffic Manager, with new logic to promote more realistic vehicle behavior at intersections and at high speeds. The API now offers new functionality to guide vehicles with a user defined route or path and vehicle lights now react to weather conditions and junctions. Vehicle physics has been enhanced with more accurate models of acceleration, gear and braking behavior.

Instance Segmentation

CARLA now has a brand new instance segmentation sensor that outputs images containing a different ID for every object. Instance semantic IDs are now available embedded in the G and B channels of the RGB output of the sensor data, alongside the standard semantic IDs in the R channel. The new functionality provides access to the ground truth identity of objects traversing the sensor field of view, enabling training, improvement, validation and evaluation of neural networks. This new functionality brings a whole new world of possibilities for training machine learning algorithms to differentiate overlapping objects in scenes.

Runtime Texture Streaming

The API now exposes functionality to update textures during runtime, allowing texture modification without relying on the UE4 editor. Users can now update and modify textures programmatically through the API, allowing texture updating to occur, for example, during the execution of a neural network training script. This can be useful, for example, when training and validating neural networks to avoid overfitting pitfalls. Continuously updating textures can be used to challenge neural networks with adversarial attacks.

# Get names of all available objects

object_names = world.get_names_of_all_objects()
    for name in object_names:
        print(name)

# Choose an object to modify
# For example target_object could be 'SM_Cartel_Add_5'

target_object = random.choice(object_names)
print('Altering texture for object: ' + target_object)

# Modify its texture

texture = carla.TextureColor(width,height)
for x in range(0,len(image[0])):
    for y in range(0,len(image)):
        color = image[y][x]
        r = int(color[0])
        g = int(color[1])
        b = int(color[2])
        a = int(color[3])
        texture.set(x, height - y - 1, carla.Color(r,g,b,a))
world.apply_color_texture_to_object(target_object, carla.MaterialParameter.Diffuse, texture, 0)

3D Ground Truth Skeletons for Pedestrians

The CARLA API now provides functionality to retrieve the ground truth 3D bone positions of pedestrians in the simulation. This opens up a wealth of possibilities for training and evaluation of human pose estimation models. A CARLA vehicle can now record the bone positions of nearby pedestrians to complement its sensor data with access to the ground truth 3D pose for comparison with model-estimated poses. This new functionality also allows easy augmentation of pedestrian behavior with custom gestures blended into the default animations like waving or pointing.

The following animation shows a custom animation handled by the CARLA API retrieving the bones then blending between the neutral pose and a custom walking animation handled through the API:

The following code spawns two pedestrians from the blueprint library and then retrieves the 3D skeletons and alters the arms’ pose:

# create several pedestrians

blueprints = world.get_blueprint_library().filter("walker.pedestrian.*")
pedestrians = []
pedestrians.append(world.spawn_actor(random.choice(blueprints), world.get_random_location_from_navigation()))
pedestrians.append(world.spawn_actor(random.choice(blueprints), world.get_random_location_from_navigation()))
...

# get the 3d bones from all pedestrians

for ped in pedestrians:
  bones = ped.get_bones()

  # modify some bones

  new_pose = []
  for bone in bones.bone_transforms:
      if bone.name == "crl_foreArm__L":
          bone.relative.rotation.pitch -= 90
          new_pose.append((bone.name, bone.relative))
      elif bone.name == "crl_foreArm__R":
          bone.relative.rotation.pitch -= 90
          new_pose.append((bone.name, bone.relative))

  # set the new pose

  control = carla.WalkerBoneControlIn()
  control.bone_transforms = new_pose
  ped.set_bones(control)

  # blend the pose

  actor.blend_pose(0.5)

New CARLA assets

New Pedestrians

4 new pedestrians have been added to the CARLA assets. 2 adults (each with 3 variants) and 2 children.

New Vehicles

The CARLA garage has built two exciting new vehicles, a high fidelity version of the Volkswagen T2 2021 and the Ford Crown Taxi. Add some charm to your city traffic with these beautiful new vehicles.

Articulated Doors

All Generation 2 (GEN-2) vehicles now have articulated doors that can be opened and closed through functions in the API. This functionality is ideal for simulating scenarios in busy city streets and car parks where doors may open unpredictably while driving past or might be blocking access to parking spaces or passageways.

Improved Traffic Manager Logic

The Traffic Manager has undergone a major update with enhanced logic for handling vehicle behavior. Traffic now behaves in a more realistic manner at intersections with vehicles driving fluidly at high speeds. Vehicle turning is now smoother under highway conditions. Collision detection and braking is enhanced to promote more realistic rapid traffic behavior. Altogether the Traffic Manager improvements in this CARLA update yield far more realistic NPC traffic.

User Defined Paths in Traffic Manager

The Traffic Manager now also includes some great new functionality to augment user control over the behavior of vehicles. The API exposes new Traffic Manager functions that can be used to guide vehicles with a user-defined custom path.

New functions:

• get_next_action: gets the next action that a vehicle will take
• get_all_actions: enables the user to query all possible actions a vehicle has available
• set_path: allows a custom route to be defined using coordinates defined as CARLA Locations
• set_route: allows a custom route to be defined using route commands like left, right or straight

path = [carla.Location(x=-506.696198, y=179.384308, z=0.038194),
                    carla.Location(x=-504.745972, y=232.868927, z=0.039417)]
traffic_manager.set_path(vehicle, path)

The set_route function allows a custom route to be defined using route commands:

route = ["Right", "Straight", "Right", "Right", "Left", "Right"]
 traffic_manager.set_route(vehicle, route)

Automatic vehicle lights for NPCs

New functionality grants the Traffic Manager control over vehicle lights. Headlights, fog lights and blinkers are now controlled according to vehicle route data and weather conditions. Blinkers activate according to the next planned turn in the route, braking lights will activate when vehicle brakes are applied. Fog lights and main beams are activated according to the weather and light conditions. During night, vehicles’ main beams are activated and in foggy conditions rear fog lights are activated.

Improved Vehicle Behavior

Vehicle physics have been improved with better modelled acceleration, braking and gear performance. Vehicle behavior has been carefully remodelled to better capture the real characteristics of the CARLA garage vehicles. Much care has been taken to reproduce published performance data on acceleration and braking as accurately as possible. Similarly the gear change behavior has been altered to better reflect real world driving.

Contributors

Here we want to acknowledge all the contributors who committed work to CARLA 0.9.13. Thank you all for your hard work!

• Anshu-man567 - fixing the manual_control_steeringwheel.py script
• simonmcm1 - fixing the import process for props without a map
• barasm-hita - spawn points extraction and OSM2ODR conversion
• amparore - fixed a bug in CARLA playback during collisions
• regaloLiuk - Traffic Manager vehicle lights
• gouchaoer - updates to semantic segmentation tutorials
• Cory Cornelius - Instance Segmentation

Changelog

• Added the option for users to set a path using map coordinates to a vehicle controlled by the Traffic Manager.
• Added a RoadOption element in each SimpleWaypoint to specify which action will the vehicle perform if it follows that route.
• Added new instance aware semantic segmentation sensor sensor.camera.instance_segmentation
• Added new API classes: MaterialParameter, TextureColor and TextureFloatColor to encode texture data and field (normal map, diffuse, etc)
• Added new API functions: apply_color_texture_to_object, apply_float_color_texture_to_object and apply_textures_to_object to paint objects in runtime
• Added set_percentage_random_left_lanechange and set_percentage_random_right_lanechange.
• Updated handling of collisions in Traffic Manager when driving at very high speeds.
• Added open/close doors feature for vehicles.
• Added API functions to 3D vectors: squared_length, length, make_unit_vector, dot, dot_2d, distance, distance_2d, distance_squared, distance_squared_2d, - get_vector_angle
• Added a seed for better reproducibility of pedestrians
• Added new API function set_pedestrians_seed
• Added new parameter –seedw in generate_traffic.py script
• Added API functions to 2D vectors: squared_length, length, make_unit_vector
• Added missing dependency libomp5 to Release.Dockerfile
• Added API functions to interact with pedestrian bones:
• get_bones / set_bones: to get/set the bones of a pedestrian
• blend_pose: to blend a custom pose with current animation
• show_pose / hide_pose: to show or hide the custom pose
• get_pose_from_animation: to set the custom pose with the animation current frame
• Added physical simulation to vehicle doors, capable of opening and closing
• Improved collision detection of the Python agents
• Added the new VehicleLightStage to the Traffic Manager to dynamically update the vehicle lights.
• Added two new examples to PythonAPI/util: Conversion of OpenStreetMaps to OpenDRIVE maps osm_to_xodr.py and Extraction of map spawn points extract_spawn_points.py

Fixes

• Fixed keep_right_rule parameter
• Fixed RSSSensor: made client side calculations threaded
• Fixed the import of props without any map
• Fixed global route planner crash when being used at maps without lane markings
• Fixed bug causing the server to sigsegv when a vehicle collides an environment object in recording mode
• Fixed cache now has an extra folder with current version of CARLA (so different cache per version)