The 0.9.13 release of CARLA is finally here, so read on to see how the new improvements will revolutionize your workflow.
CARLA now features ground truth instance segmentation! A new camera sensor can be used to output images with the semantic tag and a unique ID for each world object. Now the ground truth ID is available to differentiate objects of the same type in the camera field of view. This functionality can be used for many different purposes such as training and evaluating networks to differentiate and track overlapping objects.
Runtime texture streaming is now possible through the CARLA API. This new functionality enables the user to change the textures of every object in the scene during runtime without requiring the Unreal Editor. Produce high quality training data to improve generalization, avoid overfitting pitfalls and challenge neural networks with adversarial attacks through continuously updating textures.
CARLA’s API now provides new functions to retrieve ground truth pedestrian 3D bone positions and hence evaluate the pedestrian’s 3D pose, facilitating the validation and training of pose estimation models and also adding custom gestures to standard animations through the API.
CARLA 0.9.13 presents new assets to enrich your simulations with a selection of additional pedestrians and vehicles. 4 new pedestrians are introduced including 2 adults (each with 3 variants) and 2 children to add extra variety to the humans occupying the pavements and roads in your simulations. 2 new vehicles have also been added to the CARLA garage, the Volkswagen T2 2021 and a new Ford Crown Taxi. Generation 2 (GEN-2) cars now include articulated doors that can interact with the environment through collisions. Articulating doors are especially useful for training and testing autonomous vehicles in car parks or on tight city streets.
This release comes with major improvements to the Traffic Manager, with new logic to promote more realistic vehicle behavior at intersections and at high speeds. The API now offers new functionality to guide vehicles with a user defined route or path and vehicle lights now react to weather conditions and junctions. Vehicle physics has been enhanced with more accurate models of acceleration, gear and braking behavior.
CARLA now has a brand new instance segmentation sensor that outputs images containing a different ID for every object. Instance semantic IDs are now available embedded in the G and B channels of the RGB output of the sensor data, alongside the standard semantic IDs in the R channel. The new functionality provides access to the ground truth identity of objects traversing the sensor field of view, enabling training, improvement, validation and evaluation of neural networks. This new functionality brings a whole new world of possibilities for training machine learning algorithms to differentiate overlapping objects in scenes.
The API now exposes functionality to update textures during runtime, allowing texture modification without relying on the UE4 editor. Users can now update and modify textures programmatically through the API, allowing texture updating to occur, for example, during the execution of a neural network training script. This can be useful, for example, when training and validating neural networks to avoid overfitting pitfalls. Continuously updating textures can be used to challenge neural networks with adversarial attacks.
# Get names of all available objects object_names = world.get_names_of_all_objects() for name in object_names: print(name) # Choose an object to modify # For example target_object could be 'SM_Cartel_Add_5' target_object = random.choice(object_names) print('Altering texture for object: ' + target_object) # Modify its texture texture = carla.TextureColor(width,height) for x in range(0,len(image)): for y in range(0,len(image)): color = image[y][x] r = int(color) g = int(color) b = int(color) a = int(color) texture.set(x, height - y - 1, carla.Color(r,g,b,a)) world.apply_color_texture_to_object(target_object, carla.MaterialParameter.Diffuse, texture, 0)
The CARLA API now provides functionality to retrieve the ground truth 3D bone positions of pedestrians in the simulation. This opens up a wealth of possibilities for training and evaluation of human pose estimation models. A CARLA vehicle can now record the bone positions of nearby pedestrians to complement its sensor data with access to the ground truth 3D pose for comparison with model-estimated poses. This new functionality also allows easy augmentation of pedestrian behavior with custom gestures blended into the default animations like waving or pointing.
The following animation shows a custom animation handled by the CARLA API retrieving the bones then blending between the neutral pose and a custom walking animation handled through the API:
The following code spawns two pedestrians from the blueprint library and then retrieves the 3D skeletons and alters the arms’ pose:
# create several pedestrians blueprints = world.get_blueprint_library().filter("walker.pedestrian.*") pedestrians =  pedestrians.append(world.spawn_actor(random.choice(blueprints), world.get_random_location_from_navigation())) pedestrians.append(world.spawn_actor(random.choice(blueprints), world.get_random_location_from_navigation())) ... # get the 3d bones from all pedestrians for ped in pedestrians: bones = ped.get_bones() # modify some bones new_pose =  for bone in bones.bone_transforms: if bone.name == "crl_foreArm__L": bone.relative.rotation.pitch -= 90 new_pose.append((bone.name, bone.relative)) elif bone.name == "crl_foreArm__R": bone.relative.rotation.pitch -= 90 new_pose.append((bone.name, bone.relative)) # set the new pose control = carla.WalkerBoneControlIn() control.bone_transforms = new_pose ped.set_bones(control) # blend the pose actor.blend_pose(0.5)
4 new pedestrians have been added to the CARLA assets. 2 adults (each with 3 variants) and 2 children.
The CARLA garage has built two exciting new vehicles, a high fidelity version of the Volkswagen T2 2021 and the Ford Crown Taxi. Add some charm to your city traffic with these beautiful new vehicles.
All Generation 2 (GEN-2) vehicles now have articulated doors that can be opened and closed through functions in the API. This functionality is ideal for simulating scenarios in busy city streets and car parks where doors may open unpredictably while driving past or might be blocking access to parking spaces or passageways.
The Traffic Manager has undergone a major update with enhanced logic for handling vehicle behavior. Traffic now behaves in a more realistic manner at intersections with vehicles driving fluidly at high speeds. Vehicle turning is now smoother under highway conditions. Collision detection and braking is enhanced to promote more realistic rapid traffic behavior. Altogether the Traffic Manager improvements in this CARLA update yield far more realistic NPC traffic.
The Traffic Manager now also includes some great new functionality to augment user control over the behavior of vehicles. The API exposes new Traffic Manager functions that can be used to guide vehicles with a user-defined custom path.
get_next_action: gets the next action that a vehicle will take
get_all_actions: enables the user to query all possible actions a vehicle has available
set_path: allows a custom route to be defined using coordinates defined as CARLA Locations
set_route: allows a custom route to be defined using route commands like
path = [carla.Location(x=-506.696198, y=179.384308, z=0.038194), carla.Location(x=-504.745972, y=232.868927, z=0.039417)] traffic_manager.set_path(vehicle, path)
set_route function allows a custom route to be defined using route commands:
route = ["Right", "Straight", "Right", "Right", "Left", "Right"] traffic_manager.set_route(vehicle, route)
New functionality grants the Traffic Manager control over vehicle lights. Headlights, fog lights and blinkers are now controlled according to vehicle route data and weather conditions. Blinkers activate according to the next planned turn in the route, braking lights will activate when vehicle brakes are applied. Fog lights and main beams are activated according to the weather and light conditions. During night, vehicles’ main beams are activated and in foggy conditions rear fog lights are activated.
Vehicle physics have been improved with better modelled acceleration, braking and gear performance. Vehicle behavior has been carefully remodelled to better capture the real characteristics of the CARLA garage vehicles. Much care has been taken to reproduce published performance data on acceleration and braking as accurately as possible. Similarly the gear change behavior has been altered to better reflect real world driving.
Here we want to acknowledge all the contributors who committed work to CARLA 0.9.13. Thank you all for your hard work!