CARLA 0.9.6 release

We are proud to announce the new features included in CARLA 0.9.6!

This release brings back long-requested features, such as automatic pedestrian navigation (AI-controlled), better visual quality and a new skeleton control API, among other improvements.

Let’s take a detailed look at what is new!

Behold the new pedestrian AI!

With this new feature users can control and define the path that pedestrians will follow automatically, which can be controlled using CARLA API!

There are a few types of pedestrians, male, female, both in kid and adult forms. Also, they can spawn at random points over sidewalks and walk over them.

Currently, it is not possible for pedestrians to automatically cross the road using the AI system, but it will be soon!

pedestrian_ai

A new controller class has been created to manage the pedestrians (walkers) automatically.

carla.WalkerAIController:
  start()
  stop()
  go_to_location(destination)
  set_max_speed(speed)

To control a walker, it is just needed to spawn a controller.ai.walker and attach it to a walker actor, then initialize the controller and set its parameters.

blueprintsWalkers = random.choice(world.get_blueprint_library().filter("walker.*"))
walker_controller_bp = world.get_blueprint_library().find('controller.ai.walker')

walker_actor = world.spawn_actor(walker_bp, spawn_point)
walker_controller_actor = world.spawn_actor(walker_controller_bp, carla.Transform(), walker_actor)

world.wait_for_tick()

walker_controller_actor.start()
walker_controller_actor.go_to_location(world.get_random_location_from_navigation())
walker_controller_actor.set_max_speed(1.4)

#...

Code example 1. Spawning a pedestrian and setting its destination.

Once the simulation has started, pedestrians will walk to their destination and come back to the origin point continually.

You can test it out with the updated script spawn_npc.py which has the option to spawn pedestrians with the flag -w.

> python spawn_npc.py -w 20

Check it out in our documentation!

Improved Pedestrians

In CARLA we are growing our family!

New pedestrians are joining our library of assets. In addition to improving the looks of our previous citizens we are working on our future generations, boys and girls will populate our streets.

new_pedestrians

Figure 1. Improved pedestrian.

new_kids

Figure 2. New kids.

Last, we are working on pedestrian variations, we’ve planned different haircuts and clothing for each. Also, each piece of cloth will have different patterns and fabrics, new fashion trends and lots of styles.

pedestrian_cloth_variations

Figure 3. Cloth variations.

pedestrian_skeleton

Figure 4. Pedestrian cloth variations.

New skeleton control interface

Now all pedestrian actors have the same standard skeleton hierarchy, the same animation clips will behave similarly and fit them all.

pedestrian_skeleton

Figure 5. Pedestrian skeleton dance.

We have extended the API to allow users to take control of all the parts of a walker skeleton.

crl_root
└── crl_hips__C
    ├── crl_spine__C
    │   └── crl_spine01__C
    │       ├── ctrl_shoulder__L
    │       │   └── crl_arm__L
    │       │       └── crl_foreArm__L
    │       │           └── crl_hand__L
    │       │               ├── crl_handThumb__L
    │       │               │   └── crl_handThumb01__L
    │       │               │       └── crl_handThumb02__L
    │       │               │           └── crl_handThumbEnd__L
    │       │               ├── crl_handIndex__L
    │       │               │   └── crl_handIndex01__L
    │       │               │       └── crl_handIndex02__L
    │       │               │           └── crl_handIndexEnd__L
    │       │               ├── crl_handMiddle_L
    │       │               │   └── crl_handMiddle01__L
    │       │               │       └── crl_handMiddle02__L
    │       │               │           └── crl_handMiddleEnd__L
    │       │               ├── crl_handRing_L
    │       │               │   └── crl_handRing01__L
    │       │               │       └── crl_handRing02__L
    │       │               │           └── crl_handRingEnd__L
    │       │               └── crl_handPinky_L
    │       │                   └── crl_handPinky01__L
    │       │                       └── crl_handPinky02__L
    │       │                           └── crl_handPinkyEnd__L
    │       ├── crl_neck__C
    │       │   └── crl_Head__C
    │       │       ├── crl_eye__L
    │       │       └── crl_eye__R
    │       └── crl_shoulder__R
    │           └── crl_arm__R
    │               └── crl_foreArm__R
    │                   └── crl_hand__R
    │                       ├── crl_handThumb__R
    │                       │   └── crl_handThumb01__R
    │                       │       └── crl_handThumb02__R
    │                       │           └── crl_handThumbEnd__R
    │                       ├── crl_handIndex__R
    │                       │   └── crl_handIndex01__R
    │                       │       └── crl_handIndex02__R
    │                       │           └── crl_handIndexEnd__R
    │                       ├── crl_handMiddle_R
    │                       │   └── crl_handMiddle01__R
    │                       │       └── crl_handMiddle02__R
    │                       │           └── crl_handMiddleEnd__R
    │                       ├── crl_handRing_R
    │                       │   └── crl_handRing01__R
    │                       │       └── crl_handRing02__R
    │                       │           └── crl_handRingEnd__R
    │                       └── crl_handPinky_R
    │                           └── crl_handPinky01__R
    │                               └── crl_handPinky02__R
    │                                   └── crl_handPinkyEnd__R
    ├── crl_thigh__L
    │   └── crl_leg__L
    │       └── crl_foot__L
    │           └── crl_toe__L
    │               └── crl_toeEnd__L
    └── crl_thigh__R
        └── crl_leg__R
            └── crl_foot__R
                └── crl_toe__R
                    └── crl_toeEnd__R

Skeleton hierarchy.

A walker’s skeleton can be modified by using an instance of the WalkerBoneControl class. This class contains the transforms of the bones to be modified (as a tuple).

control = carla.WalkerBoneControl()
first_tuple = ('crl_hand__R', carla.Transform(rotation=carla.Rotation(roll=90)))
second_tuple = ('crl_hand__L', carla.Transform(rotation=carla.Rotation(roll=90)))
control.bone_transforms = [first_tuple, second_tuple]

Changes to a walker can be applied as:

walker_actor.apply_control(control)

The location and rotation of each transform are relative to their parent. Therefore when a parent bone’s transform is modified, the transforms of the child bones in model space will also be changed accordingly. Check it out in our documentation!

Snapshots!

Now it is possible to know the state of every actor in the world at a certain frame. No need for synchronous mode to be active!

The world snapshot contains a timestamp and a list of actor snapshots.

carla.WorldSnapshot:
  id
  frame
  timestamp

  has_actor(actor_id)
  find(actor_id)

# Retrieve a snapshot of the world at this point in time.

world_snapshot = world.get_snapshot()

# Wait for the next tick and retrieve the snapshot of the tick.

world_snapshot = world.wait_for_tick()

# Register a callback to get called every time we receive a new snapshot.

world.on_tick(lambda world_snapshot: do_something(world_snapshot))

Code example 2. Retrieving a world snapshot.

Actor snapshots contain the data about the physical state of the actor, and it is possible to retrieve the actual actor through its id.

carla.ActorSnapshot:
  id

  get_transform()
  get_velocity()
  get_angular_velocity()
  get_acceleration()

In order to get an actor snapshot:

> actor_snapshot = world_snapshot.find(actual_actor.id)

Check it out in our documentation!

Updated Synchronous mode

We have made a few modifications to the synchronous mode.

Now tick and apply_settings synchronize automatically with the server so there is no need that users manually wait for tick. Therefore the old recipe using wait_for_tick is no longer working on synchronous mode. Old user scripts using that recipe will be affected!

For more information and examples go to our documentation!

Importing and exporting assets has never been easier!

With this new simplified pipeline it’s easier to import and export maps & props and defining them in the CARLA Blueprint Library.

Assets will be imported as independent packages that can be plugged and exported at any time inside CARLA.

Importing

It is needed to place the files as in the following structure.

Import
|
├── Folder01
|   ├── MapToImport01
│   │   ├── Asphalt1_Diff.png
│   │   ├── Asphalt1_Norm.png
│   │   ├── Asphalt1_Spec.png
│   │   ├── Grass1_Diff.png
│   │   ├── Grass1_Norm.png
│   │   ├── Grass1_Spec.png
│   │   ├── LaneMarking1_Diff.png
│   │   ├── LaneMarking1_Norm.png
│   │   ├── LaneMarking1_Spec.png
│   │   ├── MapToImport01.fbx
│   │   └── MapToImport01.xodr
|   ├── MapToImport02
│   │   ├── MapToImport02.fbx
│   │   └── MapToImport02.xodr
|   ├── AssetsToImport
|   │   ├── PropToImport01
│   │   │   ├── PropToImport01_Diff.png
│   │   │   ├── PropToImport01_Norm.png
│   │   │   ├── PropToImport01_Spec.png
│   │   │   └── PropToImport01.fbx
│   │   ├── PropToImport02
│   │   │   └── PropToImport02.fbx
|   │   └── PropToImport03
│   │       └── PropToImport03.fbx
|   └── Package1.json
└── Folder02
    ├── AssetsToImport
    |   └── PropToImport04
    |       └── PropToImport04.fbx
    └── Package2.json

Project structure.

Finally, a JSON file needs to be created for each package, as shown below.

{
  "maps": [{
      "name": "MyTown01",
      "source": "./MapToImport01/MapToImport01.fbx",
      "use_carla_materials": true,
      "xodr": "./MapToImport01/MapToImport01.xodr"
    },
    {
      "name": "MyTown02",
      "source": "./MapToImport02/MapToImport02.fbx",
      "use_carla_materials": false,
      "xodr": "./MapToImport02/MapToImport02.xodr"
    }
  ],
  "props": [{
      "name": "MyProp01",
      "size": "medium",
      "source": "./AssetsToImport/PropToImport01/PropToImport01.fbx",
      "tag": "SemanticSegmentationTag01"
    },
    {
      "name": "MyProp02",
      "size": "small",
      "source": "./AssetsToImport/PropToImport02/PropToImport02.fbx",
      "tag": "SemanticSegmentationTag02"
    }
  ]
}

The next command will read the JSON file and take each asset and place it inside the Content in Unreal Engine. It also creates a Package1.Package.json file inside used for defining its props in the blueprint library, exposing them in the Python API and also exporting those assets if needed.

> make import

Exporting

The following command will create the distribution packages compressed in a .tar.gz file for each cooked package and place it in Carla/dist/ folder.

> make package ARGS="--packages=Package1,Package2"

Finally, you could use these distribution packages in the CARLA distribution by simply moving them to the CARLA Import folder and executing the ImportAssets.sh script.

Check it out in our documentation!

CARLA docker build

We’ve added a new way to compile CARLA: In a docker!

We use ue4-docker to use Unreal Engine inside Docker, that’s why images produced by the python package that contain UE4 Engine Tools, are prohibited from public distribution. The tutorial can be found in our documentation!

New python API reference

Finally, we have added a new section to our documentation. It is indeed the new python API reference, a compendium of all the client-side API methods.

python_api_reference

Check it out in our documentation!

Vulkan support

It is possible to run CARLA with Vulkan (experimental) or OpenGL. Vulkan performs faster than OpenGL but, as it is an experimental build, there may be some graphical or memory consuming bugs that may affect the user experience when working with CARLA.

Vulkan will prevent CARLA to run off-screen and in Docker, so to run them it is needed to use OpenGL. Therefore the -opengl flag must be activated.

Disclaimer: Despite being an experimental build, Vulkan is the preferred API to run CARLA simulator.

Don’t forget that there are two graphic quality levels, Epic and Low.

More information about Vulkan and OpenGL in our documentation!

Full list of changes

Additions and improvements

Upgraded to Unreal Engine 4.22
Added Vulkan support, if installed, CARLA will use Vulkan, use -opengl flag to launch with OpenGL
The simulator is now compiled in “Shipping” mode, faster but it accepts less command-line arguments
Added a new python script config.py that allows the user to configure the simulator from the command line
Pedestrians are back:
- Spawn pedestrians that will roam randomly on sidewalks
- The script spawn_npc.py spawns now pedestrians, adjust the number with the flag -w
- Added navigation meshes for each maps for pedestrian navigation
Allow adding custom props (FBX) to CARLA Blueprint library so they are spawnable
Simplified pipeline for importing and packaging maps and custom props
Vehicle physics:
- Added access to vehicle transmission details
- Added access to vehicle physics brake values
- Added tire friction trigger boxes for simulating slippery surfaces
Added camera gamma correction as command-line argument to manual_control.py
Added ability to set motion blur settings for RGB camera in sensor python blueprint
Added C++ client example using LibCarla
Added PythonAPI documentation generator, we documented in detail all the Python reference
API extensions:
- Added attachment type “SpringArm” for cinematic cameras
- Added waypoint’s junction_id that returns de OpenDrive identifier of the current junction
- Added world.get_actor(id) to find a single actor by id
- Added carla.WeatherParameters.Default for the default (tailor-made for each town) weather profile
- Added WorldSnapshot that contains a list of ActorSnapshot, allows capturings a “still image” of the world at a single frame
- Added world.tick() now synchronizes with the simulator and returns the id of the newly started frame
- Added world.apply_settings(settings) now synchronizes with the simulator and returns the id of the frame when the settings took effect
- Added world.remove_on_tick(id) to allow removing on tick callbacks
- Added allow setting fixed frame-rate from client-side, now is part of carla.WorldSettings
- Added is_invincible to walkers
New recorder features:
- Documented recorded system and binary file
- Added optional parameter to show more details about a recorder file (related to show_recorder_file_info.py)
- Added playback speed (slow/fast motion) to the replayer
- Allow custom paths for saving the recorded files
- More data is now recorded to replay animations:
  - Wheels of vehicles are animated (steering, throttle, handbrake), also bikes and motorbikes
  - Walker animations are simulated (through speed of walker)
New high quality pedestrians: female, girl and boy; improved meshes and textures
More color and texture variations for each pedestrian
New vehicle Audi Etron: 25.000 tris and LODs
New material for Mustang, new system that will allow us to improve all the vehicle materials
Improved vehicle Tesla
New high-quality “Default” weather tailor-made for each map
Improved the rest of weather profiles too
RGB camera improvements:
- Enabled temporal antialiasing and motion blur
- Added gamma value and motion blur as a blueprint attributes
- Enabled texture streaming for scene captures

Changes

API changes:
- Renamed frame_count and frame_number as frame, old members are kept as deprecated
- world.wait_for_tick() now returns a carla.WorldSnapshot
- The callback of world.on_tick(callback) now receives a carla.WorldSnapshot
- Deprecated waypoint’s is_intersection, use is_junction instead
Several optimizations to the RPC server, now supports a bigger load of async messages
Updated OpenDriveActor to use the new Waypoint API
Removed deprecated code and content
Exposed waypoints and OpenDrive map to UE4 Blueprints
Change the weight of cars. All cars have been compared with the real to have a feedback more real

Fixes

Recorder fixes:
- When a recorded session finish replaying, all vehicles will continue in autopilot, and all pedestrians will stop
- Fixed a possible crash if an actor is respawned before the episode is ready when a new map is loaded automatically
- Actors at start of playback could interpolate positions from its current position instead than the recorded position
- Camera following in playback was not working if a new map was needed to load
- API function ‘show_recorder_file_info’ was showing the wrong parent id
- Script start_recording.py now properly saves destruction of actors at stop
- Problem when vehicles enable autopilot after a replayer, now it works better
Fixed dead-lock when loading a new map in synchronous mode
Fixed get_actors may produce actors without parent
Fixed std::bad_cast when importing other libraries, like tensorflow, before carla
Fixed latitude in WGS84 reprojection code such that Latitudes increase as one move north in CARLA worlds
Fixed walking animations, the animations now go at the same speed as the game
Fixed loading and reloading world not using the timeout
Fixed XODR files can be found now anywhere in content
Fixed bug related with Pygame error of surface too large, added sidewalks and improved lane markings in no_rendering_mode.py
Fixed Lidar effectiveness bug in manual_control.py
Fixed wrong units in VehiclePhysicsControl’s center of mass
Fixed semantic segmentation of bike riders
Fixed inconsistent streetlights in Town03
Fixed incorrect vehicle bounds

← Previous Post Next Post →