CARLA

Introduction to CARLA

  • CARLA: An open-source autonomous driving simulator.

  • Developed for the training, validation, and prototyping of urban driving systems.

  • Offers open digital assets including urban layouts, vehicles, buildings, and pedestrians.

  • Flexible specification of sensor suites and environmental conditions.

Overview of Research Approaches

  • CARLA is utilized to assess the performance of three approaches:

    • Classic modular pipeline: Vision-based perception module, rule-based planner, maneuver controller.

    • End-to-end model trained via imitation learning.

    • End-to-end model trained via reinforcement learning.

  • Evaluated across increasingly complex scenarios in an urban environment.

  • Performance metrics provided by CARLA demonstrate the utility of this platform.

Challenges in Autonomous Urban Driving

  • Complex environment with multi-agent dynamics (e.g., traffic intersections).

  • Necessity to track various actors (vehicles, pedestrians).

  • Recognition of road signs, lights, and other communication elements.

  • Need for quick decision-making in dynamic scenarios (e.g., pedestrians entering the road).

  • Risks associated with real-world testing, as many scenarios can be dangerous.

Benefits of Simulation

  • Simulation democratizes autonomous driving research,

    • Reduces high costs associated with physical world training.

    • Allows for extensive data collection in diverse scenarios without physical risks.

CARLA Features

  • Built on Unreal Engine 4; supports a dynamic world simulation.

  • Server-client architecture:

    • Server handles simulation rendering and environment dynamics.

    • Client interacts with the server to control the autonomous agent.

Environment Configuration

  • Contains 3D models of:

    • Static objects (e.g., buildings).

    • Dynamic entities like vehicles and pedestrians.

  • Two towns created for training (Town 1) and testing (Town 2).

  • Different atmospheric conditions available (lighting, weather).

Sensors and Readings

  • Configurable sensor suite in CARLA includes:

    • RGB cameras and pseudo-sensors for ground truth data.

    • Extensive customizability of camera parameters (type, location, orientation).

Approaches to Autonomous Driving in CARLA

1. Modular Pipeline

  • Divided into perception, planning, and control modules.

  • Perception Module: Uses semantic segmentation for environment mapping.

  • Local Planner: Generates waypoints based on environmental data.

  • Continuous Control: Uses a PID controller for vehicle dynamics.

2. Imitation Learning

  • Relies on recorded driving data from expert human drivers.

  • Dataset includes driving traces that consist of observations, commands, and actions.

  • High-level commands improve robustness of learned behavior.

3. Reinforcement Learning

  • Uses A3C (Asynchronous Advantage Actor-Critic) to optimize actions based on rewards.

  • Focuses on achieving navigation goals while avoiding obstacles.

Experimental Setup and Evaluation

  • Experiments conducted in controlled navigation tasks:

    • Tasks organized by difficulty (straight path, one turn, complex navigation).

  • Performance metrics: success rate in completing navigation tasks, inference analysis, collision data, etc.

  • Results indicate:

    • Modular pipeline and imitation learning generally perform well across tasks compared to reinforcement learning.

    • Generalization to unseen towns and weather conditions remains a challenge.

Conclusion

  • CARLA presents a powerful tool for autonomous vehicle research.

  • Promotes widespread experimentation and sharing of knowledge in the field.

  • Encourages further research on improving simulation-based training methods.