The Formula SAE competition is introducing an autonomous vehicle category. To meet this challenge, we aim to develop a fully autonomous control system that enables the car to learn to drive itself through Reinforcement Learning. Reinforcement Learning is a machine learning approach that seeks to allow a robot to learn to operate based on its interaction with the environment without human input or control.
While RL has demonstrable capability at "learning" to extract relevant information from images directly through the auto-encoder paradigm, this approach adds complexity to the problem due to the non-trivial encoding of information within images. To assess how much of an impact this has, it is possible to compare the performance of RL systems that use images directly with systems that use a custom image data extraction layer using computer vision techniques.
This project proposes doing precisely this. Your mission is to create a computer vision layer for detecting and localising a slower-moving opponent vehicle on the track to support overtaking manoeuvres based on control logic learned through RL. In its simplest form, you will use images from an RGB-D active depth camera to (1) determine the position of the target vehicle in the image and (2) the vehicle's location (and orientation) in 3D space. Repeated measurements should give you speed and acceleration estimates (relative to your vehicle) and a set of historical waypoints that would allow you to interpolate a trajectory of the opponent vehicle indexed by time (also from your vehicle's frame of reference). Ambitious students may even try their hand at future trajectory extrapolation. Once the data extraction problems have been solved, the next challenge will be to design a logical structure for presenting this information to an RL algorithm. You will also need to design and develop real-world experiments (not in simulation) to verify the accuracy of your solution.
Time permitting, the project's final goal will be to compare your computer vision approach's RL training times and performance against a purely image-based RL solution (most likely in a simulated environment). This will be done with the collaboration and support of other teams and members within CARES (https://cares.blogs.auckland.ac.nz/). Note that much of the hardware and RL infrastructure for this project was developed within CARES by Henry and his team, and you will work with them on the RL side of the project while Trevor will provide support on the computer vision side.
Undergraduate
The outcomes of this project should be:
High-quality work could potentially lead to a publication at an international conference.
Students should be competent Python programmers, able to come to the Lab to work with cameras and equipment, and comfortable with Linear Algebra (computer vision mathematics).
Robotics (405.652, Lab)