Description:

To reliably engage pallets without human intervention, an autonomous forklift should be able to detect the pallet pose and determine its state/load. Such modern computer vision problems are often approached using machine learning techniques, however collecting and labelling large data sets for model training is often difficult and costly. This Crown-sponsored multi-year project seeks to understand the benefits and limitations of synthetic data for training machine learning models for warehouse applications.

In previous Part IV projects, we have proven the feasibility of using synthetic data to train a model to detect pallets from photos and video, demonstrating the model running from a live camera feed. 

Prior PTIV projects provide:

• Existing code that can generate a dataset of labelled synthetic pallet images from a simulated warehouse
• An automated benchmarking tool for training and tuning models
• An existing selection of approximately 350 photos of pallets in Crown’s test warehouse – partially manually labelled.

This year’s project seeks to build upon prior work to provide a pallet pose directly usable by an autonomous forklift, while also reporting information about the pallet’s state.

Type:

Undergraduate

Outcome:

•  Improved model performance in all categories (standalone, racked, and stacked pallets)
• A system output that provides:
  • An estimated pose of the detected pallet
  • An estimation of pallet load dimensions
•  A demonstration of the updated system in Crown’s test facility

Prerequisites

None

Specialisations

• Software Engineering

Categories

• AI & Machine learning
• Image & Voice Processing
• Robotics

Supervisor

• Mahla Nejati

Co-supervisor

• Trevor Gee

Team

• Yechan Kim
• Henri Mueller

Lab

Robotics (405.652, Lab)