The University of Auckland

Project #27: A Framework for Mixed Reality Aided Tools to Stimulate Learning Electric Circuits Fundamentals



Many students learn about electronics in their schools but by the end of their degree many will have forgotten most of the material. In the work force, when faced with a complicated circuit they will likely have no idea why certain components exists in certain places in the circuit. There are no real interactive teaching resources available except articles and tutorials online. Therefore, we propose an interactive teaching tool for high level electronics that can allow users to achieve a practical understanding of electronics theory in relation to what they are looking at. This involves automatically recognizing electrical components in a real-world circuit with Convoluted Neural Networks (CNN) and allowing interactivity through AR/Mixed Reality technology such as Microsoft’s Hololens and Google’s Cardboard. 

Target Users: 1. Tertiary students, it can help them learn and pick it up again if they forget the content. 2. Electronics hobbyists, they are willing to learn but don’t have the resources like teachers to teach them. 3. Working electricians, the tool can help with quick identification and recognition of components within electronic circuits. This tool can help with work such as repairs. 4. Working Computer, Electrical, Electronic Engineering graduates. They might have forgotten most of the electronics if the courses they took in 3rd or 4th years were unrelated to their work. This will help them in the work place. 5. Secondary students, as with hobbyists, the tool provides a fun and interactive way to learn about electronics and can help students understand higher level knowledge in a practical way.

Value Proposition: Currently on the market there are AR electronics learning tools, but they are targeted mainly at secondary students who just got introduced to electricity. They only help with learning the most basic principles such as ohms law. This project’s product will be able to let the user see high level real world circuits and interact with it. The existing AR products for education are mostly marker based which means they can be very limited in terms of functionality. Our product will be markerless which will improve the user experience and allow its use in an uncontrolled manner. Application Features and Description • Names for electrical components displayed. • Pre-defined sub circuits identified and indicated to the user • Each resistor/capacitor in the sub circuit is identified. • Equations for relevant subcircuits e.g. Cutoff frequency, Amplifier gain etc. • Interactive spice simulation of connected components • Information is overlaid on to the real world in a HUD-like manner on the device • Visual and audio based user interface, UI will be navigated using visual and audio cues from the user, allowing a seamless experience with real life interaction. • Targeting reticle for interface so the user can be confident of what they are working with/looking at. ● Interactive schematic generated of the connected components.


Objectives 1. Create an easy to use interactive interface for users to learn about high level circuits. 2. Automatic classification of electrical components in a real-world circuit board with a trained neural network. 3. Automatic recognition of electrical component parameters such as resistor value, capacitor value etc. 4. Effectively recognize sub circuits such as filters, compactors, amplifiers etc. 5. Achieve a measurable level of improvement in user understanding of electronic circuits after usage of the tool.








Allocated (Not available for preferences)


Lab allocations have not been finalised