MILO

MILO (Motion Input for Language and Operation) is a sign language detection system I built using computer vision and a custom neural network. It can recognize hand gestures and learn new ones over time, allowing those gestures to be used to control a mouse or type on a computer.

The project uses OpenCV and MediaPipe for real-time hand tracking, combined with a custom neural network for gesture classification. It supports trainable gestures, so new inputs can be added and mapped to different actions.

I created MILO to learn how to build and train neural networks and to explore more advanced machine learning concepts. All of the code was written from scratch, and it was a project I really enjoyed working on.

I plan to continue developing MILO and eventually use it as a tool to help people learn sign language.

Click on the video below to see MILO in action.

Presentation

Below is a presentation that shows the different features you can use.