Music creation through image processing

Hand tracking using image processing to control a music node tool.

Student project from 3rd semester of B.Sc Medialogy, 2015

A semester project focused on image processing. We decided that we wanted to create an application in where you could create your own simple tunes. To control this program, we created a piece of software capable of tracking your hands position on camera, assuming you were wearing a green glove.

Illustration of both contour and convexity hull with the orange arrow representing convexity defect

The program was able to be adjusted during use to calibrate the tracking of the hand and the precision of gesture tracking. The program used gestures to control simple functions such as mouse clicks and some custom buttons for the audio program. The program ended up working very well and there were several different methods to count the number of fingers present for the gestures to be accurate.

An image of the early version of the program before more measures were implemented to increase accuracy post evaluation

If you are interested in learning more about this project, what it aimed to study and the results of that study, you are more than welcome to read the report on the project here. You can also watch the obligatory AV production for a semester project right here if interested, it gives a better summation of the project without being too technical.