1
1
Canada
5 years of experience
I have been a professional developer since 2018 when I co-founded a game development company based in USA. I started studying Software Engineering at McGill University in 2021. I have also been working at a biotech company in drug discovery for rare diseases since 2021. I am interested in machine learning and how it applies to various fields. Recently, I have worked on projects using ML in health sciences and have collaborated on a research project with the Montreal Institute for Learning Algorithms (MILA).
To accomplish our goal of censoring gestures, we knew we needed 2 layers to our algorithm. The first layer needed to recognize landmarks on hands and in the second we needed to identify the gesture using the landmarks. To recognize the landmarks, we used Google’s mediapipe software, which comes with a pre-trained hand recognition model. Our innovations came from gesture recognition. We encoded the landmarks as features by creating a distance matrix between each landmark and every other landmark. These distances were then used in a support vector machine to classify the type of gesture. We found a dataset of 600 images, with 31 unique gestures. We then added to this dataset by curating over 50 images of middle finger gestures. Using this dataset, we can train a model that is good at classifying the middle finger gesture. Next, we blur the hand by using each landmark as a point and mapping a gaussian density around the points. We use the OpenCV blur function to blur the pixels that have density. The project includes an entire Django frontend and REST service for censoring videos. It also comes with a Dockerfile to deploy the stateless service.