Language Selection

English French German Italian Portuguese Spanish

Monado's hand tracking: hand-waving our way towards a first attempt

Filed under
Development
Graphics/Benchmarks
Linux

As part of my internship at Collabora, I picked up Monado's hand tracking project. Today I will outline the section I did during the summer of 2021, which was a fairly bare-bones first attempt. Keep in mind that we've moved on from the architecture I describe here and have made considerable progress since then. More than anything, this is to illustrate how awesome it is to experience an internship at Collabora.

So. I started my internship right in the middle of this project - we already had done the work on model architecture and had developed unique techniques for training these models. Now it was time to take those trained models and try to deploy them inside Monado. Optical hand tracking for XR has a bit of a reputation as a Very Hard Tracking Task, and indeed it is - getting training data, training neural nets, and deploying them in real-time, low-latency environments such as XR is every bit as hard as they say it is. And also, when I started, I had very little experience with computer vision. But, somebody needed to do this; I decided I'd be crazy and just go for it.

Read more

More in Tux Machines