• Graeme Devine

Elephant in Hand & Pose Tracking


This is running on my Samsung phone streaming remote rendering in Unreal and doing the pose work to follow my hand on the server. It tracks around 200-400 regions of interest in a picture and gives a confidence level of 0-1 about it’s pose, for most of this video it’s 1. As I move the camera around it’s gaining new regions, but it’s a) not remembering the old ones like it should even though they are marked and numbered properly and b) because of the way I’m using a single region to interpret pose (out of those 200-400, more or less where you tap your finger) it’s got orientation issues as it rotates. You can see this when you try it on a Chromebook which lacks an accelerometer to hide the mistakes, it tracks AWESOME about 50% of the time, and then craptacular the rest.


But when it works, it works great, we can track your hand, a box, your floor, your face, your hand open but not closed, so the elephant only appears when your hand is open for example.




BTW, I’m sorry the video is so blurry, it’s hard to take the videos with two hands.


I need to add a lot of visualization to the tests to find the bugs. I have other bugs in taps going from 0.0-1.0 on the X and Y axis only seeming to place, or think it places, towards the middle 50% of the screen. And there’s a crash sometimes after you disconnect and reconnect 20 or 30 times (which you do when you’re debugging).


I’ve been at this 14 hours a day for a week. It seems every mountain I climb, I head through the clouds, get to the top, only to see that I’m not actually at the top, there’s another cloud layer up higher that I need to get to. I wanted put an elephant in your hand and the metropolis pose demo to be our two demos for Wednesday. But the elephant in your hand demo just is not going to be ready for public consumption. I’ve failed you all. For that I’m really sorry. I’m super pissed off at myself.


We do have the metropolis pose demo, tracking 238 points on your face and 33 points on your body, and streaming storytelling demo. Both of which are solid. I need to get more virtual CPUs from Amazon because I am out of instances I can spin up with all these tests. Right now we can spin up one of each.



So basically I’m going to go have a glass of whisky now and call it a night. I feel really terrible about my progress tonight. I just which someone would look at the momentum we have and not want the finish line to be crossed before we can even start.


Graeme.