Update #9 - Facial Capture and 3D Reference

This morning while I was in the shower I all of a sudden had an idea, it sounds very cliché, but it is the truth. I was thinking about facial animation, and how I could get the best possible reference for it. I have experience using Adobe After Effect for tracking and the thought struck me that I could perhaps somehow use that for tracking my face if I could get the motion data imported to 3Ds Max (my 3D software of choice). I quickly did a video of myself with simple tape strips on my face and managed to track them well enough to proceed, the challenge now was getting it into 3Ds Max.

A round of googling later I happened upon an old script meant to export motion from After Effects to Maya, Max and Lightwave alike. Due to its age I was unsure it would run and do its job properly in the much newer software I have, but it actually worked. I managed to get tracking of my face onto a dummy object in my Max scene, but in order for me to take advantage of that motion for a facial rig, I would have to rig a face with bones, something I had never done before. I had never even tried, just because I always thought it was too difficult for my level of Max understanding, but it turned out I was wrong. I did manage to set up a very basic rigged face, and I did even manage to get the movement from the dummy object reflected on the correct facial bones. The 3D face moved roughly as I had moved.

It was all very rough, but the basic idea was there, and I had accomplished it in only a few hours. I have decided to continue experimenting with this facial rig and the facial capture pipeline, if I can polish both enough so that outsiders can learn to use it easily I wish to use both features both on my group project, and on the project this blog is for, my final year project. I will polish the process most for this project, as a lot of my animation work will be facial.

I am not hoping that I will ever get the facial capture process as to eliminate much need of clean-up, I want to use this more as a 3D reference, that I can apply directly to my rig. I think the mere fact that I can track my face and have the motion on a 3D model within minutes will help me greatly when it comes to getting realistic facial animation. If nothing else, this will be a very interesting experiment which I will continue to work on and develop probably beyond this project. At some point I will have to learn how to script After Effects and Max enough for me to write my own script for exporting motion with the best possible result, but right now I have no doubt in my mind that I will eventually manage to do it.