Avatar Kinect

May 19, 2012 — 4 Comments

In summer 2011, my group at Microsoft shipped an interesting Computer Vision app/mini game for XBox 360 called “Avatar Kinect”. I worked on the 2D/3D face tracking technology. You can pose in front of the Kinect camera and the application tracks movements of your head and facial features (lips, eyebrows) and renders you as an XBox avatar. Pretty cool app if you want to record little videos of yourself as an animated cartoon avatar and then post them to YouTube. Or if you want to talk to your friends as an avatar, the app allows multiparty avatar chat.

For example you can do:

Here is the internals of our face tracking technology:

We used a combination of Active Appearance Models  on “steroids” plus few other things like neural network, face detector and various classifiers to make it stable and robust. You can read more about Active Appearance Models here, here  and here . Off course the usage of Kinect camera improved precision and robustness a lot (due to its depth camera).

For more information about our face tracking tech, see my paper “Real-time 3D face tracking based on active appearance model constrained by depth data”.

Advertisements

4 responses to Avatar Kinect

  1. 

    Excuse me sir !
    Can I Get Sample Source of Tracking Avatar’s Lips

Trackbacks and Pingbacks:

  1. Face Tracking SDK in Kinect For Windows 1.5 « My Random List - May 21, 2012

    […] Avatar Kinect […]

  2. 【转】Kinect人脸跟踪算法 | 增强视觉 | 计算机视觉 增强现实 - May 25, 2012

    […] 1.5! I worked on the face tracking technology (starting from the times when it was part of Avatar Kinect) and so I’d like to describe its capabilities and limitations here. First of all, here is the […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s