Archives For February 2014

Sailing simulation demo

February 18, 2014 — Leave a comment

Few years ago (2010) I made a sailing simulation demo. This video shows some footage from this demo:


The sailboat model is motivated by the real physics of sailing, but it is not 100% correct. I made a dynamical system that roughly approximates the real thing. After reading several white papers, I realized that making a “real” simulator would be a huge endeavor and so I cut few corners :-). But even with my simplifications, the model can tack realistically, can get stuck “in irons”, the wind can blow the boat backwards, etc. Also, I implemented a simple collision system, AI that drives NPC boats (with red sails), line of sight, wave/ocean simulation, skeletal animation, light, crafted 3D models, etc. AI that controls NPC boats does not cheat and controls the same dynamical system as the player’s boat by doing global optimization over the control/goal function. The goal of the NPC boats is to catch the player’s boat with white sails. The demo uses DirectX 9.0 and uses a basic framework that comes with DX9 code samples.

You can use the demo source code if you wish (as is and with no warranty). The ZIP file with the C++ source code (VS 2010 project), shaders and the demo executable (runs on Windows with DX9 installed) is located in this file –


My team at Microsoft shipped the high definition face tracking and face shape modelling tech as part of XBox One Kinect back in November 2013. These are some facial expressions that can be tracked by Kinect in real time and animated in 3D (gray masks) with pretty good precision:



The system finds a big number of semantic points on a user’s face in 3D. The developers can then use this information to animate avatars or do some other processing (like recognize expressions, etc.). This video shows the early prototype of this technology that I made back in 2012:     This is a bit more technical video that demos the 3D mask painted over the video stream (for those who like this stuff):     The algorithm that we created is a fusion of a 2D video feature tracker based on Active Appearance Model (AAM) and a 3D tracker based on Iterative Closest Point (ICP) algorithm, which aligns 3D face model based on depth data from Kinect. Both trackers are resolved together and linked by special 2D-3D constraints such that the alignment makes sense and looks natural. We are going to publish details as a paper some time soon. We also compute 3D face shapes for a given user from a set of input frames (RGB + depth).

We published this paper that describes the face tracking and modeling algorithms in details. The resulting 3D models are pretty realistic and look like this:




Now it is up to game developers to use this tech in their titles! The API is available as part of Xbox One XDK. For example you can produce NPC faces like this in your game:

Nikolai as a textured shape model

Nikolai as a textured shape model


The 3D model that we shipped in Xbox One Kinect + face capture system is very capable and flexible. The face tracking and face shape modelling is used in just released Kinect Sports Rivals game by RARE studio. You can see the face shape modelling demo in this video: