A group of researchers are reporting on a new setup, which could finally result in every facial expression made by a gamer be cloned to the avatars face. The results produced by this method are truly amazing. Commercialization of this could be a big change in multiplayer game’s immersiveness.

Immersiveness has played a significant role in games development over the recent times. Game producers have engaged every possible technique to make the gamers feel deeply involved with the game. Customization of the gaming character (avatar) is one such method, where in the gamer feels more involved when the avatar has been customized to look like him/her or to his/her liking.

Multiplayer games have taken gaming to the next level, as the gamers get to compete with real people in real time. Most multiplayer games include chat, emoticons and VOIP for letting the gamers exchange their emotions/feelings over the game. Consider one gamer wearing extra strong body armor is being shot by an enemy with a very tiny gun. Chat, emoticons or VOIP could not really carry over the expressions or emotions of these players, would it not be awesome to see the player (avatar) being shot – laughing hard in this situation. This should be possible very soon, as a group of researchers have designed a setup which makes this possible.

Sample of expressions captured from human(left) to two different avatars

However, this process has had several bottlenecks. Human face which is non-rigid and morphological very different from that of an avatar introduces several complications. Several earlier researches focused on using one camera and achieving this. However it is almost impossible to do this, due to the excitements caused during the games, gamers move around in various directions and this produces large lateral movements of the face, which makes it difficult for a facial analysis system to track and analyze.

In the proposed setup, two cameras are installed on the edges of the screen to acquire real time images of the gamer. Whenever one camera is not able to get sufficient data (due to movement or rotation) of the face, data from the other camera is associated. A multi objective active appearance model is then used which produces data to be used in a synthetic face database. Very briefly, in the 2D frontal view of the gamer, several land marks are established and x coordinates of landmarks in the profile view are combined with it to make a 3D shape model. 2D texture of only frontal view is mapped on the 3D shape and is thus called as 2.5D AAM (active appearance model).

Transforming the observed facial expression of the gamer to the avatar is made possible by a mathematical relationship between their appearances using a database which has a collection of the gamers (human) facial expression and a second database is an optimal database of synthetic facial expressions. The gamer’s facial database is built by requesting the gamer to imitate few specific facial expressions. Results got from the facial analysis in real time are then analyzed using the mathematical model and mapped to an appropriate synthetic facial expression and thus making it all possible.

The researchers report that the accuracy (as shown above) and efficiency of the proposed method have been very high, and hope that this system would not only be used in gaming but also in large events like conferences and meetings. They also propose that this system can be used as an efficient communication where the band width is limited to transmit actual video footage.

Journal Reference: Gamer’s Facial Cloning for Online Interactive Games, Abdul Sattar, Nicolas Stoiber, Renaud Seguier, and Gaspard Breton