Digital expressions: New research makes virtual faces more realistic

Facial expressions of virtual characters can more realistically capture those of their human counterparts, thanks to new research from Universidad Autónoma del Estado de México (UAEM) (Autonomous University of Mexico State).

Over the years, technology has made video games increasingl­y realistic and motion capture particularly has been used to better replicate human behaviour in the virtual world.

This has been especially evident in the area of facial expressions with recent developments from companies such as Rockstar Games and Naughty Dog leading the way.

Rockstar Games’ LA Noire uses MotionScan technology to capture actors’ facial expressions, allowing players, aka detectives, to better spot a suspect’s lie. For Naughty Dog’s The Last of Us, special animation was used to capture actors’ facial muscles individually for a more realistic result.

While these developments have certainly brought a new level of reality to video games, the technology has, up to this point, been dependent on actors. But what if virtual characters could be made to be more realistic without the help of actors?

pic1

“I would like to see a character that is expressing itself, not precaptured, not generating canned expressions or creating them from semantic rules, but creating expressions by the same things that create our expressions,” says Javier von der Pahlen, director of creative research and development at the Central Studios division of Activision, as quoted by website IEEE Spectrum in May.

Writer Tekla Perry goes on to say that “a truly digital character, one that does its own acting rather than conveying the acting of another actor, will be created only by merging artificial-intelligence technology that passes the Turing test in both verbal and nonverbal responses — physical gestures, facial expressions and so on — with perfectly rendered computer graphics. That will be really hard.”

Could UAEM’s new research provide us with this kind of technology?

The university’s department of computer science teamed up with the Centro de Investigación y de Estudios Avanzados del Instituto Politécnico Nacional (Center for Research and Advanced Studies of the National Polytechnic Institute) and the University of Guadalajara, with students serving as human models.

Together, they are working on a virtual project known as ‘juego serio’ (‘serious game’) which, according to UAEM engineer Marco Antonio Ramos Corchado, is for educational, scientific and civil purposes. But the research can also be used to make the facial expressions of video game characters more realistic.

pic2

Currently, virtual characters mimic human behaviour through programmed commands or scripts which, Corchado says, results in a “robotic” reaction. UAEM’s research, on the other hand, studies the 43 muscles involved in human facial behaviour to generate more realistic expressions and emotions.

Students are fitted with tactile sensors that produce tiny electrical pulses as they perform different gestures. Using a 3D camera, researchers are able to capture these different gestures.

Taking into account the influence of emotions, attitudes and moods on human behaviour, and their variations depending on social context, the information is then translated into numerical data and entered into a kinesic model designed by UAEM.

This is then used to animate expressions and gestures of virtual characters, such as happiness, sadness, fear, anger, surprise and disgust.

Corchado and his team are busy simulating natural disasters and other situations as part of their ‘serious game’ to capture the varied range of human expressions. In the meantime, it may be time to get ready for an improved sense of reality in less serious, but equally important, games too.


Google Cardboard brings virtual reality and pizza to the masses

Google caused quite a stir earlier this week when it announced its build-your-own virtual reality headset made of cardboard. Simply assemble the viewer using magnets, lenses, Velcro, cardboard and a rubber band and insert your smartphone as the screen that makes the magic happen.

Just a few days after Cardboard’s release, many people have already tried out the method and experienced VR in the comfort of their own homes.

However, Google Cardboard is not the first attempt to make VR accessible to a wider audience. Altergaze, a 3D printed VR headset, generated buzz through its Kickstarter campaign earlier this year.

68

Though 3D printing makes the Altergaze headset much more affordable than high tech PC-based systems such as Oculus Rift, Google’s use of cardboard offers a whole new level of cheap.

Now that devices such as Altergaze and Cardboard are bringing VR technology to the masses, what can we do with it? How will the new availability of VR change our everyday lives?

Novelty factor aside (virtual reality in a pizza box!), Cardboard and other affordable VR headsets have many practical applications. Gaming is perhaps the most obvious use, as they have already been developed in conjunction with video game systems to fully immerse players in fictional worlds.

Beyond games, Altergaze creator Liviu B Antoni sees a whole new frontier of uses for accessible virtual reality viewers.

“360 degree films, immersive panoramic images from your holiday, virtual reality social networking, architectural presentations, VR experiences for public spaces like museums or social events are just a few examples of what the wireless and affordable VR headset has to offer outside the games industry,” he says on Altergaze’s Kickstarter page.

675070b32e843aa136febbc08f1fb92d_large

Indeed, the use of VR in films, theatre, and other areas of the entertainment industry will be transformative.

Going to a cinema could become completely unnecessary as a film experienced in virtual reality would be the same on a home headset. Attending a theatrical performance could simply entail putting on VR goggles to watch the production as if you are actually sitting in the first row.

Not to mention the educational and medical applications that will arise from more frequent use of virtual reality. Children could study the cultures of other countries by exploring cities through a VR viewer.

Medical students could practice surgeries and gain experience before they ever operate on a real person. Doctors and patients could see the human body on a cellular level to decide proper treatments.

These various uses are just a starting point. The number of ways we integrate this technology into our lives will only continue to grow as more and more people order a pizza, fold up the box and realise the potential of virtual reality.


Featured image and first body image courtsey of Google, second body image courtesy of Altergaze.