A software engineer from Taiwan has been acknowledged by the Academy of Motion Picture Arts and Sciences for his groundbreaking facial-digitizing technology that is behind the adrenaline-inducing visual effects of a number of Hollywood science-fiction blockbusters that have awed global audiences.

Ma Wan-chun, who now works as an engineer for Google, received the Scientific and Technical Achievement Award along with his colleagues at the 91st Academy Awards this month for their computer graphics technology developed at the University of Southern California’s Institute for Creative Technologies.

Ma helped created an older version of Brad Pitt in The Curious Case of Benjamin Button (2008), brought the Na’vi tribe to the big screen in Avatar (2009) and re-created Paul Walker’s performance in Furious 7 (2015) after the actor died mid-production.

An aged Brad Pitt in the 2008 fantasy romantic drama film The Curious Case of Benjamin Button. Photo: Handout
A still from the 2009 blockbuster Avatar, whose production relied on the facial-digitizing technology. Photo: Handout

Polarized spherical gradient illumination technology is a facial-appearance-capture algorithm that uses a dome-shaped object called Light Stage X. When an actor performs inside it, he is illuminated by about 300 LED (light-emitting diode) units for his appearance and gestures to be captured and analyzed by cameras from different angles.

The footage is then converted into a three-dimensional model for visual-effects computation and rendering, enabling a computer to understand the geometry, pore texture and light properties of an actor’s face, according to Ma.

The technology is a breakthrough in facial capture when producing special visual effects, allowing shape and reflectance capture of an actor’s face with sub-millimeter detail, enabling the faithful re-creation of characters’ faces. The Light Stage X structure is the foundation for all subsequent innovation and has been the keystone of the method’s evolution into a production system.

Ma told Taiwan’s Central News Agency that he had always been fascinated by visual effects in movies and video games, such as the diverse alien characters in the Star Wars franchise and the combination of real actors and animated backgrounds in the 1994 game Wing Commander 3.

As the field of computer graphics was still in its infancy in Taiwan in the early 2000s, Ma used a scholarship from Taiwan’s Graduate Student Study-Abroad Program to pursue his studies. He said that instead of only researching for his doctoral dissertation, he always wanted to put his expertise into practice, and the work at USC helped him “turn imagination into reality.”

His team’s next step is to transfer the realistic visual effects presented in cinemas to smartphones by applying the technology to the fields of augmented reality and virtual reality.