Core77

"Technology Art:" Combining Facial Tracking and Projection Mapping for Real-Time Face Hacking

This may or may not give you nightmares

By Rain Noe - April 27, 2015

in Tech

"Face hacking" might not have an artistic ring to it, but it looks stunning in execution. A collaboration between a multimedia artist, a makeup artist and a digital image engineer—Nobumichi Asai, Horoto Kuwahara and Paul Lacroix, respectively—has yielded Project Omote, which combines face-tracking technology with projection mapping to change people's faces in real-time. Here's the inaugural effort, developed last year and using model Yuka Sekimizu:

Enter a caption (optional)

"Technically, it's very difficult to project with real-time tracking," writes Asai.

Nobody has done it before, so we had to start from scratch. The process was complicated, capturing with sensors, processing the data, creating CG renderings, and using projectors. Getting the timing right was also challenging, fixing delays and making sure the projection technology had the maximum precision to get the face and visuals matched.

…We used OptiTrack sensors for [their] speedy response, and developed original programs [in] C++. To [minimize] latency (delay), we devised ways to integrate the whole process into one program. This creates amazing effects.

This year the team pushed the effects further, using members of the J-pop band SMAP:

Enter a caption (optional)

You can read more about the project here.

o

Favorite This
Q
3

Comment

Comments

Eitan Mendelowitz

9 years ago

Z

Z

Reply

Chase No Face / BELL from zach lieberman on Vimeo.
"Nobody has done this before..." That isn't true, this project was done over 3 years ago and was well covered (creatersproject, wired, etc). There are others too. Project Omote looks more sophisticated but to claim it is the first project is either disingenuous or ill informed.

!Report as spam
- Eitan Mendelowitz
  
  9 years ago
  
  Z
  
  Z
  
  Reply
  
  In it's defence, it is the first one to combine the tracking and 3d mapping along with the projection. In the video you provided the projections felt very flat and did not appear to track to the persons face.
  
  !Report as spam
test account

9 years ago

Z

Z

Reply

factitious to the core, no pun intended

!Report as spam