This week we discovered a new Disney patent was published regarding jaw tracking without markers for facial performance capture. Given the recent news about the de-aging technology being used in the upcoming Indiana Jones sequel, we wanted to see if this patent is connected to current Disney productions.

While researching the de-aging process, we found the Disney Research Hub’s YouTube channel. We saw that the company had published a “Re-Aging” video showing what appeared to be a seamless new method of de-aging or re-aging an actor’s face with the click of a button.

The results are stunning if what Disney says is true about the ease with which the process is implemented. The FRAN (Facial Re-Aging Network) process can shed 15+ years off of an actor’s face without the result looking overly fake. FRAN can also increase an actor’s age. The results, in my opinion, are far more realistic than using makeup and prosthetics.

This capture from the video shows the input image of the actor on the left and an aged output on the right:

We see some issues when an actor has a five-o-clock shadow to naturally dark facial hair that even a clean shave can’t erase. The results are a little less stunning in that situation because the face looks like it’s waxy. Check out the video below for multiple demonstrations and comparisons to other de-aging techniques.

We demonstrate how the simple U-Net, surprisingly, allows us to advance the state of the art for re-aging real faces on video, with unprecedented temporal stability and preservation of facial identity across variable expressions, viewpoints, and lighting conditions. Finally, our new face re-aging network (FRAN) incorporates simple and intuitive mechanisms that provides artists with localized control and creative freedom to direct and fine-tune the re-aging effect, a feature that is largely important in real production pipelines and often overlooked in related research work.

The video is pretty technical and isn’t meant to be an advertisement for upcoming Disney projects. Seeing the side-by-side comparisons of the actors is interesting, but also looking at how other processes compare to FRAN makes Disney’s product look lightyears ahead of what’s currently on the market.

Hopefully, this will mean we’ll get better-looking cameos in Star Wars productions in the near future. What do you think?

[Source: Disney Research’s YouTube Channel]