Tech Trends South By Southwest 2021 Deepfakes
Home » Blog » Tech Trends » Synthetic Media: More than Just Deepfakes

Synthetic Media: More than Just Deepfakes

Deepfakes are a hot topic, including at #SXSW 2021.

By Laura Kobylecky

Some people see them as a fun Internet novelty while others may see them as a sign that could lead to the total collapse of media trust. Ian Beacraft, CEO and Chief Futurist at Signal And Cipher (an agency that helps brands adapt to future tech) has a more optimistic perspective.

Synthetic media is a broad term for the production of media through artificial means. Beacraft introduces the concept with some popular Internet animations. The videos show celebrities and movie characters moving their mouths in sync to the words he is saying, creating the illusion that they, rather than Beacraft, are the ones speaking. These videos are rudimentary. The mouths have slightly blurred edges and a certain dissonance that flags them as fake. However, the result is close enough to convey the idea.

Synthetic media is a broad term for the production of media through artificial means Click To Tweet

Beacraft then brings up animated versions of himself speaking in different languages, followed by a moment where he is placed in that iconic Matrix scene and he takes the obligatory “red” pill. He further illustrates the concept with an image of Jennifer Lawrence accepting an award, but her face has been digitally replaced by Steve Buscemi. These are some examples of that synthetic media which is manipulated or made entirely by algorithms.

He addresses the more nefarious possibilities of this technology. This sort of digital replacement could be used to artificially create “revenge porn” or to alter statements made by political figures. He balances this with the more professional applications of synthetic technology. Paul Walker appeared in Furious 7, after his death, using a digital recreation. In Rogue One, a particularly young Princess Leia was brought back for a cameo.

There are three categories of deepfakes: voice cloning, puppetry, and face-swapping #SXSW 2021 Click To Tweet

Beacraft explains three categories of deepfakes: voice cloning, puppetry and face-swapping. Voice cloning involves taking an “imprint” of someone’s voice and then using that in a variety of contexts, different from the original instances. A face-swap is the digital replacement of one face with another. Puppetry involves making someone appear to say words that they did not, by digitally altering the movement of the face to match the words. All of these are in some ways based on existing media and existing people.

Digital Beings

A step further is the idea of “digital beings” with an entirely synthetic basis that do note require a “real” human as a reference. Beacraft explains that realistically-human, synthetic beings might be appealing as digital assistants. They could also have a place in video games or in AR/VR applications.

One tool that might be useful in this is MetaHuman Creator, an upcoming application made by Unreal Engine, the creators of Fortnite. This application will be a cloud-streamed app that makes digital creation faster and simpler. Beacraft sees this as an example of how these digital beings might become more significant, accessible, and relevant to the public in the future.

Beacraft also sees that digital beings might become increasingly relevant as public figures. Right now avatars like the entirely synthetic, “Lil Miquela,” have proven that digital beings can capture the public’s imagination. Lil Miquela has three million Instagram followers and has proven to be a powerful influencer. The Geico Gecko and even the KFC spokesman, Colonel Sanders, are also examples of synthetic beings, according to Beacraft. They do not exist in the “real world” but they are admired as their own entities and influence public perceptions.

Making 3D Models

Accessibility is another big trend that might change the landscape of synthetic media. Photogrammetry is one currently used way of making 3D models. It involves taking many images and stitching them together in a 3D program. Beacraft explains that new techniques have made 3d avatars easier to create. He begins a demonstration of this by showing how a single photo can become a 3D model.

In the future, digital beings might become increasingly relevant as public figures #SXSW 2021 Click To Tweet

He uses a neural network architecture called “PIFuHD.“ This is a cloud app that seems to require some level of coding skill to use and is available on GitHub. Beacraft uses a single image of himself to create a 3D model. Since the photo provides no reference for the back of his figure, the program guessed this part based on data it had been fed in its training phrase.

Beacraft puts the resulting model in a 3D modeling program to polish it up. He overlays photographs on the model to add the colors and details. The haircut is added via a texture program and reference images because apparently hair is a bit more difficult to work with. The resulting 3D human is somewhat realistic. The figure is obviously recognizable as Beacraft, but it has the usual slight “uncanny valley” strangeness and small inconsistencies that you may find in digital avatars.

Now that he has a 3D model, Beacraft uses a program called Mixamo to give it life. The program allows him to animate his model with a variety of movements. First, he makes the character walk; Next, he animates the character with a “moonwalk.” The figure’s awkward shambling makes it look a bit like a very smooth zombie. But it does work.

Open Metaverse

The advance of 3D modeling stretches beyond simple recreations of human figures. Beacraft explains that blockchain may play a key role in this future. In an “open metaverse,” digital creations might move more freely. NFT and blockchain could allow for the ownership and transportation of digital creation across platforms.

Synthetic media is not just about deepfakes and high value visual effects, but about the creation of new worlds #SXSW 2021 Click To Tweet

For example, a person could make or buy a digital garment for their avatar and be able to sell it as an individual item, supported by blockchain technology. Thanks to the open metaverse, this item might not have to remain in a specific game or ecosystem. Any other environments that embraced this technology could also import such items.

Beacraft seems to believe in an optimistic future for synthetic media. He acknowledges that many people are concerned about artificial intelligence taking away jobs as they exist today, yet he believes that the ease of use for emerging AI design technologies will offset those losses by creating new opportunities for a wide variety of people and a range of expertise.

“These losses are preparing us for the worlds we’re going to be entering today and tomorrow. Synthetic media is not just about deepfakes and high value visual effects, but about the creation of new worlds,” Bearcraft concludes.

Tech Trends’ Digital Consultancy services offer support for companies looking to enhance brand strategy with immersive technologies such as Virtual, Augmented, and Mixed Reality

Laura Kobylecky is a contributing writer to Tech Trends. She is particularly interested in new and emerging technology and culture. Connect with her on LinkedIn