By Laura Kobylecky

Some people see them as a fun Internet novelty while others may see them as a sign that could lead to the total collapse of media trust. Ian Beacraft, CEO and Chief Futurist at Signal And Cipher (an agency that helps brands adapt to future tech) has a more optimistic perspective.

Synthetic media is a broad term for the production of media through artificial means. Beacraft introduces the concept with some popular Internet animations. The videos show celebrities and movie characters moving their mouths in sync to the words he is saying, creating the illusion that they, rather than Beacraft, are the ones speaking. These videos are rudimentary. The mouths have slightly blurred edges and a certain dissonance that flags them as fake. However, the result is close enough to convey the idea.

Synthetic media is a broad term for the production of media through artificial means
Share on X

Beacraft then brings up animated versions of himself speaking in different languages, followed by a moment where he is placed in that iconic Matrix scene and he takes the obligatory “red” pill. He further illustrates the concept with an image of Jennifer Lawrence accepting an award, but her face has been digitally replaced by Steve Buscemi. These are some examples of that synthetic media which is manipulated or made entirely by algorithms.

He addresses the more nefarious possibilities of this technology. This sort of digital replacement could be used to artificially create “revenge porn” or to alter statements made by political figures. He balances this with the more professional applications of synthetic technology. Paul Walker appeared in Furious 7, after his death, using a digital recreation. In Rogue One, a particularly young Princess Leia was brought back for a cameo.

There are three categories of deepfakes: voice cloning, puppetry, and face-swapping #SXSW 2021
Share on X

Beacraft explains three categories of deepfakes: voice cloning, puppetry and face-swapping. Voice cloning involves taking an “imprint” of someone’s voice and then using that in a variety of contexts, different from the original instances. A face-swap is the digital replacement of one face with another. Puppetry involves making someone appear to say words that they did not, by digitally altering the movement of the face to match the words. All of these are in some ways based on existing media and existing people.

Digital Beings

A step further is the idea of “digital beings” with an entirely synthetic basis that do note require a “real” human as a reference. Beacraft explains that realistically-human, synthetic beings might be appealing as digital assistants. They could also have a place in video games or in AR/VR applications.

One tool that might be useful in this is MetaHuman Creator, an upcoming application made by Unreal Engine, the creators of Fortnite. This application will be a cloud-streamed app that makes digital creation faster and simpler. Beacraft sees this as an example of how these digital beings might become more significant, accessible, and relevant to the public in the future.

Beacraft also sees that digital beings might become increasingly relevant as public figures. Right now avatars like the entirely synthetic, “Lil Miquela,” have proven that digital beings can capture the public’s imagination. Lil Miquela has three million Instagram followers and has proven to be a powerful influencer. The Geico Gecko and even the KFC spokesman, Colonel Sanders, are also examples of synthetic beings, according to Beacraft. They do not exist in the “real world” but they are admired as their own entities and influence public perceptions.

Making 3D Models

Accessibility is another big trend that might change the landscape of synthetic media. Photogrammetry is one currently used way of making 3D models. It involves taking many images and stitching them together in a 3D program. Beacraft explains that new techniques have made 3d avatars easier to create. He begins a demonstration of this by showing how a single photo can become a 3D model.

In the future, digital beings might become increasingly relevant as public figures #SXSW 2021
Share on X

He uses a neural network architecture called “PIFuHD.“ This is a cloud app that seems to require some level of coding skill to use and is available on GitHub. Beacraft uses a single image of himself to create a 3D model. Since the photo provides no reference for the back of his figure, the program guessed this part based on data it had been fed in its training phrase.

Beacraft puts the resulting model in a 3D modeling program to polish it up. He overlays photographs on the model to add the colors and details. The haircut is added via a texture program and reference images because apparently hair is a bit more difficult to work with. The resulting 3D human is somewhat realistic. The figure is obviously recognizable as Beacraft, but it has the usual slight “uncanny valley” strangeness and small inconsistencies that you may find in digital avatars.

Now that he has a 3D model, Beacraft uses a program called Mixamo to give it life. The program allows him to animate his model with a variety of movements. First, he makes the character walk; Next, he animates the character with a “moonwalk.” The figure’s awkward shambling makes it look a bit like a very smooth zombie. But it does work.

Open Metaverse

The advance of 3D modeling stretches beyond simple recreations of human figures. Beacraft explains that blockchain may play a key role in this future. In an “open metaverse,” digital creations might move more freely. NFT and blockchain could allow for the ownership and transportation of digital creation across platforms.

Synthetic media is not just about deepfakes and high value visual effects, but about the creation of new worlds #SXSW 2021
Share on X

For example, a person could make or buy a digital garment for their avatar and be able to sell it as an individual item, supported by blockchain technology. Thanks to the open metaverse, this item might not have to remain in a specific game or ecosystem. Any other environments that embraced this technology could also import such items.

Beacraft seems to believe in an optimistic future for synthetic media. He acknowledges that many people are concerned about artificial intelligence taking away jobs as they exist today, yet he believes that the ease of use for emerging AI design technologies will offset those losses by creating new opportunities for a wide variety of people and a range of expertise.

“These losses are preparing us for the worlds we’re going to be entering today and tomorrow. Synthetic media is not just about deepfakes and high value visual effects, but about the creation of new worlds,” Bearcraft concludes.

Tech Trends’ Digital Consultancy services offer support for companies looking to enhance brand strategy with immersive technologies such as Virtual, Augmented, and Mixed Reality

Laura Kobylecky is a contributing writer to Tech Trends. She is particularly interested in new and emerging technology and culture. Connect with her on LinkedIn 

The post Synthetic Media: More than Just Deepfakes appeared first on Tech Trends.

By Laura Kobylecky

#SXSW Online 2021 is bringing people together in a socially distant world. For this session, “Can 5G Transform the Live Music Experience?” Cristiano Amon and Steve Aoki have come together to talk about the impact of 5G technology on music. Cristiano Amon is the president of Qualcomm Incorporated and Aoki is a Grammy Nominated DJ, Producer, Music Executive, and Designer.

5G is a popular topic in 2021. It was discussed at the Consumer Electronics Show (CES) this year and remains a point of interest at SXSW this year. Qualcomm’s website explains that 5G is “a new global wireless standard after 1G, 2G, 3G, and 4G network.” This wireless network could be a large improvement over the previous networks by delivering “higher multi-Gbps peak data speeds, ultra-low latency, more reliability, massive network capacity, increased availability, and a more uniform user experience to more users.”

5G remains a popular topic in 2021 #SXSW
Share on X

The virtual session with Amon and Aoki begins on the lighter side with a casual tour through Aoki’s opulent home. Aoki jumps off of a roof into a pool that is 16 feet deep. He shows off a large library room full of books and vinyl. Amon and Aoki discuss their mutual fondness for vinyl. Aoki shares an anecdote about how he acquired some of his vinyl after college when some friends couldn’t move with all their bulky collection. While technology has moved forward and allowed more efficient digital storage of music, Aoki seems to have maintained a deep fondness for this antiquated, tangible, method of music collection.

The home tour moves into Aoki’s home office, also known as “Aoki’s Playhouse.” Aoki’s Playhouse “pays the bills” and is also “where the magic happens,” according to Aoki. This room used to be a racquetball court. It has been remodeled and contains a large green screen, some GoPros, and the other tools that Aoki uses to perform and live stream concerts as a DJ.

Aoki talks about how the progression of technology has changed his experiences as a DJ. Nowadays, “anyone can DJ any song at any given place.” However, earlier in his career, “I used to tow in records. I could only play the 60 records that I brought. So, you go from 60 records 15 years ago, to having unlimited access to the internet.” Nowadays, when Aoki is traveling around the world, he can quickly download new music to an SD card and then insert the SD card into his unit.

@steveaoki talks about how the progression of technology has changed his experience as a DJ #SXSW 2021
Share on X

For Aoki, the possibility of 5G streaming would be a game-changer. He says it would give him “access to being able to play so much music to so many people.” One thing that 5G could help with is increasing access to computing power. Amon, of Qualcomm, explains how this would work. With 5G, the internet speed would let you connect to the cloud in real-time. Through that connection, you could get access to more computing power. Even if you were working from a phone, you could potentially access the “processing power of your studio,” from a distance, through the cloud.

@steveaoki believes that technology played a key role in keeping people connected during this time of social distancing and online conferences #SXSW 2021
Share on X

Aoki sees the value in that processing power. He explains that input and output speeds are significant to his work. For him, “It’s all about being in the flow, I don’t want to disrupt the flow.” He remembers how he used to spend a great deal of time sitting around and waiting for downloads or waiting for stems to render. He says that the faster workflow, enabled by 5G would help prevent being stuck in the “bottleneck of creativity.”

Even if you were working from a phone, you could potentially access the processing power of your studio, from a distance, through the cloud #SXSW 2021
Share on X

5G could also have a big impact on the festival experience. Aoki talks about the difficulties of staying connected with people at festivals. Currently, a person might just turn their phone on “airplane mode,” since they can’t get reception anyway. They could be at Coachella or Electric Daisy Carnival, trying to post but it’s like “a wall has gone over.” For Aoki, being able to share his experiences in real-time is important. 5G connectivity could make that possible soon.

This 5G connectivity could also enable better quality video. Amon explains that with 5G, people might be able to broadcast 4k interactive video in real-time. He also proposes the idea that Augmented Reality glasses could be a tool for sharing those perspectives, where a person could live stream high definition video taken by their glasses and the video would let other people look at the world through their eyes. The glasses could also provide real-time overlays of digital experiences so that a person could interact with the virtual world and the real world simultaneously.

A faster workflow enabled by 5G would help prevent being stuck in a bottleneck of creativity #SXSW 2021
Share on X

Aoki is enthusiastic about the ideas presented. He says “I believe not just in the merging of music and technology, but in human and technology.” Although he is an avid tech enthusiastic, Aoki remains practical about the limitations. Not everyone is caught up on the latest tech and Aoki doesn’t want them to get left behind. He explains his methods for deciding which innovations he wants to include in his process:

“When I produce I always listen on every different medium. You have to think just enough ahead but not too far ahead because if you think too far ahead they won’t get it. It goes right over their heads.”

For Aoki, the main priorities are quality content and connection and 5G may help with both of these. His music and live streams could be improved with 5G. The content would be delivered faster and with higher definition. This instant delivery would allow Aoki to take those lifestream experiences and amplify connections as he is able to provide more instant responses.

5G could also have a big impact on the festival experience #SXSW by @TheLauraKoby
Share on X

Aoki believes that “technology played a key role in keeping people connected” during this time of social distancing and online conferences. He feels that “at the end of the day I want to be able to connect with as many people as possible…in all the different ways that I can.” 5G may help Aoki fulfill those goals and enable other creators to do the same.

Tech Trends’ Digital Consultancy services offer support for companies looking to enhance brand strategy with immersive technologies such as Virtual, Augmented, and Mixed Reality

Laura Kobylecky is a contributing writer to Tech Trends. She is particularly interested in new and emerging technology and culture. Connect with her on LinkedIn 

The post 5G and the Future of Music appeared first on Tech Trends.