AI deepfake ‘news anchors’ found in pro-China footage on social media, research firm says
]
The “news broadcasters” appear stunningly real, but they are AI-generated deepfakes in first-of-their-kind propaganda videos that a research report published Tuesday attributed to Chinese state-aligned actors.
The fake anchors — for a fictious news outlet called Wolf News — were created by artificial intelligence software and appeared in footage on social media that seemed to promote the interests of the Chinese Communist Party, US-based research firm Graphika said in its report.
AI-generated people acting as ‘news anchors’ in Wolf News videos. Photo: Graphika.
“This is the first time we’ve seen a state-aligned operation use AI-generated video footage of a fictitious person to create deceptive political content,” Jack Stubbs, vice president of intelligence at Graphika, told AFP.
In one video analyzed by Graphika, a fictious male anchor who calls himself Alex critiques US inaction over gun violence plaguing the country. In the second, a female anchor stresses the importance of “great power cooperation” between China and the United States.
Advancements in AI have stoked global alarm over the technology’s potential for disinformation and misuse, with deepfake images created out of thin air and people shown mouthing things they never said.
There was no immediate comment from China on Graphika’s report, which comes just weeks after Beijing adopted expansive rules to regulate deepfakes.
See more For the first time, Graphika has observed a state-aligned information operation promoting video footage of AI-generated fictitious people. Read more in our latest #Spamouflage report: https://t.co/y6sn4z0kwo — Graphika (@Graphika_NYC) February 7, 2023
China enforced new rules last month that will require businesses offering deepfake services to obtain the real identities of their users. They also require deepfake content to be appropriately tagged to avoid “any confusion.”
The Chinese government has warned that deepfakes present a “danger to national security and social stability.”
Graphika’s report said the two Wolf News anchors were almost certainly created using technology provided by the London-based AI startup Synthesia.
The website of Synthesia, which did not immediately respond to AFP’s request for comment, advertizes software for creating deepfake avatars “based on video footage of real actors.”
The landing page of the Synthesia website showing one of the AI avatars used by Wolf News. Photo: Screenshot.
Graphika said it discovered the deepfakes while tracking pro-China disinformation operations known as “spamouflage”.
“Spamouflage is a pro-Chinese influence operation that predominantly amplifies low-quality political spam videos,” said Stubbs.
“Despite using some sophisticated technology, these latest videos are much the same. This shows the limitations of using deepfakes in influence operations — they are just one tool in an increasingly advanced toolbox.”
Support HKFP | Code of Ethics | Error/typo? | Contact Us | Newsletter | Transparency & Annual Report
China Uses AI Deepfake avatars as ‘news anchors’ to spread disinformation
]
A new report by a New York-based cyber research firm establishes how pro-Chinese news anchors purportedly created using advanced Artificial Intelligence(AI) were used as part of a state-aligned information campaign.
By Bidisha Saha: Disinformation is real, as the people on your TV screens have started becoming fake, more aptly deepfake. A new report by a New York-based cyber research firm establishes how pro-Chinese news anchors purportedly created using advanced Artificial Intelligence(AI) were used as part of a state-aligned information campaign. This further amplifies the question, can we really trust what we see?
advertisement
A report by Graphika, a New York-based research firm, has uncovered a pro-Chinese campaign using digital avatars to promote the interests of the Chinese Community Party (CCP) while also bashing the U.S. government on issues like gun violence. It raises the concern previously highlighted by cyber experts that deepfake videos could further sever people’s ability to discern reality from online forgeries, potentially being misused to nettle a political scandal.
Also Read | Biden’s State of Union address: What he said on Chinese aggression, US economy and police brutality
Deepfakes are media generated or edited by Artificial Intelligence(AI) in a whammy attempt of persuasive counterfeit further blurring the lines of fact and fiction. If you have a bit of training in machine learning, there are open-source packages available online that allow the creation of avatars of fictitious people, thus making the process of creating fake identities and spreading false information much easier.
This technology has the potential to undermine the credibility of almost everything we see online. From mobile applications that can transpose people’s faces in blockbuster movies to fake statements made by public figures like Barrack Obama, Elon Musk, Mark Zuckerburg etc. - it has the ability to create ‘digital puppets’.
WHAT DOES THE GRAPHIKA REPORT REVEAL?
The report states that Graphika observed instances of “Spamouflage,” a pro-Chinese influence operation used to promote “content that included video footage of fictitious people almost certainly created using artificial intelligence techniques.” It identified two anchors from a so-called news outlet Wolf News to plant politically motivated content, distributed by pro-China bot accounts on Facebook and Twitter, incepting a new chapter in the information war.
Also Read | Not only US, Chinese spy balloons targeted India, Japan: Report
According to Graphika, the two videos were posted at least 5 times by five accounts between Nov. 22 and Nov. 30, 2022. The posts were then re-shared by at least two more accounts, which appeared to be part of a pro-China network.“Additionally, none of the identified Spamouflage videos received more than 300 views, reflecting this actor’s long-standing challenges in producing convincing political content that generates authentic online engagement.”
Following this up, a new video was also discovered by Graphika using the same unidentified actors and the same A.I. tool to create videos targeting online conversations in Burkina Faso.
HOW DID DIGITAL PUPPETS MARKET DISINFORMATION CAMPAIGNS?
In one video, the digital avatar of a male news anchor with almost static dark hair, and a beard that is stubbly outlined narrates the United State’s lack of action on gun violence in a robotic voice with pixelated faces reading a monotonous script. Another video shows a female news anchor with oddly stagnant hair praising China’s role in geopolitical relations at the G20 international summit meeting.
Initial observation Despite featuring lifelike AI-generated avatars, the disinformation videos were low-quality and spammy in nature. The voices of the anchors were computerized, stilted and failed to sync with the mouth movements. The characters' faces seemed electronically generated, and pixellated and their hair also looked unnatural. The “Spamouflage” video ranges from one and a half to three minutes in length and uses a compilation of stock images and news footage from online sources.
Investigation Using reverse image search techniques, a wide array of promotional videos surfaced that were unrelated to China but used the same male and female presenters as the Wolf News content. Further in one marketing video of freight broker services, the male Wolf News anchor says: “Hello, my name is Mr Cruise. And I’m an avatar.” On Synthesia’s website, the female avatar is listed as “Anna” and the male avatar is named “Jason.”
Graphika linked the two fake Wolf News broadcasters to digital characters created by Synthesia, a five-year-old A.I. video creation start-up. The company sells over 65 digital avatars to choose from different genders, ages, ethnicities, voice tones and fashion choices in 120 languages and accents to clients which range from multinational companies like Reuters, Accenture, Amazon etc.
advertisement advertisement The entire process is “as easy as writing an email,” Synthesia writes on its website. The software, which costs as little as thirty dollars a month, produces videos in minutes that are mostly used for education, marketing, human resources and training. It cuts the massive costs of video production with huge pieces of equipment to just your laptop screen. RISING THREATS FROM DEEPFAKE TECHNOLOGY On Synthesia’s website, the company states that for ethical reasons, it “will not offer our software for public use” and that “all content will go through an explicit internal screening process before being released to our trusted clients.” It also says “political, sexual, personal, criminal and discriminatory content is not tolerated or approved.” advertisement Due to the wide range of videos of celebrities available on publicly accessible platforms, deepfakes involving high-profile figures become constant targets of attacks and are an increasing threat to cajoling civil disorder. Mike Beck, who is the Global Head of Threat Analysis at a cyber security company called Darktrace, says “We are at the crossroads when it comes to regulation and access to technology.” He adds, “ We are in a dangerous position given the access to computing power, giving developers access to open AI systems, we’re in a place where actually deepfakes are genuine things, they are not farfetched anymore. There is a big gap that governments need to fill but they are very very slow to do this because this is an emerging technology,” in a report to an American news channel. But, deepfake videos have been proliferating for years. Threat actors leverage the use of these misleadingly edited videos showing an individual executing fabricated actions or speeches, with the intent of conducting political scams.
— ENDS —
China used AI-generated news anchors to propagandize political content on social media: Report
]
Beijing [China], February 9 (ANI): China has used artificial intelligence-generated news anchors to propagandize political content on social media, Radio Free Asia (RFA) reported.
According to the US-based research firm Graphika’s report titled ‘Deepfake It Till You Make It’, China has since late 2022 used Spamouflage, a state-aligned influence operation to promote China’s global role and spread disinformation against the United States, the RFA reported.
A combination of Spam and Camouflage, Spamouflage, is a tactic used by spammers to evade email spam filters by replacing certain letters with numbers. Cyber security researchers use the term, as well as other terms such as Dragon bridge or Spamouflage Dragon, to refer to the pro-Chinese propaganda network on the internet.
According to RFA, Spamouflage is responsible for posting, via fake accounts targeting social media users, thousands of assets that praise China, criticize the United States, and attack the Hong Kong pro-democracy movement as well as Taiwan’s independence.
Spamouflage has recently been promoting a new and distinctive form of video content on social media platforms including Facebook, Twitter, and YouTube, according to Graphika.
The firm’s researchers spotted and analyzed two “unique” videos featuring a male and a female anchor, both speaking English and appearing to be Caucasian. The videos used the logo of a “likely fictitious media company” called ‘Wolf News’ with the accompanying slogan “Focus on hot spots and broadcast in real-time.”
In one video, the male anchor is heard criticising the US government for its “hypocritical repetition of empty rhetoric” in tackling gun violence. In the other video, the female anchor is seen talking about the importance of China-US cooperation for global economic recovery.
Both videos mirrored “past Spamouflage efforts to pass as legitimate news outlets,” said the Graphika report. The abundant similarities spotted in the videos and other Spamouflage assets on the internet led to the conclusion about the connection between the two, according to RFA.
China is, however, yet to respond to the Graphika report which points to the alarming potential misuse of sophisticated AI technology in disinformation campaigns by Chinese state actors.
China has been orchestrating cyber attacks on both allies and competitors, the Indo-Pacific Centre for Strategic Communications (IPCSC) recently reported, adding that the countries affected include Russia, the US, Iran, and its latest target, South Korea.
According to the report, there are growing concerns about which countries will be targeted next. Further, according to the report, a Chinese hacking group had launched a cyberattack on 12 Korean academic institutions.
Citing the Korea Internet Security Agency (KISA), the report said the attacked institutes included Korea Research Institute for Construction Policy, some departments of Jeju University, and the Korea National University of Education. The affected websites remained inaccessible for four days. (ANI)