By Phumzile van Damme – republished from ISS African Futures

How can we safeguard South Africa’s hard-won free and fair elections in the age of disinformation online? Phumzile Van Damme weighs in.

On 9 March 2024, Duduzile Zuma-Sambudla, the daughter of South Africa’s former president, Jacob Zuma, shared a video on X, formerly known as Twitter. In it, the former US president Donald Trump is seen saying:

Greetings all South Africans. My name is President Donald Trump. I urge all South Africans to vote for uMkhonto WeSizwe on 29 May. The African National Congress of Cyril Ramaphosa has failed all South Africans. With this new black party, led by President Jacob Zuma, all South Africans will matter.

For those acquainted with deepfake technology and South African politics, the video’s fabricated nature was evident, with many South African X users declaring it fake. However, a subset of users remained uncertain of its legitimacy, prompting questions of its authenticity.

What the Trump deepfake revealed was the remarkable sophistication of content generated by AI. While the visual aspects of the video were shoddy, Trump’s distinct voice, syntax and tone, as well as the skilful incorporation of pronunciation errors typical in non-native speakers pronouncing African-language words, make the audio aspect of the video seem authentic. The video was allegedly made using Parrot AI. This celebrity voice generator allows users to select a celebrity avatar (including prominent global leaders), input a script, and produce a video with the chosen celebrity seemingly uttering the scripted words. It also has the option to remove the “made with Parrot” watermark.

Given the high level of authenticity, particularly at the audio level, it is likely that the knowledge of the improbability of Trump advocating for the support of a specific party in South Africa played a more significant role in the deepfake’s declaration as inauthentic rather than the tell-tale signs of deepfakes.

While it may be of some comfort that many South African X users declared the video inauthentic, the person who first shared it did not. Zuma-Sambudla is a prominent leader in the recently emerged Jacob Zuma-led uMkhonto WeSizwe (MK) party, which has shared violent rhetoric, including threatening violence if the 29 May election does not go its way.

Zuma-Sambudla’s role as a disinformation purveyor emerged during South Africa’s July 2021 unrest, initially triggered by her father’s arrest. ‘Let it burn,’ she tweeted, as one of the main accounts celebrating the violence and looting and aiding in the amplification of incendiary posts and videos.

The 2023 warnings about generative AI becoming a significant vector for disinformation now have evidentiary basis as the 2024 super-election year intensifies

The destructive role of social media posts such as those by Zuma-Sambudla in fanning the flames of violence became central in what was the worst public violence witnessed in South Africa since the end of Apartheid. It left over 300 people dead, many more injured, and the economy deeply wounded as businesses and infrastructure were subjected to widespread destruction and looting. What was clear, and echoed in a statement by President Cyril Ramaphosa, was that the intention had been to ‘severely weaken – or even dislodge – the democratic state.’

The results of an investigation by South Africa’s Human Rights Commission (SAHRC) released in January 2024 further emphasised the role of social media in exacerbating violence. The SAHRC found that:

Social media platforms, whether open or closed, were utilised by individuals and specific groups to organise and aid in the spread of the unrest. These forums served to advance various agendas during the July unrest, including establishing collaborative networks, strategically spreading misinformation and disinformation, and mobilising and celebrating looting and violence.

In recent times, Zuma-Sambundla’s role as a disinformation propagator has become concretised. Shortly after the invasion of Ukraine in early 2022, the #IStandWithPutin and #IStandWithRussa campaigns launched on X. South African influencers played a notable role in these campaigns. A study by the Centre for Information Resilience (CIR) in partnership with South African counter-disinformation experts investigated the campaigns, and found that Zuma-Sambudla acted as a ‘super influencer’ – the most retweeted and mentioned user in the campaign. The CIR concluded that ‘regardless of intent, the Sambudla-Zuma account [was] the main amplifier’ of that campaign.

It is, therefore, within this context that the sharing of the Trump AI-generated deepfake must be understood. It was not shared on X by an anonymous account, or an inconspicuous user, but a key player in South Africa’s disinformation landscape.

The 2023 warnings about generative AI becoming a significant vector for disinformation now have evidentiary basis as the 2024 super-election year intensifies. Evidence of its use has surfaced in pre-election campaigns across countries like Bangladesh, the United States, Indonesia, and others. With South Africa poised to be the first African nation to hold elections this year, its experience may serve as a crucial indicator of the broader impact of AI-driven disinformation on the continent’s democratic processes.

It must, however, be noted that, unlike many countries on the African continent, South Africa has resilience factors against disinformation, such as press freedom, an independent judiciary, and free and fair elections.

WhatsApp is a highway for the swift transmission of AI-generated disinformation, which is spread rapidly without the guardrails of content moderation

Some conceptual issues around disinformation in South Africa and the rest of the continent need clarification for both local and international audiences. As noted by Admire Nare and co-authors, most studies on information disorders in sub-Saharan Africa have concentrated on the motivations behind spreading false information online, frequently using a Western gaze as an analytical and theoretical toolkit while missing local specificities and nuances.

In the region and South Africa, the surge of disinformation can largely be attributed to under-resourced newsrooms grappling with an ever-changing media environment. With news outlets putting up paywalls to survive, access to reliable, verified news is becoming cost-prohibitive for many consumers. As a result, those who cannot afford access to pay-walled news often resort to consuming unverified viral content and disseminating misinformation, often without ill intent. In many cases, there’s a sincere intention to share the news with friends and family.

Another misconception in South Africa and the continent more broadly is that, because of low internet penetration rates and social media usage, generative AI disinformation is not a big threat. This is not the case. .

As of January 2024, South Africa had an estimated 45.34 million active internet users.

Considered one of the most internet-addicted countries in the world, the country’s internet users spend an estimated average of over 9 hours on the internet per day, with the majority doing so via mobile devices. Social media usage is equally high at an estimated 26 million users, or 44% of the population.

Due to its almost universal usage, WhatsApp has become South Africa’s primary mode of communication. In rural areas and villages with typically lower internet penetration rates, WhatsApp serves as more than just a messaging platform. It provides access to information, particularly in higher education, healthcare, and reproductive health. It has also become a significant portal for disseminating and consuming misinformation.

It is, therefore, within this unique information landscape that potential perils posed by generative AI in South Africa must be understood. The country has a high internet penetration rate, with a whopping 93% of internet users predominantly relying on WhatsApp for daily communication. This makes the app the nation’s favoured digital gathering place, and sometimes unintentional spreader of disinformation. WhatsApp is quite likely a highway for the swift transmission of AI-generated disinformation shared by super influencers such as Zuma-Sambudla, which is spread rapidly without the guardrails of content moderation.

Whether MK or any other parties will go on to share generative AI disinformation remains to be seen. What is certain is that there is an urgent need for innovative approaches to digital literacy and generative AI content verification to safeguard the integrity of South Africa’s 2024 election.

ABOUT PHUMZILE VAN DAMME

Phumzile van Damme is a South African specialist in counter-disinformation strategy and operations, platform accountability, advocacy and digital rights.

She is a Technology and Human Rights Fellow at Harvard University, examining the human rights ramifications of emerging technologies, with a particular focus on Artificial Intelligence. Ms Van Damme has received global recognition for her work including the 2022 Vital Voices Global Leaders Award for her work against online harms in Africa. She was formerly a member of parliament in South Africa, she works with USAID and the Centre for Information Resilience to develop strategies and the Centre for Information Resilience to develop strategies to combat disinformation.