Muthuvel Karunanidhi, an iconic Indian actor-turned-politician, made a surprising appearance in January ahead of the Indian election.
Clad in his trademark black sunglasses, white shirt and yellow shawl, he is seen in a video congratulating a friend and fellow politician on the launch of their autobiography.
In the eight-minute speech, the patriarch of politics in the southern state of Tamil Nadu also took the opportunity to praise the stable leadership of MK Stalin, his son and the current leader of the state.
It’s a powerful endorsem*nt, especially considering Karunanidhi died in 2018.
Deepfakes are videos, images or audio clips made with artificial intelligence that mimic a person's likeness or voice.
While they can be used for fun, they can also be made to deliberately mislead people, which is what appears to be happening during the Indian election campaign.
In another video that surfaced in recent months, Bollywood star Aamir Khan is heard mocking India's ruling Bharatiya Janata Party (BJP) for failing to deliver a decade-old promise to deposit 1.5 million Indian rupees ($27,000) into the bank accounts of every Indian citizen.
It ends with his endorsem*nt of the opposition Congress Party.
The voice in the video resembles Khan's but has been artificially manipulated.
A spokesperson clarified that while the actor had raised electoral awareness through campaigns in the past, he has never promoted a specific political party.
Divyendra Singh Jadoun — who gained fame under the YouTube channel, The Indian Deepfaker — is no stranger to such content after doing work in film and advertisem*nts.
His firm Polymath Synthetic Media Solutions is one of many deepfake service providers catering to political parties and this year his team's been bombarded with requests.
"The first conversation was can you do a deepfake of an opponent political leader?" Mr Jadoun said.
Representatives of India's political parties have asked Mr Jadoun to manipulate audio of opposition candidates making gaffes during the campaign and superimpose their faces onto sexually explicit content.
He's even been asked by one party to create a low-quality fake video of their own candidate, which would be used to counter any damning real videos that emerge during the campaign.
Out of 200 requests he received, Mr Jadoun says the majority were unethical and rejected by his team.
"We won't be creating any content that is used to defame anyone or put [question] marks on some opponent leader," he said.
How AI-generated content can be ethically used in campaigns
Mr Jadoun's team creates AI-generated videos to help increase the reach of personal messages.
For example, he can shoot a 15-minute video with a party leader and use it to build an avatar that can deliver calls and video messages to hundreds of thousands of individual party workers.
The messages can be personalised to address everyone by name, and be delivered in any of the country's 22 languages.
"It's not possible for the party leader to address each and every party worker," Mr Jadoun said.
He says his team has worked with Prime Minister Narendra Modi's BJP as well as their main opponent, the Congress Party, and regional heavyweights on developing AI tools to help recognise the efforts of volunteers.
As a former student politician, the 31-year-old is used to giving rousing speeches in front of large crowds and travelling across his state, Rajasthan, to develop his network.
He knows what an effective campaign needs and says a candidate's chance of winning an election depends on the hard work of their party cadre.
"If you consider politics as a company, this is the only company in the world where its employees are working for completely free …," Mr Jadoun said.
"The only thing that they need is recognition from the particular party leader."
Technology has changed political campaigning
Mr Jadoun says when he first started in 2021, it would take him between seven to 12 days to make a low quality deepfake, which was one minute long.
Now the technology has advanced so rapidly, anyone can make one in minutes.
"Even if they have no knowledge of coding, there are so many websites," he said.
"They just have to put a single image and a video where they want to swap the face and it can create the deepfake video in just less than three minutes."
Political consultant Sagar Vishnoi, who pioneered the use of AI in Indian politics and worked on the country's first high-profile political deepfake back in 2020, says the technology has changed campaigning.
He said it has made it 50 times cheaper and estimates over the next five years, 80 per cent of campaigns will be driven by AI.
"Eight-hundred million people are connected to the internet and the data rates are so cheap," he said.
"Political parties have such good network and distribution channels within themselves, that they can reach out to more masses."
Mr Vishnoi, who runs workshops and awareness campaigns teaching law enforcement to fight deepfakes, says there is potential for serious misuse of the technology.
"[If] AI holds power to connect billions of people, it holds the power to create misinformation in 10 seconds," he said.
"It can even create riots or disturb the social fabric of the nation, by making political leaders speak about some religion or caste."
Women and marginalised groups are particularly vulnerable to deepfakes
Women and marginalised groups from conservative and religious countries are particularly vulnerable to deepfakes.
In Bangladesh, deepfake videos of female opposition politicians — including Rumin Farhana in a bikini and Nipun Roy in a swimming pool — undermined their campaigns when they emerged ahead of general elections in January this year.
The content seeks to change the perception of the voter, and specifically the voter's psychology, according to Mr Vishnoi.
The greatest challenge in countering unethical deepfakes is confirmation bias, according to Jaskirat Singh Bawa, global head of operations for fact-checking organisation Logically Facts.
"It is very difficult to change the mind of somebody who is willing to believe a lie as long as it furthers their own beliefs," he said.
"Right now we have a very heated election season going on, where a lot of individuals, political entities, parties, as well as, possibly even foreign actors stand to gain from the the discourse becoming very toxic and very contentious."
Mr Bawa said the claims he has come across were usually about attributing malice and anti-national sentiments to members of the opposition party.
But he said all parties spread disinformation.
"When it comes to supporters of any particular political ideology, as long as there's information — however false it may be — that conforms to their biases, they are willingly spread[ing] it," he said.
"It just so happens right now that the power equations are in favour of the ruling dispensation, which automatically leads to more and more people sharing more information that is getting endorsed by, let's say, supporters of the ruling party."
Divyendra Jadoun said there are a few ways to spot a deepfake, including checking the hairline of the person in the photo or video, paying attention to the movement around the eyes, or looking for any strange shadows.
But he said there was no substitute for intuition.
"Our instinct is better than any detection algorithm that is out there," Mr Jadoun said.
"If you look closely, we get to see it's a deepfake. But the issue is that people want to believe what they want to believe," he said.
