Heejin's phone received a Telegram message from an unidentified source last Saturday. "Your personal information and photos have been exposed." Let's talk about it.
The university student saw a picture of herself that was taken when she was still a student a few years ago when she entered the chatroom to read the message. The identical photo was used in the next image, but it was a phony and sexually graphic one.
Sickened, The pictures didn't stop coming even though
Heejin—not her real name—did not reply. Using advanced deepfake technology, her
face had been linked to a body performing a sexual act in each of them.
Artificial intelligence is rapidly being used to create deepfakes, most of
which comprise a genuine person's face combined with a fictitious, sexually
explicit body.
Heejin told the BBC, "I was terrified and felt so alone."
However, she wasn't by alone.
Journalist Ko Narin from South Korea broke what would become the biggest scoop
of her career two days earlier. Police were looking into deepfake porn rings at
two of the county's biggest institutions, it had recently come to light, and
Ms. Ko was certain there had to be more.
When she began her social media search, she found dozens of chat groups on the
messaging app Telegram where people were uploading pictures of women they knew
and instantly turning them into phony sexual images using artificial
intelligence (AI).
According to Ms. Ko, "people were requesting to have
photos of girls they knew turned into deepfakes uploaded every minute."
Ms. Ko found that these organizations weren't limited to pursuing college
students. There were even middle school and high school-specific rooms. A
individual student may even be assigned her own room if a significant amount of
content was produced utilizing her photographs. Generally referred to as
"humiliation rooms" or "friend of friend rooms," these
spaces frequently have access requirements.
The Hankyoreh newspaper's article by Ms. Ko stunned South
Korea. Following the example of French authorities who have recently charged
Telegram's Russian founder for crimes related to the service, police stated on
Monday that they were considering initiating an inquiry into the messaging app.
The president has urged for young men to receive better education, and the
administration has pledged to impose harsher penalties for those implicated.
In a statement sent to the BBC, Telegram stated that it "actively combats
harmful content on its platform, including illegal pornography."
"A methodical and
structured procedure"
The BBC has looked at some of these chatrooms' descriptions. One requires users
to share more than four images of a person along with their name, age, and
place of residence.
Ms. Ko remarked, "I was astounded by how organized and
methodical the process was." "The most horrifying thing I found was a
group at one school with over 2,000 members for underage students."
Following the publication of Ms. Ko's piece, advocates for women's rights began
searching Telegram as well and following leads.
Over 500 colleges and institutions had been designated as targets by the
conclusion of that week. Although the precise number affected is unknown, it is
thought that many of them are younger than 16, which is South Korea's
consenting age. The majority of those who are suspected of committing these
crimes are minors.
Heejin claimed that her worry had increased after realizing
the scope of the situation, as she was now concerned about the potential number
of people who had seen her deepfakes. She started by blaming herself. "I
kept thinking, Should I have been more careful? Did this happen because I
uploaded my photos to social media?"
Since then, a large number of women and teenagers around the nation have
deleted their social media posts and/or completely deleted their accounts out
of fear that they would be the next victim of abuse.
Ah-eun, a university student whose colleagues have been singled out, expressed
her frustration and anger, saying, "We are frustrated and angry that we
are having to censor our behavior and our use of social media when we have done
nothing wrong."
According to Ah-eun, a victim at her institution was
informed by the police not to pursue her case because it would be too difficult
to apprehend the offender and because the images were phony, the incident was
“not truly a crime."
Rooms can be turned to "secret" mode, users can remain anonymous, and
their contents can be swiftly erased without leaving a trace. This has made it
an ideal environment for illicit activity to proliferate.
Law enforcement and lawmakers reacted strongly last week,
pledging to look into these acts and prosecute those who commit them.
The Seoul National Police Agency declared on Monday that it would examine into
Telegram's role in facilitating the distribution of fictitious child
pornographic photos.
The creator of the app, Pavel Durov, was accused last week in France of helping
to spread child pornography among other offenses connected to the software.
However, because Korea has
already dealt with a similar situation, women's rights groups charge that the
government of South Korea has allowed sexual harassment on Telegram to fester
unchecked for far too long. It was discovered in 2019 that a sex ring was
forcing women and children to create and share sexually graphic photographs of
themselves over Telegram.
At the time, seven pleas from police asking for assistance with their
investigation were turned down by Telegram. Despite the fact that the leader of
the group received a sentence of over 40 years in prison, the platform remained
unaffected due to concerns about censorship.
Ms. Ko stated, "I think this has exacerbated the
situation. They sentenced the main actors but otherwise neglected the
situation."
After uncovering the Nth room sex-ring in 2019 while still a young student
journalist, Park Jihyun has since turned into a political advocate for those
who have been victimized by digital sex crimes. She claimed that students and
parents had been phoning her many times a day in tears ever since the deepfake
story surfaced.
"They are scared because they saw their school on the list that was posted
on social media."
In South Korea, Ms. Park has been at the forefront of calls
for the government to control or even outlaw the app. She stated, "The
state must regulate these tech companies to protect its citizens if they will
not cooperate with law enforcement."
The number of minors falling prey to deepfake pornography has sharply
increased, according to South Korea's Advocacy Centre for Online Sexual Abuse
victims (ACOSAV), even before the current crisis erupted.
They provided counseling to 86 adolescent victims in 2023. After just eight
months this year, it shot up to 238, according to that. Another sixty-four
adolescent victims have come forward in only the last week.
Park Seonghye, one of the center's leaders, stated that her
staff had received a ton of calls over the last week and had been working
nonstop. "We've been in a full-scale emergency, similar to a wartime
scenario," she remarked.
"There is so much more footage than there used to be, and we're worried
that it will only increase," said the person who developed the newest
deepfake technique.
The center finds damaging content and works with online platforms to have it
removed in addition to providing counseling to victims. According to Ms. Park,
there have been times when users have requested that content be deleted from
Telegram. She said, "So it's not impossible."
Telegram provided a response to the BBC explaining that its
moderators "remove millions of pieces of content every day that violate
Telegram's terms of service by proactively monitoring public parts of the app,
using AI tools, and accepting user reports."
Women's rights organizations acknowledge that victims can now be exploited more
easily thanks to new AI technology, but they contend that this is simply the
most recent instance of misogyny to occur online in South Korea.
First, there were online verbal abuse waves directed towards women. Then came
the spy cam epidemic, in which people were caught on camera utilizing locker
rooms and public restrooms.
In a joint statement, 84 women's organizations stated that
"gender equality is the solution and structural sexism is the root cause
of this."
This is a clear jab at President Yoon Suk Yeol of the nation, who has disproved
the existence of systemic sexism, slashed funds to victim advocacy
organizations, and is disbanding the government's gender equality ministry.
Young sex offender counselor Lee Myung-hwa concurred that while deepfake abuse
may have appeared to be on the rise suddenly, it had been simmering beneath the
surface for some time. The counselor at the Aha Seoul Youth Cultural Center
stated, "Deepfakes are now considered a game or prank by teenagers; they've
become part of their culture."
Ms Lee said it was paramount to educate young men, citing research that shows when you tell offenders exactly what they have done wrong, they become more aware of what counts as sexual abuse, which stops them from reoffending.
Meanwhile, the government has said it will increase the criminal sentences of those who create and share deepfake images, and will also punish those who view the pornography.
It follows criticism that not enough perpetrators were being punished. One of the issues is that the majority of offenders are teenagers, who are typically tried in youth courts, where they receive more lenient sentences.
Since the chatrooms were exposed, many have been closed down, but new ones will almost certainly take their place. A humiliation room has already been created to target the journalists covering this story. Ms Ko, who broke the news, said this had given her sleepless nights. “I keep checking the room to see if my photo has been uploaded,” she said.
Such anxiety has spread to almost every teenage girl and young woman in South Korea. Ah-eun, the university student, said it had made her suspicious of her male acquaintances.
“I now can’t be certain people won’t commit these crimes behind my back, without me knowing,” she said. “I’ve become hyper-vigilant in all my interactions with people, which can’t be good.”
0 Comments