A South Korean website called GenNomis went offline this week after a researcher made a particularly alarming discovery: tens of thousands of AI-generated pornographic images created by its software, Nudify. The photos were found in an unsecured database, and included explicit images bearing the likeness of celebrities, politicians, random women, and children.Jeremiah Fowler, the cybersecurity researcher who found the cache, says he immediately sent a responsible disclosure notice to GenNomis and its parent company, AI-Nomis, who then restricted the database from public access. Later, just hours after Wiredapproached GenNomis for comment, both it and its parent company seemed to disappear from the web entirely.GenNomis is far from the only AI startup peddling tools to generate pornography. It's a small part of a worrying trend enabled by unregulated generative AI across the world. Often known as "deepfakes" because of their lifelike nature, fake porn images and videos based on real people have exploded throughout the internet as consumers get their hands on ever-more convincing generative AI.The consequences of deepfake porn can be devastating, especially for women, who make up the vast majority of victims. Beside the obvious lack of consent when a person is digitally undressed, this stuff has been used to tarnish politicians, get people fired, extort victims for money, and generate child sexual abuse materials. Beyond sexual violence, non-pornographicdeepfakes are responsible for a huge increase in financial and cyber crimes and no small amount of blatant misinformation.It's also no surprise that GenNomis is based out of South Korea. A 2023 report on Deepfake porn found that South Korean women made up 53 percent of individuals victimized by the practice by far the most targeted group. For comparison, the US women made up the second most targeted group, ringing in at 20 percent.The rise of generative AI enabling the rampant exploitation of women coincides with a meteoric rise in sexist rhetoric and gender-based violence in South Korea, as reactionary politicians and influencers blame feminism for the rising rate of male suicide.Overall, it's a strong argument for lawmakers to take a tougher approach to regulating generative AI, though this seems unlikely due to the AI industry's current freedom to regulate itself. For comparison, China has mandated that all AI-generated media be labeled as such from the drop. Though slower to the party, western lawmakers are catching up on criminalizing deepfake porn creation and distribution, though laws and penalties vary from state to state in America.Still, for thousands of women around the world, the fact that companies like GenNomis existed at all means it's too little, too late.More on AI ethics: All AI-Generated Material Must Be Labeled Online, China AnnouncesShare This Article