The Dark Side of Open Source AI Image Generators

Posted on:
Key Points

The way that open source works means its going to be pretty hard to stop someone from potentially hijacking that, says Ajder.. 4chan, the image-based message board site with a reputation for chaotic moderation is home to pages devoted to nonconsensual deepfake porn, WIRED found, made with openly available programs and AI models dedicated solely to sexual images..

Other AI creators, including Cohen, are also concerned about the ease of creating deepfakes with a new method, InstantID, published in January by researchers at Peking University and Chinese social media company Xiaohongshu, which can be used to swap faces in images using just a single example, therefore needing less processing or preparation..

Researchers at Hugging Face, an online community and platform for open source models, have been promoting ethical tools for AI, including image guarding, which they say protects images from generative AI editing, as well as allowing developers to control access to models uploaded to the platform..

"Setting norms in a community is an undervalued and often powerful way of influencing behavior and what is considered acceptable, cool, and not cool, Widder says.. Sharing AI-generated intimate images without consent was made illegal in the UK in online safety legislation enacted in January, and the Swift scandal has added new fuel to calls for similar, federal, laws in the US..

Elena Michael, director of NotYourPorn, a UK group campaigning against image-based sexual abuse, suggests more dialog between AI startups, open source developers, and entrepreneurs as well as governments, womens organizations, academics, and civil society to explore possible deterrents to nonconsensual AI porn that dont hinder accessibility to open source models..