With AI increasingly present, it is normal for apps to start using this technology, even if for less specific purposes. This is what has now been discovered on the App Store, with 3 apps being removed by Apple, because they were allowing nudes to be generated using AI.
Apple removed 3 dangerous apps from the App Store
Apple has a reputation for being very picky about the apps it has in the App Store. The approval process for these is extensive and time-consuming, to ensure that only those that comply with the defined rules are authorized, with no room for abuse or abnormal situations for users.
Even though this process is ongoing, Apple removed 3 apps from the App Store that clearly violated its rules. These could be used to generate nude images by users, simply by uploading the images they wanted to change.
Although the images do not represent actual nudity, these proposals managed to generate images that could be used for harassment, blackmail and invasion of privacy. The discovery of these apps was made by 404 Media, which quickly reported them to Apple. The idea was for the company to evaluate compliance with the rules and its presence in the App Store.
Malicious apps could generate nudes using AI
Despite having taken the correct measures, the truth is that Apple should have acted sooner. Surprisingly, these apps have been available on the App Store since 2022, advertising the “undressing” feature on adult websites. The complaint also suggests that apps were initially allowed to remain on the App Store if they removed ads from adult sites.
However, one of the apps displayed ads until 2024, when Google removed it from the Play Store. It is worrying to see the reactive nature of iPhone app store moderation and the potential for programmers to exploit these types of flaws in the imposed rules.
This situation happens just a few days before Apple presents iOS 18 and its (supposed) AI features. Image generation has been one of the ways used to explore this technology, although with well-defined rules regarding its use.
