In an ongoing loneliness epidemic, the rise of AI chatbot companions and romantic partners might be meeting some people’s needs, but researchers found these bots aren’t the best of friends when it comes to protecting secrets.
*Privacy Not Included, a consumer guide from Mozilla Foundation that evaluates privacy policies for tech and other products, reviewed 11 chatbots marketed as romantic companions and found that all of the chatbots earned warning labels, “putting them on par with the worst categories of products we have ever reviewed for privacy.”
Among the privacy issues *Privacy Not Included found when reviewing the bots were a lack of user privacy policies and information about how the AI companions work, as well as Terms and Conditions saying companies were not responsible for what could happen when using their chatbots.
“To be perfectly blunt, AI girlfriends are not your friends,” Misha Rykov, a researcher at *Privacy Not Included, said in a statement. “Although they are marketed as something that will enhance your mental health and well-being, they specialize in delivering dependency, loneliness, and toxicity, all while prying as much data as possible from you.”
For example, CrushOn.AI, which markets itself as a “no filter NSFW character AI chat,” says under its “Consumer Health Data Privacy Policy” it “may collect” information on a user’s “Use of prescribed medication,” “Gender-affirming care information,” and “Reproductive or sexual health information,” in the character chats “to facilitate” and “monitor” the “chat for safety and appropriate content.” The company also said it “may collect voice recordings” if users leave a voicemail, contact customer support, or connect with the company over video chat. CrushOn.AI did not immediately respond to a request for comment from Quartz.
“To be perfectly blunt, AI girlfriends are not your friends.”
RomanticAI, a chatbot service which advertises “a friend you can trust,” says in its Terms and Conditions users must acknowledge they are “communicating with software whose activity we cannot constantly control.” RomanticAI did not immediately respond to a request for comment.
*Privacy Not Included found that 73% of the bots it reviewed shared no information on how the company manages security issues, and 64% didn’t have “clear information about encryption” or even if the company uses it. All but one of the chatbots either mentioned selling or sharing user data or didn’t include information on how it uses user data. The researchers found less than half of the chatbots allowed users the right to delete personal data.
A day after OpenAI opened its GPT store in January, which allows anyone to make customized versions of its ChatGPT bot, Quartz found at least eight “girlfriend” AI chatbots after a search for “girlfriend” on the store. (We also quickly decided AI girlfriend chatbots wouldn’t last.) OpenAI actually bans GPTs “dedicated to fostering romantic companionship or performing regulated activities,” showing that, alongside privacy issues, companionship bots might be difficult to regulate overall.
Jen Caltrider, director of *Privacy Not Included, told Quartz in a statement the companies behind the chatbots “should provide thorough explanations of if and how they use the contents of users’ conversations to train their AI models” and allow users to have control over their data, such as by deleting it or opting out of having their chats used to train the bots.
“One of the scariest things about AI relationship chatbots is the potential for manipulating their users,” Caltrider said. “What is to stop bad actors from creating chatbots designed to get to know their soulmates and then using that relationship to manipulate those people to do terrible things, embrace frightening ideologies, or harm themselves or others? This is why we desperately need more transparency and user control in these AI apps.”
Disclaimer
The information contained in South Florida Reporter is for general information purposes only.
The South Florida Reporter assumes no responsibility for errors or omissions in the contents of the Service.
In no event shall the South Florida Reporter be liable for any special, direct, indirect, consequential, or incidental damages or any damages whatsoever, whether in an action of contract, negligence or other tort, arising out of or in connection with the use of the Service or the contents of the Service. The Company reserves the right to make additions, deletions, or modifications to the contents of the Service at any time without prior notice.
The Company does not warrant that the Service is free of viruses or other harmful components
This article originally appeared here and was republished with permission.
