Snap might have to withdraw its AI chatbot, watchdog says

The UK's data watchdog has told Snapchat it might have to stop offering its generative AI chatbot My AI.

An initial probe into the company suggested a "worrying failure" by parent company Snap over potential privacy risks, especially to children.

However, the Information Commissioners Office (ICO) stressed that its findings were "provisional".

Snap said it was "closely reviewing" the ICO's decision and would "work constructively" with the watchdog.

The ICO has sent a preliminary enforcement notice to the tech giant. At this stage the notice is a signal to Snap to ensure My AI complies with data protection rules which includes the Children's Design Code.

The watchdog's code contains 15 standards that online services need to follow. This ensures they are complying with their obligations under data protection law to protect children's data online.

The ICO said that if a final enforcement notice was to be adopted, Snap might not be able to offer the My AI function to UK users until the company carries out "an adequate risk assessment".

My AI is – according to the company – "an experimental and friendly" chatbot designed to be a personal sidekick to each Snapchatter who chats with it.

Powered by OpenAI's ChatGPT, it can be used as an assistant to plan day trips or create menus.

The feature has more than two million chats per day happening on the app, according to Snap's boss Evan Spiegel.

It was made available to all Snapchat users in April, after being launched for a fee in February.

Since then, the social media platform said "a lot of progress" had been made in its capabilities although it admitted that "mistakes may occur".

"My AI may answer incorrectly, provide biased answers or note it is unsure of the answers so don't rely on its advice", the company said.

Image source, Getty Images

Snap has also been criticised for being unclear over whether the chatbot can access private information such as location data.

"Snapchat can only ever access your location if you consent to share it," the firm said.

One of the other concerns about My AI is – because of how young users of Snapchat skew – whether they really understand the implications of data collection.

"Privacy is a foundational value for us – it is critical to our core use case of helping people visually communicate with their friends and family," the platform stressed.

Information Commissioner John Edwards said, "The provisional findings of our investigation suggest a worrying failure by Snap to adequately identify and assess the privacy risks to children and other users before launching My AI.

"We have been clear that organisations must consider the risks associated with AI, alongside the benefits.

"Today's preliminary enforcement notice shows we will take action in order to protect UK consumers' privacy rights."

A Snap spokeswoman said: "We are closely reviewing the ICO's provisional decision.

"Like the ICO, we are committed to protecting the privacy of our users.

"In line with our standard approach to product development, My AI went through a robust legal and privacy review process before being made publicly available.

"We will continue to work constructively with the ICO to ensure they're comfortable with our risk assessment procedures."

Snap will now have a chance to respond to the regulator's concerns before the ICO makes its final decision.