Openai Private Safety Processing Track Ai Misuse

As AI models grow more powerful, fueling concerns over their potential misuse, OpenAI appears to be walking a tightrope between protecting enterprise customers’ privacy and keeping tabs on how its technology is being used.

The ChatGPT-maker on Wednesday, August 19, previewed a new system called Private Safety Processing that is designed to identify patterns across user interactions without giving OpenAI staffers access to the underlying content.

Private Safety Processing is built on OpenAI’s existing Zero Data Retention (ZDR) systems for its eligible API (Application Programming Interface) customers. These ZDR safety systems are already capable of evaluating each interaction individually.

However, newer risks posed by AI models capable of handling more complex, long-horizon tasks appear to have prompted OpenAI to look for better ways to catch such interactions without compromising customer privacy. “Private Safety Processing is designed so we can continue to offer ZDR,” OpenAI said. It is currently being tested with a select group of customers.

The AI startup further emphasised that customer content remains unavailable to OpenAI personnel for review, and enterprise customer data is not used to train its models unless customers explicitly opt-in.

Private Safety Processing extends the scope of the automated protections already used in ZDR deployments.

The ZDR or Zero Data Retention policy involves using AI agents within the OpenAI API to monitor for abuse on a per session basis. Doing this ensures that customer data is not retained by companies while still enabling them to scan for bad activity without the need for human intervention. Under ZDR, AI interaction data remains on customer-controlled infrastructure.

OpenAI said it will also introduce an option where the interaction data is encrypted and stored on OpenAI infrastructure, with keys controlled by the customer.

Private Safety Processing extends the scope of the automated protections already used in ZDR deployments. Regardless of where it is stored, retained customer data will be scanned by OpenAI’s automated systems to identify patterns in a privacy-preserving manner.

Even when the automated systems flag an interaction, OpenAI personnel will not be able to access the customer content, as per the AI firm. Instead, OpenAI said it will receive a ‘narrowly defined signal’ indicating the type of activity involved. This signal will be used to determine whether OpenAI needs to act on it.

There are not many details on how this system will work, including how it will handle false positives. OpenAI has only said the following: “Customers can investigate alerts and enforcement decisions using information available in their own systems. If they want to appeal, clarify legitimate activity, or support an investigation into verified abuse, they can choose to share relevant information with OpenAI.”

The preview comes a day after OpenAI announced its decision to halt the development of its most powerful, unreleased models, codenamed Astra, for a little more than two weeks. It is the first time OpenAI has made such a move, which follows a remarkable security incident involving Hugging Face, the popular platform where developers host AI models.

Two unreleased OpenAI models escaped the sandbox of an internal cybersecurity evaluation and compromised Hugging Face’s production systems. Other major AI providers such as Anthropic, Meta, and China’s Moonshot AI have reported similar incidents in recent weeks.

The string of hacking incidents has spurred a clamour for safety guardrails that can stop such abuse from happening.

OpenAI’s privacy-centric safety approach to monitor for AI misuse comes as it gears up for an anticipated IPO amid a highly competitive race with arch-rival Anthropic to win the business of enterprise customers. Recent reports indicate that OpenAI’s Q2 grew more slowly than Anthropic, whose annualised revenue run rate is now reportedly $65 billion.

Anthropic is also working on its IPO, with investors suggesting that it could be valued at $2 trillion when it goes public.

In terms of data retention, Anthropic’s updated policy in July has aggravated some customers as it enables the Claude-maker to keep user data – including chat sessions and conversations – for a period of 30 days but only when it comes to covered models like its Mythos-class models.

Human review of customer data can occur but only “through a controlled access path” that involves “a small set of approved reviewers.” Every one of those review sessions is “recorded in a tamper-proof log that reviewers cannot suppress or modify,” Anthropic said.

However, enterprises that handle large volumes of sensitive data are reportedly uneasy about Anthropic’s policy. OpenAI, meanwhile, appears to be moving in the opposite direction, potentially positioning itself as an alternative for customers that want stronger privacy safeguards.