A recent study by scientists at the University of North Carolina at Chapel Hill has highlighted the problems of removing sensitive data from large linguistic models (LLMs) such as OpenAI's ChatGPT and Google's Bard. The research indicates that there is no universal method for removing data from a large pre-trained linguistic model.

LLMs are pre-trained on databases and then tuned to generate consistent results. Once a model is trained, its creators cannot go back to the database and delete specific files to prevent the model from generating related results. Essentially, all of the information that a model is trained with exists somewhere within its weights and parameters, where they are undefinable without actually generating results.

This becomes problematic when LLMs trained on massive data sets generate sensitive information such as personally identifiable information, financial records, or other potentially harmful/unwanted results. In a hypothetical situation where an LLM has been trained on sensitive banking information, for example, there is usually no way for the AI creator to find those files and delete them.

The researchers were able to develop new defense methods to protect LLMs from some "extraction attacks," i.e., deliberate attempts by malicious actors to use inducement to bypass a model's protection barriers in order to make it produce sensitive information. However, as the researchers write, "the problem of sensitive information removal can be one where defense methods are always playing catch-up with new attack methods."