ChatGPT & Data Privacy: What are the implications of using the chatbot?

The supervisory authorities in Germany are currently investigating data protection issues related to ChatGPT. The trigger was a decision by the Italian data protection authority GPDP. The data protection authority found that there is no legal basis for collecting user data from "conversations" with the chatbot.

Furthermore, users are not sufficiently informed about the processing of their data, and adequate protection for the processing of data of minors is not ensured. In addition, there was reportedly a data breach affecting data from "conversations" as well as payment information. The authority subsequently prohibited the use of the chatbot in Italy.

We summarize the status quo in Germany and the most important facts about ChatGPT and data protection.

What is ChatGPT and who owns it?

ChatGPT stands for Chatbot Generative Pre-trained Transformer and is a language-based application. This means users can communicate with ChatGPT via text input and have answers generated for their questions.

How ChatGPT works

The chatbot is based on the deep learning principle, a subfield of machine learning. Simply put, behind ChatGPT is an artificial neural network that functions similarly to the human brain and can learn to understand texts and make certain decisions.

With each conversation, the chatbot learns more and is thus able to provide human-like answers.

ChatGPT is the chatbot of the former research company OpenAI LP, which now operates as a for-profit entity and is controlled by the non-profit organization OpenAI Inc. The company was funded, among others, by Microsoft and Elon Musk. Musk has since distanced himself from OpenAI. Supporters include Amazon Web Services, Infosys Technologies, and co-founders of LinkedIn and PayPal.

ChatGPT & Data Protection: What's the Problem?

To continuously improve, ChatGPT accesses millions of texts and information freely available on the internet. Additionally, the AI uses user input from now Individuals and businesses for training.

It cannot be ruled out that personal and sensitive information is processed in this context – as happened in Italy, where in 2023 a data privacy incident led to the disclosure of personal information in other people's chats.

A look back: Why did Italy ban ChatGPT?

After the data protection violation in Italy came to light, the country's data protection authority accused OpenAI, among other things, of violating the data protection principles of purpose limitation and data minimization, processing data without a legal basis, and not adequately ensuring the rights of data subjects.

For the ban on data processing via ChatGPT in Italy to be lifted again, the supervisory authority demanded a series of measures from OpenAI. The provider had to:

• publish a data privacy notice that, among other things, describes the logic of the data processing required for ChatGPT's operation
• introduce age verification
• establish a clear legal basis, whereby, from the perspective of the Italian authority, only user consent or a legitimate interest would be considered
• take measures to enable the exercise of data subject rights, such as the right to erasure or access to data – even for non-users
• educate Italian citizens about data processing for AI training purposes in an information campaign via radio, television, newspapers, and the internet

On April 28, 2023, ChatGPT was allowed to be used in Italy again. According to the local supervisory authority, OpenAI restored the service in Italy with improved transparency and enhanced rights, in accordance with data protection requirements.

How do German data protection experts view ChatGPT?

Some data protection experts in Germany at the time shared the concerns of the Italian supervisory authority and called for similar measures. The responsible state data protection supervisory authorities require extensive information for a review.

Due to its complexity, however, the data protection assessment of Artificial Intelligence was a challenge. For example, a review of ChatGPT's data protection lacked

• Information about the data sources
• Information about the algorithms behind automated data processing and
• clarity on whether data is shared with third parties with commercial interests.

In 2023, German data protection authorities extensively addressed ChatGPT and data protection. In its activity report for 2023 , the State Commissioner for Data Protection of Lower Saxony provided a data protection assessment for ChatGPT and concluded, among other things, that data subject rights can only be implemented to a limited extent.

Key facts about ChatGPT and data protection

The basis for the data protection review was the General Data Protection Regulation (GDPR). If personal data is processed in ChatGPT, various data protection challengesarise. The following overview summarizes the most important ones.

Consent to data processing

The GDPR permits data processing if there is a legal basis according to Art. 6 para. 1 GDPR. For example, data processing must be necessary for the fulfillment of contractual obligations or based on a legitimate interest of the operator. In doing so, the protection interests of data subjects must not be violated.

Without another legal basis from Art. 6 para. 1 GDPR, processing may only occur with the consent of the data subject. This requires transparent information for the data subject about the data processing and its implications.

Since an AI like ChatGPT is considered a black box regarding data protection, companies generally cannot provide detailed information about data processing. Effective consent is therefore almost impossible.

Transparency regarding data processing

The GDPR stipulates that personal data must be processed in a comprehensible manner for the data subject. The lack of transparency regarding the functioning of the AI model behind ChatGPT also contradicts this requirement.

Nevertheless, companies must fulfill their information obligations under Articles 13 and 14 GDPR. According to Article 13 GDPR, they must inform the data subject in a comprehensible and easily accessible manner about the data processing and the functioning of the AI used. This also includes information on the scope of data processing, the legal basis, and the recipients.

Protection of data subjects' rights

Data subjects must be informed about their data protection rights. Even when using ChatGPT, companies must, according to the GDPR, fulfill the rights of data subjects and be able to delete processed data, e.g., upon request. However, this can be problematic if it is unclear how processing takes place and how personal data is stored. As a rule, at least prompts and outputs are stored for some time, so responsible companies must provide information about them and, if necessary, carry out deletion.

OpenAI's data protection role

Commercial ChatGPT users can integrate the chatbot into their internal processes. They are therefore responsible for data processing under data protection law. Since OpenAI has access to the company's data in this constellation, the Data Processing Agreement (DPA) offered by OpenAI should be concluded. Only in this way can a company ensure that data is used only on its behalf and not, for example, for training the AI model.

Data transfer to a third country

The chatbot is offered by the US company OpenAI. However, contracts between OpenAI and EU customers are often concluded with the Irish company, OpenAI Ireland Ltd., so no direct data transfer to the US company takes place. Indirectly, however, this company, along with other affiliated companies, is listed as a sub-processor of the Irish company. It must therefore still be checked whether OpenAI provides appropriate data protection guarantees and contracts to ensure an adequate level of protection for data transferred to OpenAI affiliated companies or other sub-processors.

How can companies use ChatGPT as securely as possible?

ChatGPT offers many advantages for users, but the chatbot can also cause harm. For example, criminals can use the tool to improve their phishing attacks. Furthermore, there is great uncertainty regarding ChatGPT's data protection. As long as the algorithm behind the chatbot is a black box, private users and companies should use the application with the utmost caution. Currently, there is no secure way to use ChatGPT and remain GDPR-compliant. However, AI users can take precautions to reduce the risk of a data breach as much as possible.

Stay up-to-date with newsletters and downloads

Companies considering using ChatGPT or already taking their first steps with the tool should actively inform themselves about further developments and official reviews. Our Proliance Newsletter.

Additionally, we offer you guides and checklists for background knowledge on Artificial Intelligence and the safest possible use of AI tools like ChatGPT.

Outlook for ChatGPT in Europe

Data privacy surrounding ChatGPT is still largely unclear. Nevertheless, schools, universities, and private individuals continue to use the chatbot diligently. This also applies to companies. With the European Union's AI Act, there are now at least binding guidelines for the use of ChatGPT in businesses.

The AI-Act harmonizes the regulations for AI systems in EU member states. The directive divides AI applications into four risk categories. The higher the risk level, the stricter the regulations.

Despite the AI Act, it remains the responsibility of companies to ensure data protection when using AI tools like ChatGPT. It is important to train employees on how to use Artificial Intelligence and to establish guidelines for AI usage. Furthermore, companies, together with their data protection officer, should regularly check whether data protection security measures are up-to-date and comply with current legal requirements.