Published: January 2025

This briefing note outlines some of the risks associated with generative AI tools that should be considered prior to and during their use.

1. Privacy and data protection

2. Hallucination

3. Inappropriate content and bias

4. Regulatory compliance​

Finally this note provides steps to consider to minimise risks and use generative AI responsibly.

Privacy and data protection​

Depending on individual T&Cs, generative AI tools such as ChatGPT may use user prompts to further train their models. This means that there is a risk that any information shared with them may be presented in a response to another person or organisation. While some tools allow the user to turn this feature off, the data entered may still be retained by the company owning the tool depending on the specific terms and conditions of use.

Hallucination​

Due to the way that generative AI works, it is incapable of critical thinking, understanding contexts and fact-checking its outputs like a human does. It simply generates what it deems to be the most statistically likely output based on the information contained within the prompts and its training data. This means these tools can sometimes convincingly present the user with responses that are nonsensical, inaccurate or misleading, which is referred to as hallucination.

Inappropriate content and bias

The outputs of a generative AI tool will reflect the data that it was trained on. Publicly available tools are often trained on vast amounts of data available from the internet, which means that responses may include content that is false, inaccurate, inappropriate or included without consent from its owner. Likewise, any biases that are present in the training dataset will also be reflected in the outputs. The information may also be out of date depending on when the model was trained and if the training data has been updated since then. Finally, generative AI models may be exposed to self-degradation as more AI-generated content is published online and subsequently used to further train the models. This cyclical process leads to reinforcement of existing model flaws and biases, progressively decreasing the quality of its outputs.

Regulatory compliance​

As with any other tool, generative AI use must be in line with existing laws and regulations around data protection, such as UK GDPR. In summer 2024, this included (but is not limited to) not sharing personal data of third parties in prompts for public tools without their consent, and not using generative AI to make significant decisions about individuals on a fully automated basis. Users should also be mindful of Intellectual Property (IP) rights such as copyright. Materials that are owned by third parties should not be included in prompts for generative AI tools without consent from the third party.

How to use generative AI responsibly?​

Steps can be taken to minimise the risks associated with the use of generative AI tools. These include:

• A human should review and sense-check all generative AI outputs to minimise risks related to bias, inappropriate and false content particularly in high-stakes scenarios or sensitive contexts.

• T&Cs of generative AI tools should always be reviewed prior to use to understand how user data will be handled and processed. This review should be repeated at a regular basis as it is common for these tools to have frequent T&Cs updates.

• Users should not share confidential or sensitive information with public generative AI tools that may use prompts to further train their models.

• Users should remain mindful of legislation – in summer 2024 this included UK GDPR and IP rights  –  when writing generative AI prompts and not include any personal data or materials owned by third parties unless specific consent has been given. If your organisation has a Data Protection Officer, it may be helpful to discuss generative AI tools with them prior to use.

• Generative AI tools and features may be added to existing applications and systems without users being made aware of this. As such, users should review any new functionalities in existing systems to check if they are powered by generative AI.

Further information and resources

• Generative AI: Opportunities for charities
• Generative AI: Ethics for charities
• Generative AI: Getting started
• Generative AI: Research application and assessment