Gemini is Google’s newest family of Large Language Models (LLMs). The Gemini suite currently houses 3 different model sizes: Nano, Pro, and Ultra.
Although Gemini has been removed from service due to politically biased content, new findings from HiddenLayer – unrelated to that issue – analyze how an attacker can directly manipulate another users’ queries and output, which represents an entirely new threat. These vulnerabilities were disclosed to DeepMind per responsible disclosure practices.
While testing the 3 LLMs in the Google Gemini family of models, HiddenLayer found multiple prompt hacking vulnerabilities, including the ability to output misinformation about elections, multiple avenues that enabled system prompt leakage, and the ability to inject a model indirectly with a delayed payload via Google Drive. These vulnerabilities enable attackers to conduct activities that allow for misuse and manipulation. In new research released from HiddenLayer today, “New Google Gemini Content Manipulation Vulns Found – Attackers Can Gain Control of Users’ Queries and LLM Data Output – Enabling Profound Misuse,” HiddenLayer deep dives into these vulnerabilities, including a proof-of-concept of an Indirect Injection.
Who should be aware of the Google Gemini vulnerabilities:
- General Public: Misinformation generated by Gemini and other LLMs can be used to mislead people and governments.
- Developers using the Gemini API: System prompts can be leaked, revealing the inner workings of a program using the LLM and potentially enabling more targeted attacks.
- Users of Gemini Advanced: Indirect injections via the Google Workspace suite could potentially harm users. The attacks outlined in this research currently affect consumers using Gemini Advanced with the Google Workspace due to the risk of indirect injection, companies using the Gemini API due to data leakage attacks, allowing a user to access sensitive data/system prompts, and governments due to the risk of misinformation spreading about various geopolitical events.
Gemini Advanced currently has over 100M users, and so the ramifications of these vulnerabilities are widespread. With the accelerating adoption of LLM AI, companies must be aware of implementation risks and abuse methods that Gen AI and Large Language Models offer in order to strengthen their policies and defences.
Here is a link to the report :https://hiddenlayer.com/research/new-google-gemini-content-manipulation-vulns-found/
Related
This entry was posted on March 12, 2024 at 8:40 am and is filed under Commentary with tags HiddenLayer. You can follow any responses to this entry through the RSS 2.0 feed.
You can leave a response, or trackback from your own site.
Newly-Found Google Gemini Vulnerablities Give Attackers Control Over Users’ Queries & Content
Gemini is Google’s newest family of Large Language Models (LLMs). The Gemini suite currently houses 3 different model sizes: Nano, Pro, and Ultra.
Although Gemini has been removed from service due to politically biased content, new findings from HiddenLayer – unrelated to that issue – analyze how an attacker can directly manipulate another users’ queries and output, which represents an entirely new threat. These vulnerabilities were disclosed to DeepMind per responsible disclosure practices.
While testing the 3 LLMs in the Google Gemini family of models, HiddenLayer found multiple prompt hacking vulnerabilities, including the ability to output misinformation about elections, multiple avenues that enabled system prompt leakage, and the ability to inject a model indirectly with a delayed payload via Google Drive. These vulnerabilities enable attackers to conduct activities that allow for misuse and manipulation. In new research released from HiddenLayer today, “New Google Gemini Content Manipulation Vulns Found – Attackers Can Gain Control of Users’ Queries and LLM Data Output – Enabling Profound Misuse,” HiddenLayer deep dives into these vulnerabilities, including a proof-of-concept of an Indirect Injection.
Who should be aware of the Google Gemini vulnerabilities:
Gemini Advanced currently has over 100M users, and so the ramifications of these vulnerabilities are widespread. With the accelerating adoption of LLM AI, companies must be aware of implementation risks and abuse methods that Gen AI and Large Language Models offer in order to strengthen their policies and defences.
Here is a link to the report :https://hiddenlayer.com/research/new-google-gemini-content-manipulation-vulns-found/
Share this:
Like this:
Related
This entry was posted on March 12, 2024 at 8:40 am and is filed under Commentary with tags HiddenLayer. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.