Rezilion, an automated software supply chain security platform, today announced a new report, “Expl[AI]ning the Risk: Exploring the Large Language Models (LLM) Open-Source Security Landscape,” finding that the world’s most-popular generative artificial intelligence (AI) projects present a high security risk to organizations.
Generative AI has surged in popularity, empowering us to create, interact with, and consume content like never before. With the remarkable advancements in LLMs, such as GPT (Generative Pre-Trained Transformers), machines now possess the ability to generate human-like text, images, and even code. The number of open-source projects that integrate these technologies is now growing exponentially. By way of example, since OpenAI debuted ChatGPT seven months ago, there are now more than 30,000 open-source projects on GitHub using the GPT-3.5 family of LLMs.
Despite the booming demand for these technologies, GPT and LLM projects present various security risks to the organizations that are using them, including trust boundary risks, data management risks, inherent model risks, and general security concerns.
Rezilion’s research team investigated the security posture of the 50 most popular generative AI projects on GitHub. The research utilizes the Open Source Security Foundation (OSSF) Scorecard to objectively evaluate the LLM open-source ecosystem and highlight the lack of maturity, gaps in basic security best practices, and potential security risks in many LLM-based projects.
The key findings highlight concerns, revealing very new and popular projects with low scores:
- Extremely popular, with an average of 15,909 stars
- Extremely immature, with an average age of 3.77months
- Very poor security posture with an average score of 4.60 out of 10 is low by any standard. For example, the most popular GPT-based project on GitHub, Auto-GPT, has over 138,000 stars, is less than three months old, and has a Scorecard score of 3.7.
The following best practices and guidance is recommended for the secure deployment and operation of generative AI systems: educate teams on the risks associated with adopting any new technologies; evaluate and monitor security risks related to LLMs and open-source ecosystems; implement robust security practices, conduct thorough risk assessments, and foster a culture of security awareness.
An alarming amount of time is dedicated to security – especially when it comes to software. Rezilion’s automated software supply chain security platform helps customers to manage their software vulnerabilities efficiently and effectively. Maintaining a detailed and current database on the latest software vulnerabilities and the strategies to mitigate them remains paramount to customers’ success in navigating this complex security landscape. Rezilion provides its users with the same OpenSSF scorecard insights as part of the product offering for customers to make more informed decisions regarding adopting and managing any open-source project.
I also got some commentary Yotam Perkal, Director of Vulnerability Research at Rezilion who authored this report.
What was the most concerning finding from the survey and why?
The most concerning finding from the survey is the inadequate maturity and security posture of the open-source ecosystem surrounding LLMs. As these systems gain popularity and adoption, it is inevitable that they will become attractive targets for attackers, leading to the emergence of significant vulnerabilities. This finding raises concerns about the overall security of LLMs and highlights the need for improved security standards and practices in their development and maintenance.
What should organizations know about LLM risk before integrating Gen AI tools?
Organizations should be aware that integrating Generative AI tools, including LLMs, comes with both unique challenges and general security concerns. They need to address the specific risks associated with LLMs, such as data privacy, protection against attacks on the models, and securing the infrastructure involved in their deployment. Additionally, organizations must consider broader security implications and ensure that industry security standards are followed to promote ethical and responsible use of generative AI technology.
How can they prepare for this risk and who is responsible for this?
Organizations can prepare for LLM risks by adopting a secure-by-design approach when developing Generative AI-based systems. They should leverage existing frameworks like the Secure AI Framework (SAIF), NeMo Guardrails, or MITRE ATLAS™ to incorporate security measures into their AI systems. It is also imperative to monitor and log LLM interactions and regularly audit and review the LLM’s responses to detect potential security and privacy issues and update and fine-tune the LLM accordingly. Responsibility for preparing and mitigating LLM risks lies with both the organizations integrating the technology and the developers involved in building and maintaining these systems.
What are some other risks GPT and LLMs can pose to organizations?
The risks associated GPT and LLMs can pose are varied and can affect all aspects of the CIA triad (Confidentiality, Integrity and Availability). These risks can lead to bypass of access controls, unauthorized access to resources, system vulnerabilities, ethical concerns, potential compromise of sensitive information or intellectual property and more.
How will this risk through LLM to organizations evolve in the next 12-18 months?
Over the next 12-18 months, the risk through LLMs to organizations is expected to evolve as the popularity and adoption of these systems continue to grow. Without significant improvements in the security standards and practices surrounding LLMs, the likelihood of targeted attacks and the discovery of vulnerabilities in these systems will increase. Organizations must stay vigilant and prioritize security measures to mitigate evolving risks and ensure the responsible and secure use of LLM technology.
To download the full report, please visit: https://info.rezilion.com/explaining-the-risk-exploring-the-large-language-models-open-source-security-landscape
The Petro Canada Cyberattack Could Cost The Company In Multiple Ways… And Who’s Behind This Cyberattack?
Posted in Commentary with tags Petro Canada on June 28, 2023 by itnerdWe are now in day six of the Petro Canada/Suncor cyberattack. The app is still down, and stations are still only accepting cash. While the company admits that there has been attack, few details beyond what I have had outlined are available. Though there are rumours that it is worse than what we know. Whatever is going on, it’s going to cost them a lot in multiple ways:
Another question that has surfaced in recent days is who is behind this and what is their motivation. To give you some views on this question, I sought the commentary of a variety of experts:
Mike Hamilton, Former CISO of the City of Seattle and former Vice-Chair of the DHS State, Local, Tribal, and Territorial Government Coordinating Council (SLTTGCC) and CISO of Critical Insight
This is not the first time a Canadian energy sector company has been recently compromised. A pipeline company was compromised earlier this year, and a recent intelligence report stated that Russian actors are actively seeking to disrupt Canada’s energy infrastructure. Rather than being victims of opportunity, these events seem to be strategic acts of nation-state actors and not cyber criminals looking for a score.
About a year ago Canada announced that it would boost oil and gas production to assist the European Union cut its use of Russian energy. Notably, and in at least one of the incidents, actors were able to manipulate the operational technologies (OT) and did so. This suggests that the tools and tactics being used were more sophisticated, making these events significantly different than the ransomware attack against the IT (not OT) network of Colonial Pipeline.
According to the intelligence report, these events will likely continue for the duration of the war in Ukraine and are intended to produce psychological impacts in the population and yes, the United States is also a target for this activity. Whereas the actual destruction or permanent disruption of this infrastructure would constitute an act of war, temporary disruptions to energy generation and transmission are likely to proliferate and insofar as possible create the perception that it’s a criminal act. (Note that disrupting distribution is the domain of domestic nutjobs.)
Ron Brash, VP of Research and critical infrastructure software security firm, aDolus Technology
In the respect of comparing Suncor and Colonial, they are not the same and are not really in the same business. Had Suncor been Enbridge, this would have been a vastly different story, but based on POS outages, rewards programs and corporate AD/credentials – it appears again to be more of a Honda-like event and some operations affected. Given the size and nature of Suncor/Petroncan, it’s more likely that cardlocks, volume tracking and maybe metering/pos on pumps were affected. Some warehouse activities such as product management and shipping may have been stalled or degraded, but like downstream retail – they can often be run with a clipboard, measuring stick, calculator and an alternative form of payment (all except cardlock of course).
This could be a focused event because of Canada and other allies’ stance on the war in Ukraine, but evidence points to more of an inconvenience vs being a major incident or an organization such as a significant pipeline. It may have been entirely opportunistic, and dressed up under a guise. However, more accurate details are needed before a true impact assessment can be surmised.
Ron Fabela, field CTO at cybersecurity firm XONA Systems
It’s incredibly difficult to tie any intention to the Suncor cyber event with geopolitical actions or threats. This very loose hypothesis comes from reports of “A pro-Russia hacktivist group claims to have breached the network of a Canadian gas pipeline company in February and caused damage that resulted in loss of profits, according to a document found among a tranche of US classified intelligence assessments leaked online recently.” (source Kim Zetter https://zetter.substack.com/p/leaked-pentagon-document-claims-russian). Note that at the time, this breach and impact was communicated in the past tense, meaning an event that already occurred earlier this year.Even so, impacts reported by Petro-Canada (and parent Suncor Energy) indicate a potential standard ransomware attack against point of sale systems and supporting backend systems (Nothing close to the reported “[…]show their access to the Canadian facility and indicating that they had the ability to increase valve pressure, disable alarms, and initiate an emergency shutdown of the facility” (source again from Kim Zetter).)At this time, there’s no indication that this event is having Colonial Pipeline-like impacts on Canadian infrastructure or customer confidence. However any cyber event, whether a direct APT attack or opportunistic ransomware, that affects critical infrastructure operations should be taken seriously and as a recipe for what’s ahead. The prevalence of ransomware targeting remote access services like with Colonial Pipeline or exposed vulnerable technologies such as MoveIT is only going to continue to have secondary impact on the safe and reliable operations of critical systems. Regardless of geopolitical intent this continues to be a concern for not just the US, but critical infrastructure organizations around the world. My advice: create plans around incident response, implement technologies that support visibility and zero trust architectures, take those first concrete steps into preventing future attacks instead of waiting for them to strike close to home.
For the sake of Petro Canada and Suncor, I hope that they’re making every effort to address this because the longer it goes on, the more likely that it won’t end well for them. On top of that, I hope that there’s a focused effort to find who did this and bring them to justice.
1 Comment »