CircleCI, a company that develops testing and deployment tools for software engineers, has shared details about how hackers broke into its systems last month and compromised customer data. CircleCI chief technology officer Rob Zuber said hackers gained access to its networks after infecting an employee’s laptop with malware. And here’s what happened next:
On December 29, 2022, we were alerted to suspicious GitHub OAuth activity by one of our customers. This notification kicked off a deeper review by CircleCI’s security team with GitHub.
On December 30, 2022, we learned that this customer’s GitHub OAuth token had been compromised by an unauthorized third party. Although that customer was able to quickly resolve the issue, out of an abundance of caution, on December 31, 2022, we proactively initiated the process of rotating all GitHub OAuth tokens on behalf of our customers. Despite working with GitHub to increase API rate limits, the rotation process took time. While it was not clear at this point whether other customers were impacted, we continued to expand the scope of our analysis.
By January 4, 2023, our internal investigation had determined the scope of the intrusion by the unauthorized third party and the entry path of the attack. To date, we have learned that an unauthorized third party leveraged malware deployed to a CircleCI engineer’s laptop in order to steal a valid, 2FA-backed SSO session. This machine was compromised on December 16, 2022. The malware was not detected by our antivirus software. Our investigation indicates that the malware was able to execute session cookie theft, enabling them to impersonate the targeted employee in a remote location and then escalate access to a subset of our production systems.
Because the targeted employee had privileges to generate production access tokens as part of the employee’s regular duties, the unauthorized third party was able to access and exfiltrate data from a subset of databases and stores, including customer environment variables, tokens, and keys. We have reason to believe that the unauthorized third party engaged in reconnaissance activity on December 19, 2022. On December 22, 2022, exfiltration occurred, and that is our last record of unauthorized activity in our production systems. Though all the data exfiltrated was encrypted at rest, the third party extracted encryption keys from a running process, enabling them to potentially access the encrypted data.
Clearly the threat actors knew who to target to get what they wanted. That’s scary. The company has put out a security alert that has been consistently updated since this incident happened. I’d spend some time reading this if you are using CircleCI products. An example of this is that Datadog’s RPM GPG signing keys and its passphrases were exposed during this breach. Anyone who uses their products, and any vendor who uses those products are potentially at risk.
Kevin Bocek, VP of Security Strategy and Threat Intelligence at Venafi had this to say:
“Another day, another software supply chain attack. It’s clear that this type of threat isn’t going away. Targeting a developer tool and delivery platform, like CircleCI, was clearly intended to fly under the radar and slip into other development environments. In this case, they were able to gain access to Datadog’s environment meaning that its RPM GPG signing machine identities were exposed. Fortunately, Datadog has responded quickly to rotate the impacted identities and it doesn’t appear that they’ve been abused. But if an attacker had seized this opportunity, then it would have given them a very powerful weapon – potentially allowing them to spread across Datadog’s customer networks by enabling them to sign and send malware while appearing completely trusted. This could have had serious repercussions.
“This incident demonstrates the growing risk of attacks targeted at developers, machine identities and modern development pipelines. When combined with the speed of modern development, widespread use of automation and use of the cloud, an attacker with access to powerful machine identities can create ripples fast which are extremely hard to protect against or remediate. In a machine-driven world, having a control plane to manage the lifecycle of your machine identities is essential. As this incident shows, you can be doing all the right things and still find yourself exposed. All businesses – whether they be a software publisher, or a consumer of software – need to be able to automate controls that say who and what can and can’t be trusted, and to have the agility to respond to change.”
This isn’t a trivial hack and should not be treated as such. If you’re reliant on CircleCI products, you should be ensuring that you are not exposed. And you should double check with your vendors that they have done their due diligence as well.
Lumenova AI Empowers Companies to Make Responsible AI a Part of Their DNA
Posted in Commentary with tags Lumenova AI on January 18, 2023 by itnerdLumenova AI, the platform that automates the Responsible AI lifecycle, announced today that it has officially launched to enable organizations to understand deeply complex models and gain insights into what’s driving the AI decision-making process. With the company’s leading-edge technology, its clients can detect deep-rooted biases, optimize their models’ reliability and robustness, and build trust and transparency into their AI.
While most business leaders believe AI is their key to competitive advantage, many report it is still not used as widely as possible in 2023 — a decade into the AI revolution. Amongst the barriers, enterprises need help understanding their own deeply complex models, scaling their systems and reaching deployment. Responsible AI, a framework that helps organizations address ethical and legal challenges, explain how their AI works, and improve data integrity, security and resilience, as well as accuracy and validity around Artificial Intelligence, is part of the solution.
The company offers its customers the most complete Responsible AI solution. It provides for policy development to help prepare for regulations and internal requirements with its custom frameworks. It offers technical evaluation by opening the algorithmic “black box” to evaluate risks and quickly detect performance opportunities. It also provides monitoring and remediation to help assess and manage potential issues proactively.
This is not a one-time endeavor. Achieving a successful Responsible AI program is a journey and a complex process involving multiple steps and stakeholders. Lumenova AI’s platform, extended team of business consultants and ML engineers help customers navigate the ever-changing regulatory landscape through end-to-end guidance and practical support.
Driving the new generation of Responsible AI
Lumenova AI is committed to delivering value through its state-of-the-art AI Trust Platform that enables businesses to make AI ethical, fair, transparent and accurate. With over 15 years of experience working with related technologies, such as data, machine learning and the cloud, and over three years of experience in Explainable AI, we pledge to be pioneers of change and help organizations make Responsible AI a key pillar of their ML strategy.
About Lumenova AI
Headquartered in Los Angeles, Lumenova AI empowers organizations to make AI ethical, transparent and compliant with new and emerging regulations and internal policies. As an end-to-end solution, Lumenova AI streamlines and automates the entire Responsible AI lifecycle, so enterprises can efficiently map, manage and mitigate AI risk, comply with regulations and maximize the inherent value.
Lumenova caters to a diverse group of stakeholders, including business leaders, IT leaders, and the risk and compliance community. It allows them to analyze and optimize model performance, increase robustness and promote predictive fairness across all dimensions of trust. Their extended team of technical experts and business consultants can also provide strategy and execution consulting for enterprises to design and deploy Responsible AI at scale.
For more information, please visit www.lumenova.ai.
Leave a comment »