The IT Nerd

California recently adopted an AI Resolution that’s in alignment with the Biden Administration’s guidelines for responsible AI. Spearheaded by Sen. Dodd, this resolution reinforces California’s influential role in shaping regulatory frameworks:

Senate Concurrent Resolution 17 highlights the significant challenges posed by the use of technology, data, and automated systems, including incidents of unsafe, ineffective, or biased systems and unchecked data collection that threatens privacy and opportunities. At the same time, the resolution recognizes the potential benefits of AI, including increased efficiency in agriculture and data analysis that could revolutionize industries.

The resolution affirms the state’s commitment to President Biden’s vision for safe AI and the principles outlined in the White House Office of Science and Technology Policy’s “Blueprint for an AI Bill of Rights.” The five principles — Safe and Effective Systems; Algorithmic Discrimination Protections; Data Privacy; Notice and Explanation; and Human Alternatives, Consideration and Fallback — will guide the design, use, and deployment of automated systems in California.

SCR 17 was approved Monday in the Assembly with a unanimous voice vote after being previously approved by the full Senate. It does not require the governor’s signature.

Ani Chaudhuri, CEO, Dasera had this comment:

Today, with the California Legislature adopting the nation’s first AI-drafted resolution, we’re witnessing a pivotal moment in the intersection of technology, governance, and society. As someone deeply entrenched in data security and governance, this resolution isn’t just a piece of legislative text; it’s a testament to how our society is evolving and the responsibilities we must shoulder as we traverse this path.

1. Safe and Effective Systems: AI’s promise lies in its ability to improve our world, but this can only be realized if the systems themselves are safe and effective. Any AI system must be meticulously tested in controlled and real-world scenarios. But it’s more than just about ensuring systems don’t malfunction—it’s about ensuring they function in a way that aligns with our societal values and norms.
2. Algorithmic Discrimination Protections: Biases in AI systems have made headlines repeatedly, tarnishing this transformative tech’s image. Eliminating biases isn’t a ‘nice-to-have’—it’s a fundamental necessity. Every stage of AI development, from data collection to model training, should be scrutinized to ensure no group is unduly disadvantaged.
3. Data Privacy: In an era where personal data is often compared to oil in its value, safeguarding this data is paramount. While AI systems thrive on data, we must implement stringent measures to ensure data privacy isn’t compromised. From where data is stored to how it’s accessed to who has rights to it—every aspect needs to be governed with the utmost responsibility.
4. Notice and Explanation: The days of black-box algorithms must end. Stakeholders, from the public to policymakers, should clearly understand how AI decisions are made. It’s not about revealing trade secrets but ensuring transparency so these systems can be trusted.
5. Human Alternatives, Consideration, and Fallback: As magnificent as AI is, it isn’t infallible. There should always be a human touchpoint—a fallback mechanism—that can intervene when things go awry. Automated systems should be designed with the understanding that humans are the ultimate safeguard.

Sen. Dodd’s resolution serves as a blueprint for California, the entire nation, and potentially the world. The principles highlighted are about safe AI deployment and ensuring AI uplifts society without trampling on individual rights.

To my colleagues in the tech industry: let’s take this as a call to action. We have the responsibility not only to innovate but to ensure that our innovations are imbued with integrity, respect, and a profound sense of duty to the betterment of society.

AI has the potential to transform society. But it needs guardrails around it. Otherwise the potential exists for it to run amok and harm society instead of help it. Which is why I feel that this l feel that this resolution is a great move.

There’s a new AI FraudGPT tool discussed in this Netenrich report called “FraudGPT: The Villain Avatar of ChatGPT,” and the recent appearance of the WormGPT, used to launch BEC attacks as discussed in this SlashNext report. Both reports are very much worth reading as AI is clearly being used for evil.

I did a Q&A on this with David Mitchell, Chief Technical Officer, HYAS and got this commentary: 

• Any differences & similarities of these tools/offerings?

“The only difference will be the goal of the particular groups using these platforms — some will use for phishing/financial fraud and others will use to attempt to gain access to networks via other means. “     

• Are these just riding on the ChatGPT brand, or are they new AI iterations?  

“GPT stands for “Generative Pre-trained Transformer”, which is a specific model of AI use case, not a brand per-se. The dark versions being sold may have different sets of training and data sizes, but the overarching point is that they have no guardrails or ethics ingrained. “     

• Why now and will we see more of this attack vector? 

“As with any new technology, soon after it is released, nefarious actors begin adopting it in order to learn its weaknesses to exploit. In the case of GPT, nefarious actors are adopting the technology and enhancing it for their needs. “    

• Can these AI assisted attacks be detected by currently installed defenses? 

“Historically, these attacks could often be detected via security solutions like anti-phishing and protective DNS platforms. With the evolution happening within these dark GPTs, organizations will need to be extra vigilant with their email & SMS messages because they provide an ability for a non-native English speaker to generate well-formed text as a lure.”

These new AI based attack tools are going to make life miserable for defenders. Thus hopefully defences can be made to make AI based attack tools less dangerous.

Members of the G7 Group of nations are together today to discuss AI regulation:

G7 government officials will hold the first working-level AI meeting on May 30 and consider issues such as intellectual property protection, disinformation and how the technology should be governed, Japan’s communications minister, Takeaki Matsumoto, said.

The meeting comes as tech regulators worldwide gauge the impact of popular AI services like ChatGPT by Microsoft-backed OpenAI.

The EU is coming closer to enact the world’s first major legislation on AI, inspiring other governments to consider what rules should be applied to AI tools.

Japan, as this year’s chair of G7, “will lead the G7 discussion on responsive use of the generative AI technology”, Matsumoto said, adding the forum hoped to come up with suggestions for heads of state by year-end.

Kevin Bocek, VP Ecosystem and Community at Venafi starts out with this comment:
 
“We are still in the early stages of understanding the impact of AI on both businesses and the public, and it’s a constantly moving target, with new use cases and products being announced on a daily basis. So, it is very encouraging to see world leaders putting AI at the heart of discussions and starting to think about the best way to move forwards. As part of this process, it is vital that they recognize that smart organizations will not slow down the innovation that we’re seeing with Generative AI, and that the results will be overwhelmingly positive. However, there are known and unknown risks that need to be skillfully mitigate. 

As such, the priority for regulations must be to contain risks while encouraging exploration, curiosity and trial and error. But any steps to achieve this can’t be approached with a “set and forget” mentality. Regulators need to establish policies and guidelines that are reviewed and refreshed frequently as we explore the power of AI in more depth. This means the governments will need to constantly collaborate and communicate with experts in the field to avoid neglect and exploitation.”

Ani Chaudhuri, CEO, Dasera follows up with this:

“The forthcoming G7 meeting on AI regulation highlights a critical juncture in our technological evolution. It’s encouraging to see top-level discussions taking place around intellectual property protection, disinformation, and governance in AI – topics that are integral to the development and responsible use of AI tools.

The creation of the “Hiroshima AI process” demonstrates a welcome commitment from global leaders to address the challenges of AI technology. It is a positive step towards fostering a future where AI aligns with our shared democratic values and upholds a high standard of trustworthiness.

However, while discussions on international standards are crucial, equally important is the ability to adapt these standards as the AI landscape continues to evolve rapidly. For AI to be truly beneficial, we must focus not only on legislation but also on transparency, user control, and education about these technologies.

Moreover, AI ethics should not be an afterthought. Building ethical considerations into AI systems from the outset is vital to ensure the technology respects privacy, maintains security, and protects human rights. This, in my opinion, should be at the forefront of G7 discussions. I look forward to the outcomes of these important conversations and the future of AI regulation.”

I will be interested to see what comes out of these meetings and if companies in the AI space abide by any regulation that appears. That’s the key as rules are meaningless if they are not adhered to.

There’s an open letter signed by over 1200 people who are asking for an immediate six-month halt on AI technology more powerful than ChatGPT-4. The open letter was created by an organization called the Future of Life Institute. The aim of this organization is to “steer transformative technology towards benefitting life and away from extreme large-scale risks.” Among those who signed are Steve Wozniak who co-founded Apple, Elon Musk the clown prince of tech and the guy who runs Twitter, SpaceX, and Tesla among other companies. This does bring up all sorts of questions about AI and how it should be used.

I have a number of comments on AI in general and specifically this open letter. The first is from Baber Amin, COO, Veridium:

Thoughts on AI development and application:

“For great leaps in technology, we often need to establish safety measures and regulations – for example, when we split the atom to harness nuclear power. While nuclear energy has provided many advantages in fields like medicine and energy, it has also given rise to the terrible threat of nuclear weapons. However, the difficulty of accessing and managing nuclear materials has provided a natural form of protection.

“AI model development and training, on the other hand, lack these same natural barriers, making it easier to develop without appropriate safety measures in place. That’s why it’s important to take a step back and create responsible systems that are accurate, transparent, trustworthy, and potentially even capable of self-regulation.

Risks for companies using the OpenAI API.

      “As organizations turn to OpenAI’s API for their artificial intelligence needs, it’s important to keep in mind the following considerations:

1. Data Privacy: OpenAI’s models are trained on large amounts of data, which until recently could have included sensitive information from organizations. Starting March 1, OpenAI will no longer use customer data submitted via API to train their models without explicit consent. However, the data will still be kept for 30 days for monitoring purposes.
2. Bias: OpenAI’s training data comes from the real world, which means it may contain biases that are reflected in their models. Organizations using OpenAI should be aware of this possibility and take corrective measures.
3. Misinformation and Fake Data: OpenAI’s generative models can create text that is indistinguishable from real data, which could be used to generate fake news or blog posts. Organizations need to be cautious of inadvertently spreading misinformation.
4. Phishing Attacks: OpenAI’s generative models can also be used to create sophisticated phishing attacks or deepfakes, which could lead to propaganda and possible slander.
5. Spam: Lastly, OpenAI’s generative AI can be used to generate spam, resulting in unsolicited emails or social media posts, causing reputational damage to an organization

     “By keeping these considerations in mind, organizations can use OpenAI’s API effectively and responsibly.

      “For security protections, looking at OpenAI, they do have the following security controls in place, which all seem very reasonable.  

• Data encryption at rest and in transit.
• Access control around data and models.
• Monitoring for suspicious activity.
• Patching for latest security patches.
• Auditing of access to data and models.

Matt Mullins, Senior Security Researcher, Cybrary is next:

   “There are a number of benefits to AI and its applications that are being explored. While there are a great deal of efficiencies created there, other non-beneficial aspects arise. The disruption of a number of industries being the most profound, in ways that were not easily predictable. Things associated (typically) with “human-ness” are being found to be more vulnerable than other aspects.

   “For example… art, music, essays, and other things that were an established trope of human creativity as normality are significantly being destabilized as AIs are able to quickly ingest, seed, and innovate in ways that were not previously predicted.

   “Aside from these disruptions, the potential for attacks on baseline ‘truth’ have been established as well. Consider the modification of voice, visual imagery, and video which can all be done so effectively that a zoom call could potentially be spoofed. The ramifications of such realistic mimicry have direct threats to establishments of truth and sub sequentially democratic process itself.

Overall, AI is presenting a removal of entry level aspects to IT and security. Beyond this entry level the veil seems to be easy to pierce with a critical eye for understanding code. The bigger issues presented are the capabilities that AI presents to disrupt how we see the world.”

David Maynor, Senior Director of Threat Intelligence, Cybrary has this to add:

Addressing major tech calling for a 6 mo. AI moratorium:

   “It is funny that technologist that have been disruptive to industries and use mantras like “fail fast” are aligning against AI research. While conspiracy theories point to worrying about a Skynet like AI turning on humans I personally feel that AI availability will disrupt the disruptors and make their fiefdoms ripe for replacement.”

It will be interesting to see how this play out. I for one do not see the AI arms race as I call it stopping anytime soon unless governments get interested in terms of slowing down AI development.

UPDATE: Dr. Chenxi Wang (she/her), Founder and General Partner, Rain Capital added this comment:

A pause in the AI fever is needed, not just from the business standpoint, but also from the point of view of security and privacy. Until we understand how to assess data privacy, model integrity, and the impact of adversarial data, continued development of AI may lead to unintended social, technical, and cyber consequences.