Check Point has a very interesting article that they’ve posted to their website describing who cyber criminals are able to bypass restrictions placed on the ChatGPT AI to create “better” malware:
CPR researchers recently found an instance of cybercriminals using ChatGPT to “improve” the code of a basic Infostealer malware from 2019. Although the code is not complicated or difficult to create, ChatGPT improved the Infostealer’s code.
And:
However, CPR is reporting that cyber criminals are working their way around ChatGPT’s restrictions and there is an active chatter in the underground forums disclosing how to use OpenAI API to bypass ChatGPT´s barriers and limitations. This is done mostly by creating Telegram bots that use the API. These bots are advertised in hacking forums to increase their exposure.
To top it all off, there’s this:
In an underground forum, CPR found a cybercriminal advertising a newly created service: a Telegram bot using OpenAI API without any limitations and restrictions.
As part of its business model, cybercriminals can use ChatGPT for 20 free queries and then they are charged $5.50 for every 100 queries.
The net result is that this could literally be leveraged by anyone to create extremely dangerous malware. Which frankly is quite scary if you’re on the side of the fence where you have to defend against these attacks. But is this a bypass of ChatGPT’s restrictions? Craig Burland, CISO, Inversion6 doesn’t think so:
Describing this as a bypass is a bit of an exaggeration. ChatGPT doesn’t, at this point, impose controls on the API. Abuse of the API is prohibited by policy. Hackers haven’t cleverly bypassed security or exploited a deep-seated vulnerability. They’ve simply leveraged an incomplete feature. Given what OpenAI has accomplished, I expect this control is already on their roadmap and will be implemented shortly.
ChatGPT is in the midst of the hype cycle where every success or failure is shouted from the rooftops. ChatGPT is a complex and intriguing tool, but in the end, it is just a tool. We need to temper our human emotions, be patient as it matures, and figure out how best use it.
Hopefully this is on the roadmap to be addressed sooner rather than later. Otherwise I suspect that we’re all going to be in a lot of trouble.