Viewpoint: ChatGPT, Laden With Hacker-Inserted Malware, Is Writing Code for Companies

If yours is like many companies, hackers have infiltrated a tool your software development teams are using to write code. Not a comfortable place to be.

Developers have long used sites like stackoverflow.com as forums where they could get code examples and assistance. That community is rapidly being replaced by generative AI tools such as ChatGPT. Today, developers ask AI chatbots to help create sample code, translate from one programming language to another, and even write testcases. These chatbots have become full fledged members of your development teams. The productivity gains they offer are, quite simply, impressive.

Only one problem; how did your generative AI chatbot team-members learn to code? Invariably by reading billions of lines of open-source software, which is full of design errors, bugs, and hacker-inserted malware. Letting open-source train your AI tools is like letting a bank-robbing getaway driver teach high school driver’s ed. It has a built-in bias to teach something bad.

There are well over a billion open-source contributions annually to various repositories. Github alone had over 400 million in 2022. That’s a lot of opportunity to introduce bad code, and a huge “attack surface” to try to scan for issues. Once open-source has been used to train an AI model, the damage is done. Any code generated by the model will be influenced by what it learned.

Code written by your generative AI chatbot and used by your developers can and should be closely inspected. Unfortunately, the times your developers are most likely to ask a chatbot for help are when they lack sufficient expertise to write the code themselves. That means they also lack the expertise to understand if the code produced has an intentionally hidden backdoor or malware.

I asked LinkedIn how carefully people inspect the quality and security of the code produced by AI. A couple of thousand impressions later, the answers ranged from “very, very carefully”, to “this is why I don’t use generative AI to generate code”, “too early to use” and “[too much risk of] embedded malware and known design weakness”. But the fact remains that many companies ARE using generative AI to help code, and more are jumping on the bandwagon.

So what should companies do? First, they need to carefully inspect and scan code written by generative AI. The types of scans used matter; don’t assume that generative AI malware will match well-known malware signatures. Generated code changes each time it’s written. Instead, use “static” behavioral scans and Software Composition Analysis (SCA) to see if generated software has design flaws or will do malicious things. It also isn’t a good idea to let the same generative AI that produces high risk code write the testcases to see if the code is risky. That’s like asking a fox to check the henhouse for foxes.

While the risks of generating bad code are real, so are the benefits of coding with generative AI. If you are going to trust generated code, the old adage to “trust, but verify” applies.