Zscaler 블로그

Zscaler 블로그의 최신 업데이트를 이메일로 받아보세요

CXO Insights

AI software supply chain risks prompt new corporate diligence

image
>AI software supply chain risks prompt new corporate diligence

If your organization is using generative AI, you may want to widen your cybersecurity purview.

Like the risks of any cloud software supply chain, it is often hard to understand or predict downstream threats to cybersecurity that originate in the AI software supply chain. Many vulnerabilities can originate in your second or third-tier software supplier and go undetected until it is too late.

The mass enterprise adoption of AI/ML, LLMs, virtual agents, and chatbots over the last few years can reveal risk fault lines in shared code and infrastructure pushed through devops pipelines. Even with proper vendor risk management and due diligence, ‌your tech stack ‌could‌ be eventually compromised. 

CISOs are on the look out for novel attack vectors or future breaches of the upstream assets partners use to deliver their cloud-delivered AI services directly or to engineering teams for further tooling. These include code libraries, training data, and open source infrastructure as code (IaC) templates. Risks include manipulated or poisoned data sets and adversarial attacks.

Here’s an industry insider’s take on the scope of the challenges we face this year:

“By 2025, supply chain security will demand a whole new layer of vigilance, where even the datasets and AI models feeding into our applications are analysed for adversarial tampering. A secure supply chain won’t just be about code but curating safe and verifiable AI training sources,” Cache Merrill, founder of software development company Zibtek, recently told TechRepublic.

Even if your SaaS vendor mgmt and AI supply chain is about as airtight as it can be, security problems could arise from errors in the outputs of “trusted” GenAI tools, such as those used to generate code. 

Perils of AI-generated code

Using GenAI to code new software is not new and it’s turning developers into productivity superheroes. Savvy software development teams are augmenting their coding practices with AI, such as in quality assurance and testing. There’s an entire marketplace of AI coding solutions ready to take your business to new heights.

However, without proper testing, inspection, and overall adherence to an organization’s official software development life cycle (SDLC), AI-created code could introduce weaknesses and even cause outages. When bad code is made, put in a public repository, and others use it in their projects, the problems can quickly spread into the software supply chain. 

AI code generation models and the code they output can have exploitable vulnerabilities

image

 

The arrival of autonomous or agentic AI, which lets AI create its own code, gives AI a lot of power, but also makes the challenge even harder. AI agents can become multi-agent systems that are shared over the internet.

Zscaler Chief AI Officer, Clauadionor Coelho explains: “AI agents will cause a tremendous increase in the attack surface. On top of LLM-based attacks, we can add leaking of internal confidential data and model integrations and runtime data, and application code attacks, to name a few.”

All of this brings a new formidable risk to the AI ‌software supply chain, especially since they’re compounded by potential vulnerabilities in the training data and models used to make the code generators as well.

“By enabling tools to dynamically change or even enabling LLMs to generate new tools whenever an AI agent detects that it cannot perform a task with the current toolset, we open the door to completely new types of attacks that have never been considered,” Coelho warns. 

The stealth of data-poisoning attacks  

Another type of vulnerability that can lurk in the AI software supply chain is data poisoning. Here, an attacker contaminates or manipulates a model’s training data for an intended goal. Attacking AI this way involves no hacking in the traditional sense, making it hard to detect.

An attacker can manipulate an LLM’s training data to produce code that imports a malicious package for instance, or a library that can carry out phishing attacks or cause unexpected behaviors. Hugging Face, a community for AI developers, serves as a prime example of an attempted data poisoning attack when researchers identified a hundred malicious models on the platform. 

Lessons learned from cloud software supply chain risks

After software supply chain breaches at Target and Home Depot ten years ago, software supply chain risk made headlines again in 2020 with the SolarWinds data breach. The hack embedded malware code in the company’s network and monitoring platform, gaining backdoor access to all the systems connected to all the SolarWinds customer machines running the Orion platform. In the process, it exposed 30,000 public and private organizations–including public-sector agencies. Other supply chain attacks like MOVEit and Log4J have underscored the need for third party and software supply chain risk mitigation, and enterprise security teams have responded.

With AI at the center of IT innovation today, it will likely overtake low-code and no-code software development. The scale of code generated and shared openly this year and into the future will be staggering. 

Mitigating risks to the AI software supply chain

There are many things security and engineering leaders can do to help mitigate against the risks of the AI software supply chain.

Start with a cybersecurity program consisting of multi-factor authentication, endpoint detection and response, data encryption, data backups, and regular system updates. The next step is modern hygiene that includes a zero trust architecture and advanced capabilities like app-to-user segmentation, and honeypots and decoys.

To further reduce and mitigate the unique AI software supply chain risks covered in this article, consider how the various steps below can have a positive impact, including on your DevSecOps cycle:  

  • Customize your third-party risk assessments for AI - Adapt your protocols and provisions for GenAI and LLMs. Check for certifications like ISO/IEC 27001 and your vendor risk assessment reports are conducted by a reputable Governance, Risk, and Compliance organization. When building AI in-house, ensure the chosen platform(s) fits your data governance, intellectual property, and privacy policies. Extend your procurement due diligence to cover the AI software supply chain and inquire about validation and data cleansing mechanisms.
  • Consult guidance, such as secure software development practices and the NIST Cybersecurity Framework - Integrate AI software supply chain security measures into your CI/CD pipelines and expand secure-by-design principles to include code generation models and other AI systems.  Research and implement advanced defense strategies such as adversarial training and data input validation.  
  • Use continuous threat exposure management (CTEM) to scan for hidden vulnerabilities across your software supply chain - Define a scope that encompasses the software development process, including third-party APIs, microservices, code libraries, and AI vendors involved in creating software for your organization.

By making your entire ecosystem and supply chains that underpin GenAI, LLMs, etc., part of your overall threat universe, you can better safeguard your enterprise against emerging threats in the AI age.  

Resources:
CSET: Cybersecurity Risks of AI Generated Code 
NIST: Integrating Software Supply Chain Security in DevSecOps CI/CD Pipelines
Cloud Security Alliance (CSA): AI Controls Matrix (in peer review) 
 

form submtited
읽어주셔서 감사합니다

면책 조항: 이 블로그 게시물은 정보 제공만을 목적으로 Zscaler에서 작성하였으며, 정확성, 완전성 또는 신뢰성에 대한 어떠한 보증도 없이 "있는 그대로" 제공됩니다. Zscaler는 제공된 정보를 기반으로 수행된 조치나 오류 또는 누락에 대해 어떠한 책임도 지지 않습니다. 이 블로그 게시물에 링크된 제3자 웹사이트나 리소스는 편의를 위해서만 제공되며, Zscaler는 해당 웹사이트의 콘텐츠나 관행에 대해 책임을 지지 않습니다. 모든 콘텐츠는 예고 없이 변경될 수 있습니다. 이 블로그에 접속함으로써 귀하는 본 약관에 동의하며 귀하의 필요에 따라 정보를 확인하고 사용할 전적인 책임이 귀하에게 있음을 인정합니다.

Zscaler 블로그의 최신 업데이트를 이메일로 받아보세요

양식을 제출함으로써 귀하는 당사의 개인정보 보호정책에 동의하는 것입니다.