Why AI Infrastructure Needs Attention
During ISO/IEC 27001 and SOC 2 scoping calls, a common question arises: "Do our AI tools need to be in scope?" The answer is often yes, but the real challenge is determining which controls apply.
Microsoft recently highlighted attacks on AI infrastructure components like LiteLLM (a gateway proxy), RAGFlow (a retrieval platform), and Kestra (a workflow orchestrator). Attackers targeted these systems to steal credentials and exploit compute resources. The issue wasn't model vulnerabilities but control-plane compromise.
Your team needs to recognize that AI infrastructure isn't just another SaaS integration. These components hold keys, database connections, and execution privileges. They're critical control points, and your security frameworks should treat them as such.
Should AI Gateways Be in Your SOC 2 System Description?
Yes, if they process customer data or control access to systems that do.
Gateways like LiteLLM route requests, manage keys, and log interactions, creating a trust boundary. Under SOC 2's Common Criteria, section CC6.1 requires identifying and managing logical access to systems. If a gateway can access customer data or issue credentials, it's in scope.
Consider this: if the gateway were compromised, could an attacker access customer data or impersonate your application? In the LiteLLM case, attackers accessed the gateway's PostgreSQL backend and extracted sensitive information. This is a CC6.1 and CC6.2 failure without proper access controls and monitoring.
Add the gateway to your system description. Document its functions, connections, and access controls. Your auditor will expect this.
What ISO/IEC 27001 Controls Apply to AI Orchestration Platforms?
Focus on Annex A 8.3 (Information Security in Supplier Relationships), 8.9 (Configuration Management), and 8.16 (Monitoring Activities).
Orchestration platforms like Kestra execute workflows that can access secrets and interact with data stores. Attackers have used these platforms to gain shell execution and deploy cryptocurrency miners. The workflow engine often becomes the entry point.
Annex A 8.3 applies if you're using a third-party service. Identify what information the supplier processes and their controls. If self-hosting, 8.9 requires secure configuration baselines, restricting workflows, and controlling access.
Annex A 8.16 requires monitoring. Log workflow execution, track secret access, and alert on unexpected activity. In the Kestra compromise, attackers moved from workflow execution to container-level access. Detection depends on visibility into these transitions.
How Do You Assess Risk for AI Components You Didn't Build?
Use your supplier risk assessment process, but expand threat scenarios.
ISO/IEC 27001 clause 6.1.2 requires information security risk assessment. For AI infrastructure, consider credential exposure, data leakage, compute abuse, and downstream access.
In the RAGFlow case, attackers intercepted credentials by modifying the application path. Your risk assessment should consider not just the component's intended function, but potential misuse by attackers.
Document your assessment in your Risk Treatment Plan. Identify the asset, threat, potential impact, and chosen treatment. If accepting residual risk, state it explicitly and set a review date.
Do You Need Separate Access Controls for AI Infrastructure?
Yes, controls should match the privilege level.
SOC 2 CC6.2 requires restricting logical access to authorized users. For AI gateways and orchestration platforms, this group should be small. These components often have broad access, holding keys for multiple providers and connecting to databases.
In the LiteLLM compromise, attackers harvested credentials from the gateway's environment. The gateway had access to everything needed to route requests, which the attacker exploited.
Apply Annex A 8.2 (Privileged Access Rights): restrict administrative access, require multi-factor authentication, log all actions, and review access quarterly. Treat administrative access to gateways like domain controller access.
What Evidence Do Auditors Expect for AI Component Security?
Provide configuration baselines, access logs, change records, and monitoring evidence.
For SOC 2, auditors will test CC6.6 (logical access removal) and CC7.2 (system monitoring). Provide access review records, change logs, and monitoring evidence showing detection of anomalous activity.
For ISO/IEC 27001, expect requests tied to Annex A 8.9 and 8.16. Show configuration management records and monitoring logs. Demonstrate understanding of normal behavior for these components to differentiate from checkbox compliance.
Should You Map AI Infrastructure to Your ISMS Asset Inventory?
Absolutely, with specific details about each asset.
ISO/IEC 27001 clause 8.1 requires operational planning and control of information security processes. Your asset inventory should list each AI component, its function, data processed, credentials held, and system connections.
For a gateway: "LiteLLM proxy, routes application requests to OpenAI and Anthropic, holds provider API keys and database connection string, connects to PostgreSQL backend, processes all user prompts." For an orchestration platform: "Kestra workflow engine, executes data-processing jobs, has Docker access, retrieves secrets from Vault, connects to S3 and Snowflake."
This detail makes your risk assessment meaningful and control selection defensible. It also streamlines scoping conversations with auditors.
Next Steps
If you're adding AI infrastructure to your compliance scope, start with your existing control framework. ISO/IEC 27001 Annex A and SOC 2 Common Criteria cover access control, configuration management, monitoring, and supplier management. Apply these controls to new asset types.
Review ISO/IEC 27003 for ISMS implementation guidance and ISO/IEC 27004 for monitoring approaches. For SOC 2, revisit the Trust Services Criteria focusing on CC6, CC7, and CC8.
Your technical security controls should reflect the trust these components hold. Microsoft observed attackers moving quickly from initial access to credential theft and persistence. Your detection and response capabilities need to match that speed.
AI infrastructure is infrastructure. Treat it accordingly.
