What Changed
Unit 42 from Palo Alto Networks ran AI models against multiple codebases as part of early access programs with Anthropic and OpenAI. The results: AI-augmented teams completed what would normally take two years of penetration testing in just three weeks, discovering nearly 100 vulnerabilities. More than 40% of these vulnerabilities were rated high or critical, with most exploits involving leaked credentials and broken access controls.
This isn't a vendor demo or a research paper. This is production security work happening at enterprise speed, and it's forcing a fundamental question: if AI can do in weeks what your team does in years, what's your job now?
Key Findings
Finding 1: Speed creates new accountability gaps
When vulnerability discovery accelerates by a factor of 30, your remediation process becomes the bottleneck. You'll find more critical issues faster than your development teams can patch them, faster than your risk committee can prioritize them, and faster than your documentation can keep up. The constraint isn't discovery anymore; it's decision-making and execution.
Action: Map your current vulnerability-to-patch timeline. Identify where human approval gates slow remediation. For issues AI flags as critical, define pre-approved response paths that don't require three meetings.
Finding 2: The orchestrator role is real, not metaphorical
Unit 42's Sam Rubin stated: "The job of the security person will be an orchestrator security engineer." This isn't about managing people; it's about managing AI agents that run security tasks. You're designing the workflow, validating the output, and owning the outcome when an AI agent misses something or flags a false positive.
Action: Document what your AI tools actually do versus what you think they do. For each security task you've automated or augmented, write down: what inputs the AI needs, what it produces, what validation steps you run, and who's accountable if it fails.
Finding 3: Expertise still separates signal from noise
Unit 42's Shay Nahari compared AI-augmented security to vibe coding: someone without a coding background can produce code with AI, but it won't be production-ready or as robust as code from a seasoned developer using AI. The same applies to security work. An AI agent can flag 100 vulnerabilities, but an experienced security engineer knows which 10 actually matter in your architecture.
Action: Audit your team's skill distribution. If you're hiring for "AI-augmented security roles," define what baseline expertise you need before the AI amplification matters. You can't orchestrate what you don't understand.
Finding 4: Control frameworks haven't caught up
ISO/IEC 27001:2022 Annex A doesn't have a control for "ensure AI agents don't leak credentials during automated pentesting." SOC 2 CC6.1 (logical and physical access controls) doesn't specify how you validate that an AI model isn't introducing new access control vulnerabilities while testing your existing ones. Your audit evidence requirements were designed for human-executed controls.
Action: Review your ISMS procedures for vulnerability assessment (ISO/IEC 27001 Clause 8.8) and penetration testing (often tied to Annex A Control 5.7). Add a section that defines how you validate AI-generated findings, who reviews them, and what evidence you retain. Your auditor will ask.
Finding 5: Threat actors have the same tools
OpenAI and Anthropic gave Unit 42 early access to advanced models, but those models will eventually be public or accessible through other channels. If defenders can compress two years of pentesting into three weeks, so can attackers. The asymmetry isn't in the tools anymore; it's in how fast you can respond to what the tools find.
Action: Update your incident response plan (ISO/IEC 27001 Annex A Control 5.24) to account for AI-accelerated attacks. Define what "fast response" means when an attacker can enumerate your entire attack surface in hours, not weeks.
What This Means for Your Team
You're not getting replaced; you're getting reassigned. The work shifts from execution to oversight, from finding vulnerabilities to deciding what to do about them, from writing detection rules to validating that AI-written rules don't introduce blind spots.
This creates three immediate problems:
Problem 1: Your evidence collection breaks. SOC 2 Type II audits expect you to demonstrate that controls operated effectively throughout the audit period. If an AI agent ran your vulnerability scans, your auditor will ask: how do you know the agent didn't miss something? What's your validation process? Where's the human review?
Problem 2: Your training requirements expand. ISO/IEC 27001 Clause 7.2 requires competence for roles affecting ISMS performance. If your security engineers are now orchestrating AI agents, what competence do they need? How do you demonstrate it? Your certification body will expect documented training on AI tool limitations, not just feature lists.
Problem 3: Your risk register needs new entries. Relying on AI for security work introduces new risks: model drift, adversarial inputs, hallucinated findings, over-reliance on automated results. These aren't theoretical; they're operational risks that affect your ISO/IEC 27001 risk treatment plans and your SOC 2 risk assessment documentation.
Action Items by Priority
Priority 1 (This Quarter): Document your current AI tool usage in security operations. For each tool, define: what task it performs, what human validation you apply, who's accountable for the output, and what evidence you retain. Update your ISMS procedures to reflect this reality.
Priority 2 (Next Quarter): Revise your competence requirements (ISO/IEC 27001 Clause 7.2) for security roles. Define what "AI orchestration" means in your context. Build training that covers AI tool limitations, validation techniques, and accountability frameworks.
Priority 3 (Within Six Months): Update your risk assessment to include AI-specific risks. Add entries for: reliance on AI-generated findings, potential for adversarial manipulation of AI tools, and gaps in AI model coverage. Define corresponding risk treatment plans.
Priority 4 (Ongoing): Establish a validation cadence for AI-generated security work. Don't wait until audit season to discover that your AI pentesting tool missed an entire class of vulnerabilities. Run periodic human-led reviews of AI findings to calibrate your confidence in the tools.
