Your security team monitors for data exfiltration. You've got data loss prevention (DLP) rules, cloud access policies, and endpoint controls. But when an attacker moves through your environment after an initial breach, are you tracking what AI-related assets they're collecting?
Most organizations aren't. That's why AI model theft, credential harvesting, and cloud compute hijacking are succeeding at scale.
Why These Mistakes Keep Happening
The issue isn't a lack of security awareness. It's that AI assets don't fit neatly into existing data classification schemes. Your team knows how to protect customer databases and source code repositories. But what about RAG pipeline configurations, fine-tuning datasets, API keys for third-party AI services, and model weights stored in object storage?
These assets exist in a gray zone. They're not traditional intellectual property or operational data, and they're not covered by most data protection policies. So when threat actors move laterally through your environment, they exfiltrate these assets without triggering controls designed to catch database dumps or source code theft.
The Google Threat Intelligence Group observed adversaries targeting proprietary AI models and source code, exfiltrating API credentials, and co-opting victim cloud environments to sustain unauthorized AI workloads. This isn't theoretical. It's happening during routine intrusions across healthcare, government, military, and media organizations.
Mistake 1: Treating AI Credentials Like Standard API Keys
Why it happens: Your credential management policies were written before AI services became infrastructure. Teams provision API keys for AI platforms the same way they'd handle a payment gateway integration, with basic rotation schedules and access logging.
The consequence: In one incident, attackers compromised an AI media generation company and stole proprietary source code, prompts, skills, model scripts, and secrets. Those credentials enabled the attacker to extract model capabilities at scale. Google observed campaigns involving more than 100 million prompts targeting audio, video, and image generation capabilities through compromised account credentials.
The fix: Implement separate access tiers for AI service credentials. High-privilege keys that can query models at volume or access training infrastructure require multi-party authorization, hardware token authentication, and real-time usage monitoring. Set hard rate limits per credential, not just per service. Monitor for prompt patterns that suggest distillation attacks, such as systematic queries across capability boundaries or unusually high output token consumption.
Mistake 2: Leaving Model Artifacts in Standard Cloud Storage
Why it happens: Data scientists and ML engineers work fast. They save model checkpoints, training datasets, and fine-tuning runs to the same S3 buckets or Azure Blob storage they use for application data. Your cloud security team applies standard access policies, which weren't designed for the sensitivity of model weights.
The consequence: When attackers compromise cloud credentials during an intrusion, they discover model artifacts alongside everything else. In a healthcare breach, threat actors stole drug research and corporate data, including a proprietary AI model. The model itself represented years of research investment but was protected with the same controls as quarterly reports.
The fix: Establish a separate storage tier for AI artifacts with mandatory encryption at rest using customer-managed keys, not platform-managed ones. Require separate IAM roles for model storage access that can't be assumed by standard application service accounts. Implement object-level audit logging with alerts for bulk downloads or cross-region replication. Your backup and disaster recovery processes should treat model weights as you would treat encryption keys, not as you would treat application data.
Mistake 3: Not Monitoring for Unauthorized AI Workloads
Why it happens: Your cloud cost monitoring looks for billing spikes. Your security monitoring looks for known-bad infrastructure. Neither is configured to detect when an attacker deploys LLM inference workloads using your compromised compute quotas.
The consequence: The China-based cyberespionage group UNC6508 compromised cloud environments to deploy LLM infrastructure for its own use, researching how to deploy LLMs locally and investigating vulnerabilities in AI models. This "LLMjacking" lets attackers avoid the cost barrier of premium model access while using your infrastructure for offensive operations.
The fix: Create baseline profiles for GPU and high-memory compute usage across your environment. Alert on new container deployments or virtual machine instances that match LLM inference patterns, such as specific memory-to-CPU ratios or sustained high-bandwidth network activity. Monitor for downloads of popular model frameworks or weights from public repositories to production cloud accounts. Your cloud security posture management tools should flag any compute instance running longer than your typical batch job duration in regions where you don't normally operate.
Mistake 4: Ignoring AI Assets in Incident Response Playbooks
Why it happens: Your Incident Response Plan covers database breaches, ransomware, and business email compromise. It doesn't include procedures for determining whether attackers exfiltrated model training data, stole fine-tuning configurations, or copied agent instruction sets.
The consequence: During containment and recovery, your team focuses on traditional indicators. You rebuild compromised systems, rotate credentials, and restore from backups. But you don't assess whether the attacker extracted enough information to replicate your AI capabilities or maintained access to your AI service accounts. Mandiant investigated breaches by data extortion groups that specifically targeted AI models, skills, prompts, and source code, suggesting these assets have clear monetization value.
The fix: Add an AI asset inventory section to your incident response runbooks. When an intrusion is detected, your response checklist should include: review of all AI service account activity for the exposure window, analysis of prompt logs for systematic extraction patterns, verification that model artifacts weren't accessed or copied, and rotation of all AI platform credentials regardless of direct evidence of compromise. Your forensic timeline should track access to training infrastructure, not just production systems.
Mistake 5: Assuming AI Security Is an AI Team Problem
Why it happens: You've got a data science team or an AI center of excellence. Security leadership assumes they're handling the protection of their own assets, the same way application teams manage their own code security.
The consequence: AI teams optimize for experimentation velocity. They're not thinking about how an attacker who compromises a developer laptop could pivot to training infrastructure. They're not considering how exposed API keys in configuration files become entry points for automated credential harvesting. One financially motivated threat actor used compromised cloud infrastructure credentials to deploy an autonomous multi-agent attack framework, planning and executing a mass credentials harvesting campaign in less than six hours.
The fix: Your security architecture review process must include AI infrastructure as a mandatory component, not an optional add-on. Every AI service integration requires the same threat modeling you'd apply to a payment processor or identity provider. Your privileged access management program should explicitly cover AI platform administrative roles. And your security awareness training needs to address AI-specific risks for developers, not just generic phishing scenarios.
Prevention Checklist
Use this as a quarterly review framework:
- AI service credentials are inventoried separately from standard API keys
- Rate limits and usage quotas are enforced per credential, with alerts on anomalies
- Model weights and training data reside in dedicated storage with customer-managed encryption
- Cloud monitoring alerts on GPU/high-memory instance creation in unexpected regions
- Incident Response Plan includes AI asset assessment procedures
- AI platform administrative access requires multi-factor authentication and approval workflow
- Prompt logs are retained and analyzed for systematic extraction patterns
- Security architecture review covers AI infrastructure for every new model deployment
- Backup and recovery procedures treat model artifacts as sensitive cryptographic material
- Threat intelligence feeds include indicators specific to AI credential abuse and LLMjacking
The threat actors targeting your AI assets aren't using novel techniques. They're applying standard post-compromise tradecraft to a new category of high-value targets. Your existing security controls can protect these assets, but only if you recognize them as assets worth protecting in the first place.
