Threat actors are using AI tools to scan public GitHub repositories for credentials. Previously, human intervention was needed to find, read, and test these keys. Now, AI can do it all in minutes. Microsoft Threat Intelligence reported a case where an AI agent moved from scanning GitHub for secrets to exfiltrating data from Azure Key Vault in just five minutes.
For Enterprise Risk Officers, the question has changed. It's not just about having a secrets policy. It's about proving that exposed secrets are found and neutralized faster than an AI can exploit them. This checklist outlines the controls needed to meet that challenge. Each item has a clear state and a description of what success looks like. Map them to your existing obligations under NIST SP 800-53 (IA-5 authenticator management, SA-11 developer testing), CIS Controls v8.1 (Control 3 data protection, Control 5 account management, Control 6 access control management), and ISO/IEC 27002 as required by your auditors.
Prerequisites
Before you begin, ensure you have:
- An inventory of every GitHub organization and Azure DevOps project your company owns, including forgotten and personal-account repositories used for work.
- A named owner for non-human identities (service principals, OAuth applications, API keys).
- Log retention for Entra ID sign-ins (including service principal and non-interactive sign-ins), Azure Activity, Key Vault diagnostics, and Microsoft 365 unified audit.
- Authority to rotate credentials without a multi-week change window.
If any of these are missing, address them first. Every item below depends on them.
The Checklist
1. Scan every repository you own for committed secrets.
Run secret scanning across all GitHub repositories and enable GitHub Advanced Security for Azure DevOps. Done means every repository is in scope, including archived ones. Good looks like: a current coverage report showing 100% of repositories scanned, with exceptions documented and approved.
2. Turn on push protection.
Push protection blocks supported secrets before they reach the repository. Done means push protection is enabled at the organization level. Good looks like: blocked-push events appear in your metrics, and bypasses require a logged justification.
3. Enable public monitoring beyond your own repositories.
Secrets can leak into secondary public repositories. GitHub's public monitoring extends detection beyond repositories you own. Done means the feature is on and alerts route to a staffed queue. Good looks like: a named responder triages these alerts under a defined service level.
4. Treat every exposed secret as compromised and rotate it.
Removing a secret from code doesn't fix the issue. Exposed credentials remain usable until revoked or rotated. Done means each finding closes only on rotation or revocation. Good looks like: every closed finding links to a rotation record with a timestamp.
5. Migrate secret-based authentication to managed identities.
Managed identities remove the secret from the codebase entirely. Done means you have a prioritized migration backlog for every application using client secrets. Good looks like: a falling count of secret-authenticated applications reported as a Key Risk Indicator (KRI) each quarter. Align with NIST SP 800-63 guidance on authenticator lifecycle.
6. Enforce least privilege on workload identities.
A single compromised application credential can open multiple services. Done means you've reviewed every service principal's role assignments against actual need. Good looks like: no service principal holds broad subscription-level rights without a documented exception.
7. Restrict what a Key Vault reader can reach.
Attackers target operations that return credentials. Done means you know which applications can read which vaults, and the list is short. Good looks like: a rotation runbook per vault, so a compromised reader triggers a known, tested sequence.
8. Apply Conditional Access to workload identities.
Named-location blocks for known malicious addresses can be applied to service principals. Done means you've confirmed licensing and scope. Good looks like: a documented policy and a record of which applications it doesn't cover.
9. Hunt in non-human sign-in logs.
Review service principal and non-interactive sign-in logs. Done means these queries run on a schedule and alert a human. Good looks like: a documented hunt cadence and a tuned alert.
10. Detect adaptive API behavior.
Machine-speed retry-and-adjust patterns against control-plane APIs are a detection signal. Done means your SIEM has a rule for clusters of HTTP 400 responses followed by success. Good looks like: a tested detection with a defined response owner.
11. Rehearse the containment sequence.
Containment means removing all client secrets, certificates, and federated credentials. Done means the sequence is written into your Incident Response Plan under NIST SP 800-61 and exercised in a Tabletop Exercise. Good looks like: the team can state how long full containment takes because they've timed it.
12. Assess vendor-owned applications.
Your exposure includes your suppliers' secret hygiene. Done means critical vendors with tenant access answer secret-management questions in due diligence. Good looks like: contract language requiring prompt notice of credential exposure.
13. Put AI credentials in scope.
AI keys should be in the same inventory, scanning, and rotation regime as every other secret. Done means no AI credential exists outside the inventory.
Common Mistakes
- Deleting instead of rotating. The commit history and any copies persist. Deletion feels like closure; it isn't.
- Scanning only the main branch or only current repositories. Secrets live in history, forks, and archived projects.
- Assuming a policy equals a control. A written rule against hardcoded secrets prevents nothing without push protection.
- Ignoring service principal logs. Most monitoring programs focus on human users; attacks often target non-human identities.
- Skipping the vendor angle. Your tenant can be reached through someone else's leaked credential.
Next Steps
Score each item done or not done, then rank the gaps by blast radius, not by ease. Items 4, 5, and 6 usually carry the most risk reduction. Report the managed identity migration rate and the time from secret detection to rotation as KRIs, and ask your security leadership, whether in-house or a Virtual CISO (vCISO), to bring both numbers to your next risk committee. Agentic AI shortened the attacker's timeline to minutes. Your remediation timeline needs a number, an owner, and evidence.
