Exposed Secrets, Broken Trust: What the DOGE API Key Leak Teaches Us About Software Security

Earlier this week, Krebs on Security reported that an employee of the Department of Governmental Engineering (DOGE) accidentally published a secret API key that granted access to unreleased AI models from Elon Musk’s xAI venture source. Even more disturbing: researchers found the DOGE API key still worked after Elez was notified of the leak and had not been revoked.

This wasn’t the first time we’ve seen secrets slip into public view, and it won’t be the last. Exposed keys, passwords, tokens, and certificates have become a systemic problem, buried in code and firmware across the digital landscape.

To unpack the implications of the DOGE API key leak and what your organization can do to prevent similar incidents, I sat down with Tom Pohl, LMG Security’s Director of Penetration Testing. A couple of years ago, Tom gave a DEFCON talk called “Private Keys in Public Places,” and his team regularly uncovers secret keys hidden in code, both by accident and by design.

“Secrets don’t belong in code. Period.” The DOGE API Key Error.

Sherri: Tom, when you saw the DOGE API Key leak, what was your first reaction?

Tom: Frustration, but not surprise. Anytime you’re putting API keys in source code, you’re doing it wrong. This is a basic, well-understood mistake in software development. And the fact that the key still worked after the exposure tells me that key rotation wasn’t built in. That’s an architectural flaw.

Sherri: The DOGE API Key reportedly gave access to unreleased AI models from xAI. What could someone actually do with that kind of access?

Tom: Potentially a lot—download models, manipulate outputs, scrape proprietary data, or even impersonate services. And if the models were hooked into anything critical, that opens the door for serious downstream impact.

Hunting Keys in the Wild

Sherri: You’ve talked about how common it is that these types of secrets are exposed. Can you give us some examples?

Tom: Absolutely. In my DEFCON talk, I showed how I found Apple- and Google-blessed TLS certificates—with private keys—embedded in Fortinet firewall firmware. These weren’t expired, they were valid production certs. All I did was unzip the firmware and search for keywords. These aren’t advanced hacks, they’re basic techniques.

I’ve also found hardcoded admin credentials in network appliances, AES keys embedded in compiled Java JARs, and authentication keys in printer firmware. These issues are everywhere.

Sherri: And organizations often don’t even know they’re exposed?

Tom: That’s the scary part. If you don’t proactively test for these issues, you’ll never know a secret has leaked until someone exploits it.

How Penetration Testers Use Keys to Break In

Sherri: Can you talk about how you actually find and use these keys during your penetration tests?

Tom: Definitely. During internal or external penetration tests, if I spot that a client is running a specific vendor’s software, I’ll go download that exact version, reverse engineer it, and see if I can find any hardcoded secrets or vulnerabilities. If I find something, I’ll use it to try to gain access or escalate privileges inside the client’s network.

Sherri: So, you’re not just scanning, you’re actively dissecting vendor software?

Tom: Right. That’s one of the things that sets our team apart. We don’t just run a scanner and call it a day. We do what real attackers would do: dig into software, look for hardcoded secrets or backdoors, and leverage those to breach systems.

When Your Vendors’ Weakness Becomes Your Breach

Sherri: Can you give an example where vendor code posed a direct risk?

Tom: One time, we were testing a water utility that was running HMI software exposed to the internet. I downloaded the same software from the vendor’s site, analyzed it, and found a full backdoor—no auth required. The system used the same TLS certificate and private key across all customer installs. That meant once we discovered the issue on one client’s system, it was trivial to exploit it on every client’s system.

Sherri: Your vendors’ risk is your risk!

Tom: 100%. That’s why vendor vetting and deep-dive penetration testing are so important. You need to ask the hard questions about how software is built and make sure your environment is secure using real-world testing.

Developers Under Pressure, Security as an Afterthought

Sherri: Why do you think developers keep making the same mistakes?

Tom: Because they’re told to ship fast, not to ship securely. Secrets get hardcoded for convenience during development, and no one ever pulls them out. And many dev teams don’t get training in secure software development—education is critical, but so are guardrails like static code analysis and CI/CD checks.

Sherri: It sounds like organizations need both process and technical solutions?

Tom: Exactly. Use static analysis tools to scan code for secrets before it’s pushed. Integrate these into your build pipeline. And above all, use a secrets management system—something that separates secrets from code and allows rotation without major redeployments.

Why Secrets Management Needs to Be a Priority

Sherri: One part of the DOGE API Key story that stood out to me is that the key reportedly still worked even after it had been discovered. Why does that happen?

Tom: Because the organization didn’t have a process or the architecture to revoke and replace secrets quickly. If you can’t rotate a key without rebuilding or redeploying code, you don’t own the key—it owns you.

Sherri: What should organizations be doing to prevent this?

Tom: Build secrets rotation into your architecture from day one. And don’t wait for a breach to figure out how you’ll revoke keys. Practice it. Treat exposed secrets like a full-blown incident: revoke, investigate, and monitor.

Key Takeaways: How to Keep Your Secrets Secret

1. Never embed secrets in code or firmware. Make sure developers and vendors use a secrets manager like AWS Secrets Manager, Azure Key Vault, or HashiCorp Vault.
2. Automate scanning for secrets. Tools like TruffleHog, GitGuardian, and static code analysis platforms can flag exposures early.
3. Educate your developers and IT staff. Train your team on secure coding and software management practices, especially around secrets handling.
4. Vet your vendors. Ask for third-party code review reports, and ensure they follow secure development lifecycle (SDLC) best practices.
5. Test for exposed secrets. If you need help, LMG Security’s penetration testing services include deep analysis of software, firmware, and cloud configurations for leaked credentials and insecure authentication methods.

Don’t Let Leaked Keys Open the Door to Attack

At LMG Security, we’ve seen the devastating impact of secrets exposures firsthand—whether it’s a leaked admin credential, a firmware-embedded private key, or a hardcoded token with full access to cloud APIs.

Whether you’re developing software or integrating third-party tools, it’s time to treat secrets management as a top-tier security priority. Our penetration testing and vendor risk management services help uncover these risks before attackers do.

Let us help you find your weak spots—before they make the headlines. Contact us to schedule a call.