Some Thoughts On Threat Modeling, Exploitability, and Agents/AI

A number of interesting data points have come out over the last few months about the latest foundation models and their bug/vulnerability finding capabilities, Agent Security, time-to-exploit for Zero-Days, and just this last few weeks the OpenAI/Hugging-Face Incident.

Going to try to pull some of these together, ask a few provocative comments, and figure out the implications.

Tl;dr - Foundation Models along with Agents used for attacks will fundamentally challenge our current assumptions about internet security and threat modeling.  Robert Graham is one of the folks I think who has most clearly said - “By any rational measure, the Internet is indeed secure enough.” Foundation Models paired with Agents as attackers might be about to fundamentally change that and upset the apple cart.

Let’s look at some of the underlying shifts that are happening right now. Historically, the data has shown that most bugs and vulnerabilities in the real world simply never get exploited. In fact, many organizations are so confident in this empirical trend that they’re putting their money where their mouth is, relying on warranties and insurance like those from Root Evidence. Because we can’t possibly patch every single CVE without spending all of our time chasing our tails, we’ve leaned on the assumption that we only need to worry about the ones that actually present an active threat.

But that assumption is starting to crack. While there is a lot of hype around AI tools finding bugs, today they mostly rely on having direct access to source code. It’s one thing to have a model inside your environment scanning your own codebase—where I’d argue the bigger risk is still vulnerabilities in the open-source dependencies you use—but it’s quite another for an external attacker to gain that same visibility. That said, even without looking at source code, AI agents are becoming incredibly efficient at exploiting the vulnerabilities that are already known.

This shift means our threat modeling has suddenly become much more complicated. Most people don’t need to turn on "Lockdown Mode" on their iPhones because it’s a specific defense against high-tier nation-state actors like the NSO Group, and it comes with significant usability trade-offs. However, if automated agents can start attacking any vulnerability they find, the luxury of ignoring "unexploited" bugs disappears. We’re moving toward a world where the sheer scale of automated attacks could make the current state of the internet—where we all just live with thousands of bugs—completely untenable.

So, we have to ask ourselves which world we’re actually living in now. Are we still in a place where we can rely on historical data to prioritize only a handful of vulnerabilities, or are we entering a new reality where any and every vulnerability is a target waiting to be hit?

What is old will be new again

 There has been a lot of focus the week on so on session-token theft and IP restrictions to help mitigate stolen session tokens.

I see that as a useful belt+suspenders approach right now - but I'm reminded that many years ago when we started making significant progress against phishing, attackers moved to malware, session theft, but then ultimately to Man-in-the-Browser (MitB) attacks. Sessions tokens like this at financial institutions that had short lifetimes weren't very useful - so attackers just got persistence on end-user devices. https://en.wikipedia.org/wiki/Zeus_(malware)

For enterprises the battle here isn't against session-theft per-se - it is against malware. Because attackers are going to - just like they did last time - migrate to more real-time exploitation/use of sessions rather than stealing cookies and reselling them in an ecosystem.

The recent session theft attacks should be a wake-up call to folks not just to look towards better session cookie protection (https://blog.chromium.org/2024/04/fighting-cookie-theft-using-device.html) but also to ensure that you're tackling your malware exposure because attackers aren't going to give up once session tokens are hard to steal - they're just going to modify the malware that is today stealing session tokens to instead do exactly what Zeus did before.

Whose credentials are they? Mine, or yours?

I've been spending a bunch of time lately thinking about usernames and passwords, and other types of credentials, and concept of "ownership".

When you get a credit card, on the back it typically says something like - "Your card is issued and serviced by XYZ Bank pursuant to a license from Visa USA.  Its use is subject to the terms of your Cardmember agreement".

The credit card isn't really your property, it is the property of the bank, and you are just being allowed to use it for payments.

When you sign up for an account online and create a username and password, that website has a decision to make:

1. Those credentials belong to the website.  They aren't the users property, they are the property of the website and their use, etc.  is subject entirely to the terms-of-service of that website.
2. Those credentials belong to the user.  Their use, when the user should use them, where else the user uses them, etc.  are entirely in control of the customer.

Since users often (always?) reuse credentials across websites, etc.  any individual websites attitude towards user credentials is dictated a lot about how they view user credentials.

A website that would like to pretend that credential reuse doesn't occur, or isn't its concern, might not protect them in the same way as a website that believes users maintain a sort of property interest in those credentials, might use them at other sites, and only the user themselves can make a decision about exactly how important those credentials are.

I'm not suggesting that one is right or wrong, but that I think this attitude towards credentials and who owns them can play a major role in how websites view their rights and obligations as it relates to their users.

Why do people expect so much more from mobile platforms?

Reading Veracode's recent post: Mobile Security – Android vs. iOS, which is an infographic comparing Android and iOS security, I'm left with a few questions, some of which I posted as a comment on their site.

While the graphic does a good job of summarizing the notable differences between these two mobile platforms, I think it approaches the problem with a set of underlying assumptions:

1. They assume that mobile platforms are fundamentally different that desktop platforms, in terms of what services/facilities/etc.  they should provide.
2. The assume a different/new/enhanced level of responsibility by the mobile platform vendor for security and privacy than we've typically expected from platform providers.

For example, in the section on basic security capabilities they say - "Security and privacy aren't thoroughly tested and unauthorized access to sensitive data has already occurred in both the App store and Android Marketplace."

While this is undoubtedly true, the same can be said about the PC, the Mac, Linux, and any other software/OS platform that is "open" and doesn't try to control and lock down all third-party software distribution.   

Perhaps the underlying argument is that new platforms should come with more security controls and the ecosystem should be more secure and guaranteed to be so by the platform provider.  I haven't seen those promises made explicitly by mobile platform vendors though they do make it implicitly a lot of times.  

Mostly what I see are people expecting much more from their mobile phone platform than they do from their desktop/laptop platform, and I'm not entirely sure why.  Are there a few new threats?  Sure.   Location privacy, and the ability to perform actions that cost money.  The latter not really being new though as malware that used people's modems to call premium phone numbers is a pretty old-school attack.

I'm all for platforms themselves becoming more secure over time.  Most/all of the mobile platforms have made huge strides in this area over legacy desktop platforms.  

What I don't quite understand is why folks are trying to hold mobile platforms to a higher standard for third-party software that it isn't clear they should be in the business of policing in the first place.

Malware prevalence != Infection rates

There have been a number of presentations of late that have tried to document howend-users get infected with malware.

Both Google's malware report and a recent report from CSIS purport to tell us how people get malware, based on how what malware they detect most frequently online, and what exploits it uses to get onto a client machine.

Google goes so far as to say:

Social engineering has increased in frequency significantly and is still rising. However, it’s important to keep this growth in perspective — sites that rely on social engineering comprise only 2% of all sites that distribute malware.

Google may well be right in the numbers they are reporting (I don't doubt their analysis) but this number tells us nothing about the frequency with which users encounter those malicious sites that employ social engineering to infect users.

Percent of sites on the internet is not directly correlated to a sites popularity. As a quick thought experiment, what if facebook.com or twitter.com or even google.com were distributing social-engineering malware. They would represent a very small percent of total websites, and yet a tremendously large number of users.

My hope is that companies such as FireEye can provide the world some details on exactly what exploits they are seeing with that frequency (have they already done that?), but even there the numbers in a corporate environment may not align that well with what a home-user sees, as many companies that deploy FireEye also do web-filtering that prevents users from ever visiting certain types of sites.

The bottom line is that right now we can approximate what causes infections by looking at what the attackers are doing, but we don't truly know which of those attacks are having success and at what frequency.

If someone has more data to provide on that, I'm all ears...

Combating Cybercrime

Cross-posting this to my personal blog as I'm sure some folks that see this, don't see the other blog: http://www.thesecuritypractice.com/

We've just published a whitepaper titled "Combating Cybercrime: Principles, Policies, and Programs".

You can read a quick summary at this blog post, or download and read the paper itself. While we don't believe we have all of the answers to combating crime online, we do believe we've presented a set of principles as well as several workable policy and technology options that will help make progress against this problem.

Please do let us know your thoughts.

Thank you

[Non-Security]Please Help Fight Leukemia

Hello,

I don't that often use my blog to talk about non-security topics but today I'm making an exception. Last April Leukemia became a very personal topic for me and my family. If you'd like to learn more, please check out: http://svmb.heros.llsevent.org/Elise