Skip to content

Offensive testing

Can AI do penetration testing?

Parts of it, and the parts divide more cleanly than the argument usually allows. A penetration test is not one activity but a sequence: enumerate the surface, generate hypotheses about what might be exploitable, attempt exploitation, chain what worked into a path, judge what the path means for the business, and write it down so someone can fix it. Machines are now genuinely better than people at the first and fourth of those, competitive at the third, and not close on the fifth. The interesting consequence is not that AI does or does not do pentesting; it is that the economics of the middle steps changed, which changes what a test can cover in a fixed budget. This page walks the sequence step by step and says which side each one falls on, with the industry's own risk documentation as the reference rather than vendor claims.

Machines win
Coverage
Humans keep
Impact
Unverified is
A hypothesis
Changed
The economics
Written byRobin ÖsterdalFounder & CEOReviewed byMalthe Bang NorengaardCo-founder & CTO

Reviewed against OWASP and NIST primary sourcesLast reviewed 4 min read

Kort sagt

  • A pentest is a sequence of six activities, and AI is strong on some and weak on others.
  • Enumeration and path chaining are where machines are clearly ahead, because both are volume problems.
  • Business impact is where they are weakest, because it depends on context no scanner has.
  • An unverified finding is a hypothesis. Shipping hypotheses moves triage cost to the customer.
  • The real change is budget: broader coverage for the same money, not a cheaper version of the same test.
  • OWASP names misinformation as LLM09:2025, and a confident wrong finding is the specific hazard here.
The sequence, step by step
Machineat volumeOperatorwith context
Enumerate the attack surface~
Generate exploitation hypotheses
Attempt exploitation
Chain findings into a path~
Judge business impact
Confirm the finding is real~
Decide what not to touch

Partial for the operator on enumeration and chaining is not a competence judgement. It reflects that both are volume problems, and a person doing them exhaustively is a person not doing the last three rows.

Källa: NIST AI Risk Management Framework

Where machines are clearly ahead

Enumeration is a coverage problem. Every host, subdomain, endpoint, parameter and version is a thing to be listed, and the failure mode of human enumeration is not error but fatigue. A system that does not get bored produces a more complete map, and completeness is most of the value of that phase.

Path chaining is the second, and it is the less obvious one. Individually unremarkable findings become serious when they compose, and finding the composition means holding many partial results in mind at once and testing combinations. That is the shape of problem machines handle well and people handle in the order they happened to look.

The consequence is a coverage change rather than a speed change. A test that previously sampled a large estate because sampling was all the budget allowed can now cover it, and the findings that emerge from the parts nobody used to reach are frequently the interesting ones.

AI-driven penetration testing delivered as an engagement. Our tooling drives the attack; a Cryvanta operator scopes it and verifies every finding.

Where they are not close

Business impact is the clearest gap. Whether an exposed administrative interface is a critical finding or an accepted risk depends on what sits behind it, who uses it, what compensating controls exist, and what the organisation has already decided to tolerate. None of that is present in the traffic, and a confident severity rating produced without it is a guess with a number attached.

The second gap is knowing what not to touch. A test runs against a live estate, and judgement about which actions are safe in that estate at that hour is contextual. This is the same boundary problem that shows up in automated response, and it has the same answer: a scoping decision made by a person in advance.

The third is verification. An exploitation attempt that appears to succeed may have succeeded, or it may have hit a honeypot, a stale record, or an artefact of the testing setup itself. OWASP lists misinformation as LLM09:2025, and this is where that risk lands in offensive work: a fluent, well-evidenced, wrong finding costs a customer more than no finding, because it consumes engineering time on a fix for something that was never broken.

What an unverified finding costs downstream
StageWith verificationWithout
Report deliveryFindings that reproduceA list including false positives
Customer triageFix workRe-testing the report first
Engineering timeSpent on the fixSplit between fix and disproof
Trust in the next reportMaintainedEvery finding now suspect
Evidence for a regulatorReproducible casesA document that did not hold up

Källa: OWASP Top 10 for LLM Applications 2025

What this means for scoping a test

If coverage is the thing that improved, scoping should move with it. A test scoped as a sample of an estate was shaped by an economic constraint that has partly lifted, and continuing to scope that way spends the gain on nothing.

The questions worth asking a provider change accordingly. Not how fast the test runs, but how much of the estate it covers and how that was decided. Not whether AI is used, but which steps it performs and which a person performs.

One question does most of the sorting: who verifies a finding before it reaches you, and what does verification mean here? A provider whose answer is a confidence score has told you the triage is still yours.

The attack surface a test is measured against

21.3 %
of European intrusions began with vulnerability exploitation, the part a test can find

Källa: ENISA Threat Landscape 2025

60 %
began with phishing, which testing reaches only if social engineering is in scope

Källa: ENISA Threat Landscape 2025

9.9 %
came from botnets, and a further 8 % from malicious applications

Källa: ENISA Threat Landscape 2025

53.7 %
of recorded incidents involved essential entities under NIS2

Källa: ENISA Threat Landscape 2025

Where the regulated buyer sits in this

For an organisation inside NIS2 scope, testing is not an end in itself. Article 21 requires policies for assessing the effectiveness of risk management measures, and a test is the usual way to make that assessment concrete rather than asserted.

That framing changes what a good deliverable is. A regulator reads for whether an assessment happened, what it covered, and what was done about what it found. Reproducibility matters more than volume of findings, and a report full of unverified items is weaker evidence than a shorter one that holds.

It also means the scoping decision is itself part of the record. What was in scope, what was excluded and why, and who decided are the questions that come after the findings, and they are easier to answer if they were written down before the test rather than reconstructed after it.

Questions

Common questions

Can AI perform a penetration test?
It performs parts of one well: enumeration, exploitation attempts and chaining findings into a path. It does not reliably judge business impact or decide what is unsafe to touch in a live estate.
Is an AI pentest as good as a human one?
It is better on coverage and worse on judgement. The useful comparison is not one against the other but which steps each performs, since most credible offerings combine both.
What is the main risk of AI-driven testing?
Confident wrong findings. OWASP lists misinformation as LLM09:2025, and in offensive work that means a fluent, well-evidenced result that does not reproduce, consuming engineering time on a fix for nothing.
Does a pentest satisfy NIS2?
No single activity does. Article 21 requires policies for assessing the effectiveness of measures, and a test is a common way to make that assessment concrete. Reproducibility matters more than the number of findings.
What should we ask a provider?
Which steps are automated and which are performed by a person, how much of the estate is covered and how that was decided, and who verifies a finding before it reaches you.

Primärkällor

Källor

Varje regulatoriskt påstående på den här sidan går att spåra till en av källorna nedan. Ingen av dem är en konsultblogg.

Further reading

Working out what to ask a vendor in this category?

We will go through your evaluation criteria with you, including the questions that make us look worse. Half an hour, no preparation needed.