Reviewed against OWASP and NIST primary sourcesLast reviewed 4 min read
Kort sagt
- A pentest is a sequence of six activities, and AI is strong on some and weak on others.
- Enumeration and path chaining are where machines are clearly ahead, because both are volume problems.
- Business impact is where they are weakest, because it depends on context no scanner has.
- An unverified finding is a hypothesis. Shipping hypotheses moves triage cost to the customer.
- The real change is budget: broader coverage for the same money, not a cheaper version of the same test.
- OWASP names misinformation as LLM09:2025, and a confident wrong finding is the specific hazard here.
| Machineat volume | Operatorwith context | |
|---|---|---|
| Enumerate the attack surface | ✓ | ~ |
| Generate exploitation hypotheses | ✓ | ✓ |
| Attempt exploitation | ✓ | ✓ |
| Chain findings into a path | ✓ | ~ |
| Judge business impact | ✕ | ✓ |
| Confirm the finding is real | ~ | ✓ |
| Decide what not to touch | ✕ | ✓ |
Partial for the operator on enumeration and chaining is not a competence judgement. It reflects that both are volume problems, and a person doing them exhaustively is a person not doing the last three rows.
Where machines are clearly ahead
Enumeration is a coverage problem. Every host, subdomain, endpoint, parameter and version is a thing to be listed, and the failure mode of human enumeration is not error but fatigue. A system that does not get bored produces a more complete map, and completeness is most of the value of that phase.
Path chaining is the second, and it is the less obvious one. Individually unremarkable findings become serious when they compose, and finding the composition means holding many partial results in mind at once and testing combinations. That is the shape of problem machines handle well and people handle in the order they happened to look.
The consequence is a coverage change rather than a speed change. A test that previously sampled a large estate because sampling was all the budget allowed can now cover it, and the findings that emerge from the parts nobody used to reach are frequently the interesting ones.
AI-driven penetration testing delivered as an engagement. Our tooling drives the attack; a Cryvanta operator scopes it and verifies every finding.
Where they are not close
Business impact is the clearest gap. Whether an exposed administrative interface is a critical finding or an accepted risk depends on what sits behind it, who uses it, what compensating controls exist, and what the organisation has already decided to tolerate. None of that is present in the traffic, and a confident severity rating produced without it is a guess with a number attached.
The second gap is knowing what not to touch. A test runs against a live estate, and judgement about which actions are safe in that estate at that hour is contextual. This is the same boundary problem that shows up in automated response, and it has the same answer: a scoping decision made by a person in advance.
The third is verification. An exploitation attempt that appears to succeed may have succeeded, or it may have hit a honeypot, a stale record, or an artefact of the testing setup itself. OWASP lists misinformation as LLM09:2025, and this is where that risk lands in offensive work: a fluent, well-evidenced, wrong finding costs a customer more than no finding, because it consumes engineering time on a fix for something that was never broken.
| Stage | With verification | Without |
|---|---|---|
| Report delivery | Findings that reproduce | A list including false positives |
| Customer triage | Fix work | Re-testing the report first |
| Engineering time | Spent on the fix | Split between fix and disproof |
| Trust in the next report | Maintained | Every finding now suspect |
| Evidence for a regulator | Reproducible cases | A document that did not hold up |
What this means for scoping a test
If coverage is the thing that improved, scoping should move with it. A test scoped as a sample of an estate was shaped by an economic constraint that has partly lifted, and continuing to scope that way spends the gain on nothing.
The questions worth asking a provider change accordingly. Not how fast the test runs, but how much of the estate it covers and how that was decided. Not whether AI is used, but which steps it performs and which a person performs.
One question does most of the sorting: who verifies a finding before it reaches you, and what does verification mean here? A provider whose answer is a confidence score has told you the triage is still yours.
The attack surface a test is measured against
- 21.3 %
- of European intrusions began with vulnerability exploitation, the part a test can find
- 60 %
- began with phishing, which testing reaches only if social engineering is in scope
- 9.9 %
- came from botnets, and a further 8 % from malicious applications
- 53.7 %
- of recorded incidents involved essential entities under NIS2
Källa: ENISA Threat Landscape 2025
Källa: ENISA Threat Landscape 2025
Källa: ENISA Threat Landscape 2025
Källa: ENISA Threat Landscape 2025
Where the regulated buyer sits in this
For an organisation inside NIS2 scope, testing is not an end in itself. Article 21 requires policies for assessing the effectiveness of risk management measures, and a test is the usual way to make that assessment concrete rather than asserted.
That framing changes what a good deliverable is. A regulator reads for whether an assessment happened, what it covered, and what was done about what it found. Reproducibility matters more than volume of findings, and a report full of unverified items is weaker evidence than a shorter one that holds.
It also means the scoping decision is itself part of the record. What was in scope, what was excluded and why, and who decided are the questions that come after the findings, and they are easier to answer if they were written down before the test rather than reconstructed after it.
Questions
Common questions
- Can AI perform a penetration test?
- It performs parts of one well: enumeration, exploitation attempts and chaining findings into a path. It does not reliably judge business impact or decide what is unsafe to touch in a live estate.
- Is an AI pentest as good as a human one?
- It is better on coverage and worse on judgement. The useful comparison is not one against the other but which steps each performs, since most credible offerings combine both.
- What is the main risk of AI-driven testing?
- Confident wrong findings. OWASP lists misinformation as LLM09:2025, and in offensive work that means a fluent, well-evidenced result that does not reproduce, consuming engineering time on a fix for nothing.
- Does a pentest satisfy NIS2?
- No single activity does. Article 21 requires policies for assessing the effectiveness of measures, and a test is a common way to make that assessment concrete. Reproducibility matters more than the number of findings.
- What should we ask a provider?
- Which steps are automated and which are performed by a person, how much of the estate is covered and how that was decided, and who verifies a finding before it reaches you.
Primärkällor
Källor
Varje regulatoriskt påstående på den här sidan går att spåra till en av källorna nedan. Ingen av dem är en konsultblogg.
- OWASP Top 10 for LLM Applications 2025— Misinformation as LLM09, the offensive-work hazard
- OWASP LLM06:2025 Excessive Agency— Scope boundaries as a permission problem
- NIST AI Risk Management Framework— The measure function and verification
- NIS2 Directive Article 21, EUR-Lex— Assessing the effectiveness of measures
- ENISA Threat Landscape 2025— The threat population a test prepares for
Further reading
