How to read a penetration test scope. The three concrete categories of finding that only manual exploitation produces — and why most "pen tests" sold to mid-market firms are vulnerability scans in a different jacket. Approximately 5 minutes.
Two out of every twelve enterprise firewalls we have tested fail a fundamental TCP split-handshake check. That is not a CVE in a vulnerability database. No automated scanner finds it. The finding only appears when a human attacker tests the firewall the way a human attacker would attack it.
This is the dividing line between a real penetration test and an automated scan packaged with a fancier invoice. Both have a place. Both are sold under the same name. Knowing which one you are buying is the difference between a defensible security posture and a paper trail.
An automated vulnerability scan runs known signatures against your environment. Nessus, Qualys, Rapid7, OpenVAS — they all work the same way: probe a target, compare responses against a database of known weaknesses, output a list with CVSS scores. The output is voluminous and looks impressive. The work is run by a tool, often supervised by an entry-level analyst whose job is to filter out obvious false positives before generating the report.
This is genuinely useful. Automated scanning catches the unpatched systems, the obvious misconfigurations, the default credentials someone forgot to change. It runs cheap and runs often, which it should — vulnerability management is continuous hygiene, not a one-time event. A mature security program runs scans weekly or daily, in addition to (not instead of) periodic penetration testing.
Three categories of finding require a human:
Chained exploitation. A single low-severity finding rarely matters. Three medium-severity findings, chained together in a specific sequence, can produce domain admin in a Windows environment. The scanner reports each finding in isolation; the human attacker chains them. The risk only becomes visible when someone executes the chain.
Business logic flaws. An application allows users to purchase items. The scanner confirms the purchase endpoint requires authentication and accepts encrypted payments. A human attacker notices that the price field is editable in the request payload and changes the price to one cent. No CVE describes that. No scanner finds it. The application "works as designed." The design is the vulnerability.
Implementation-specific weaknesses. Your firewall vendor ships with a TCP split-handshake quirk. Two out of twelve enterprise firewalls we have tested fail it; attackers who know the trick bypass rules by impersonating trusted internal connections. Your scanner does not know about this because there is no signature for the behavior. A human pen tester knows about it because that is part of current adversary tradecraft.
If you are being quoted a "penetration test," look for these tells:
The pricing is per-IP or per-host. "$50 per IP, ten IPs, $500." That is a scan price. Real penetration tests are priced by scope and objective. The cost of properly testing a single high-value application can exceed the cost of scanning the entire network because the work is different work.
The sample report is a list. Ask for a redacted sample report. If it is a CVSS-ranked list of findings with stock remediation language, you are looking at scanner output. A real penetration test report contains an executive summary that tells a story, an attack narrative that walks through how access was gained, escalated, and used, and findings with custom remediation language tied to the client's specific environment.
The engineer assignment is hand-waved. "We have a team of certified analysts." Ask: which specific person will perform this work, what is their background, and how many engagements have they personally led? At a real boutique, the answer is one name with twelve to twenty years of experience and a portfolio of engagements they can describe. At a scan vendor, the answer is a job title.
Selling automated scans as pen tests is profitable. The scan costs the vendor almost nothing to run, the client pays a pen-test rate, and the report is generated by template. Margin is excellent. The client gets a document for their compliance file. Nothing exploits the gap until something does — at which point the report's narrow scope provides plausible deniability, because "we ran a pen test, see, here's the report."
The auditors are also slowly catching on. PCI-DSS 4.0 explicitly tightens the definition of penetration testing. HIPAA OCR settlements increasingly distinguish scanning from testing in their corrective action plans. Cyber insurers are asking sharper questions on renewal. The market is correcting, slowly.
A real boutique penetration test is custom-scoped against your specific environment, executed by a senior engineer using current adversary tradecraft, and documented in language your executives can read and your engineers can act on. The deliverable is more than findings — it is a prioritized remediation roadmap and, where appropriate, a re-test included in scope to confirm fixes hold.
That kind of engagement takes time. Two to four weeks of testing for a mid-market environment is normal. The report takes another week to write. The whole thing costs more than a $500 scan and substantially less than the engagement-of-record contract a Big Four firm would propose for similar coverage.
The output is a document your auditor takes seriously, your insurer credits in the renewal, and — most importantly — your board can use to make actual decisions. That is what we mean when we say penetration testing at Caveo.
Scope a real pen test. Custom-scoped, senior-led, fixed-fee. Engagements typically run 2–4 weeks.
Talk to a partner →