You click "run scan." Two minutes later, the report comes back. Zero findings.
The obvious interpretation is that you're clean. But there are at least four other interpretations sitting in the same pixel arrangement:
- The scanner binary wasn't installed on the worker.
- It was installed, but it crashed on startup.
- It started, but it couldn't reach the target.
- It reached the target, but the authentication headers were wrong and every endpoint returned 401.
Most vulnerability management platforms don't distinguish between these. They just show you a green checkmark and move on. That checkmark is the single most dangerous UI element in security tooling.
The failure modes are real
None of these are hypothetical. They all happen weekly, in production, in platforms that rely on third-party CLI tools for scanning. A concrete list:
- Missing binary. The SAST worker expects
semgreponPATH. Someone rebuilt the container without it. The scan returns immediately with no output. - CLI flag injection crash. A target URL contains a shell metacharacter. The scanner launches with the wrong arguments, panics, and exits non-zero. The platform swallows the exit code.
- Network partition. The scan worker couldn't resolve the target's DNS. Timeout. No findings.
- Auth drift. The API scanner's bearer token expired two weeks ago. Every request returns 401. The crawler finds zero paths to test. Zero findings.
- Docker daemon unreachable. Half the ecosystem ships as Docker images. If the daemon isn't running, the container scanner yields nothing.
In every case above, the truthful report is "I don't know whether your surface is clean — I couldn't look." The report your stakeholders receive is "you're clean."
The right response is noisy
When a tool can't do its job, the platform should say so — in the same queue as real findings, at the same visibility.
The pattern we use in Vornin is what we call a synthetic finding. If gitleaks, semgrep, trivy, or nuclei is missing or crashes mid-scan, the platform emits a finding:
- Severity: Low (it's not a vulnerability, but it's also not nothing)
- Category: "Scanner Unavailable"
- Title: concrete — "Gitleaks binary not found on worker," not "scan failed"
- Evidence: which host, which tool, which exit code, which command (credentials redacted)
- Remediation: install the binary, or route the scan through an agent that has it
This finding lands in the same triage queue as everything else. It shows up in the executive dashboard. It counts against MTTR. It gets an SLA timer. It cannot be ignored, because it looks exactly like a real problem — because operationally, it is a real problem.
Why "clean" and "I didn't look" must not share a render
Here's the test: imagine a CISO who never logs into the tool. They get a weekly PDF summary. The summary says "12 scans completed this week, 3 critical findings, 41 total."
If the SAST scanner silently failed all 12 times, that same PDF looks better. Zero SAST findings. Zero SAST work to do. Cleaner report.
The platform has now optimized for the wrong thing. Quieter reports are not better reports — they're a metric the tool is accidentally gaming.
What to ask your current vendor
If you're evaluating your current tooling (or a replacement), ask these three questions:
- "What happens when a scanner binary is missing?" The acceptable answer: a finding appears. The unacceptable answer: "it's logged in the backend."
- "What happens when a scan worker's network is partitioned during a scan?" Same test. An unreported failure is the wrong answer.
- "Show me an example of a scan where a tool failed. What does the report look like?" If they can't show you one, they don't distinguish.
Silent success is a design choice. It's usually made by engineers who are optimizing for a demo, not for a Tuesday morning when everyone is already distracted.
Your vulnerability management platform should be loud when it can't do its job. Everything else you believe about your security posture hinges on it.