First fixed scans, without AI
Well-known open-source scanners at a fixed version read your repository: gitleaks for secrets in code and history, osv-scanner for vulnerable and malicious dependencies from your lockfiles, semgrep and bandit for risky code, hadolint and trivy for Dockerfiles and configuration, lizard and scc for complexity and size. The same code gives the same result: for us that is a test, not a promise.
Fixed tools also read your git history: how old the code is, how active, how many people worked on it, and whether most of the work comes from one person. A secret we find is reported with its file, line and kind, never its value.
Then an AI model reads the code, and we verify every claim
An AI model reads the code read-only, one part at a time, and hands in what it finds as claims. The model doesn't decide what you see, our code does: every claim passes twelve fixed rules before it reaches your workspace. Anything a rule rejects goes back to the model once and then to our team, never to you. A few of those rules:
- The file and the line exist, and the quote is really there
- A name in the claim, such as a function, a package, a setting or a route, exists in your code
- A claim that something is missing is checked again with a search
- A document that is much older than the code it describes can't be called current
- A claim that contains a secret, a link or an instruction to the reader is rejected
- The model must actually have opened the files it talks about; we log that, not the model
Every automated check proves itself
None of our automated checks goes live until it can prove itself: a planted test case containing the problem, which the check must find, and a clean test case in which it must find nothing. Those tests run every night and before anything of ours goes to production. If such a test case disappears, the build fails.
The catalogue holds 275 checks today, across twelve areas. Not every check is equally far along: some have no test case of their own yet and sit on a dated list of exceptions that may only shrink. The documentation catalogue is not under this rule yet.
- Code
- 12 checks
- Security
- 80 checks
- Legal and compliance
- 2 checks
- Documentation
- 127 checks
- Quality
- 9 checks
- Optimisation
- 7 checks
- Architecture
- 5 checks
- Maintainability and lifespan
- 10 checks
- Data
- 12 checks
- Continuity
- 2 checks
- Process and delivery
- 6 checks
- AI risks
- 3 checks
What you see, and what you never see
Every check gets a word with an icon, never colour alone: tested by you and checked by us, reported by you and not checked, not proven or failed, not applicable. An area in which nothing was measured is never green.
- What you see: for every finding the file and the line, the commit, the scan result or the CI report
- What you never see: the value of a secret. We don't show it and never send it to an AI model
- What we never do: change your code or fix a finding. In the pilot we don't run your tests; if that changes, the terms will say so first. You decide what to fix
- What you can do: verify every finding at the place it names, dispute it with a reason, or mark it as an accepted risk. A secret can never be filtered away
What is not proven yet
Not everything we do is equally well proven yet. This is where it stands:
- The documentation catalogue, meaning which document is present, thin or missing, is decided by the model from file names and the rules above. The first measurement on our own gold set is scheduled; until then this is our least proven part
- How often a check misses something has not been measured. We know every check finds its planted case, but not how often it misses a variant. A measurement on public, deliberately vulnerable projects is the next step
- We don't find everything. If something isn't among the findings, that doesn't mean it is fine. A check is a snapshot, not a certificate; our terms say the same