Your security team should be able to turn a database claim into a test it can inspect. Which executable ran? What did it check? What result would count as a failure? The answers should travel with the evidence.
That is the evaluation approach we are building for 8DB, our omnimodal database. In a recorded September 17 replay, its generic Linux evaluator passed all 31 catalog tests and all 31 frozen-baseline comparisons after fresh package extraction. The run used prebuilt product binaries, without mounting product source, in network-disabled containers.
Built so an evaluator can rerun it offline. The package connects each claim to an executable check and its retained result. The external handoff is being prepared around that repeatable process.
For an assurance team, the immediate value is a concrete starting point: named claims, exact test assignments and results that can be traced back to a particular build.
Four questions worth testing
The generic catalog groups its formal tests around four questions. Each question has declared boundaries, so the result can inform a real deployment decision.
| Question | What the selected tests examine | Boundary that matters |
|---|---|---|
| Does protected storage preserve the tested data? | Record and index operations, sealed persistence, marker scans and reopen behavior | The recovery case closes and reopens the store. It does not establish forced-crash or physical power-loss recovery. |
| Do governance rules affect access? | Tested clearance, compartment and epoch boundaries under the declared root configuration | The trusted root and host remain part of the security model. Network isolation is checked separately. |
| Does sealed transfer reject the tested attacks? | Authenticated envelopes, tampering, wrong keys and replay cases | This covers the declared primitive and in-process two-store transfer, rather than a complete deployed network. |
| Can a recipient verify committed data? | Signatures, signed Merkle membership and specified tampering cases | Authenticity and membership concern the signed record; an inspection or sensor observation still needs appropriate source trust. |
The catalog assigns 26 tests to those claims and five to supporting capabilities. All 31 must be present for complete-catalog acceptance. A supporting result cannot repair a failed formal claim, and an omitted test cannot disappear into an overall success label.
These questions connect to the product's native capabilities, including protected records, graph relationships, vector and lexical indexes. They establish the behavior of the listed API paths and fixtures. A new workload or integration deserves its own acceptance criteria.
Make the result repeatable
The retained successful replay used product source identity 29dd1584e, a pinned binary and a package manifest. Two builds of the transport archive produced the same digest. The evaluator then ran from a fresh extraction with its kit mounted read-only and network access disabled.
The procedure records the environment, commands, original output, process exits and result files. It executes the product tests, compares results with the frozen baselines, and checks the accounting for the complete catalog. Baselines stay fixed during evaluation.
That separation is useful. A test can produce a plausible result while the surrounding evidence is incomplete. In the retained first attempt, the product tests and baseline comparisons passed, but the accounting check refused an unexpected launcher warning. The corrected attempt supplied the test account with its own home directory and reran the product phases. The binary, fixtures and acceptance thresholds stayed unchanged.
The original attempt remains in the evidence history. A reviewer can see what changed and why the later result was accepted.
Give each finding a precise meaning
Agree the disposition rules before the run. A useful claim review distinguishes four outcomes:
- Pass: the required checks completed and met the agreed criteria for the exact artifact and environment.
- Fail: a controlled check produced evidence that the tested requirement was not met.
- Inconclusive: missing evidence, an uncontrolled deviation or an incomplete required check prevents a decision.
- Not tested: the check was deferred or never began.
The visual lets you explore those distinctions. Its cases illustrate a review policy; they are not additional product tests or a live certification service.
Evaluation workflow · illustrative review policy
A result you can trace back to its conditions
Change the review scenario:
All required checks meet the agreed criteria for this artifact and environment.
Next action: Retain the evidence and report the bounded result.
Keep independent review independent
A source-free package lets an evaluator test product behavior without receiving the proprietary implementation. It still requires trust in the supplied executable, the adequacy of the tests and the environment. Inspectable harnesses and raw records give the evaluator more to challenge than a vendor's summary alone.
The external handoff is being prepared as a named, authenticated package for an agreed environment. Its manifest, prerequisites and retained outputs give the evaluator a clear starting point and a repeatable route through the checks.
Put product evidence to work
The evidence can help a program describe how a component contributes to its security requirements. NIST's OSCAL Component Definition model supports reusable implementation descriptions; its Assessment Results model records findings and supporting evidence. The distinction keeps proposed implementation, observed results and system authorization in their proper places.
Published algorithm checks and interoperability results give the evaluator another useful evidence layer. Match the evaluation package to the assurance requirements of the intended deployment.
The generic database evaluator and the separate PQC lifecycle kit also have different artifacts and acceptance scopes. A result from one cannot be added to another to declare a combined package qualified. Keeping those identities explicit helps a team choose the evidence its deployment actually needs.
Bring one requirement your team must defend
Choose a database requirement whose failure would matter: protected persistence, a governance boundary, a sealed transfer or a verifiable record. Tell us the target environment and the evidence your team needs to accept it.
Request an 8DB evaluation-scope discussion. We can map that requirement to the existing catalog, identify additional tests and agree the prerequisites for a controlled handoff. The aim is a decision your team can explain, backed by a run it can inspect and ultimately repeat for itself.
Evidence scope 8Braid generic Linux replay · 31 tests and baseline matches · Independent handoff in preparation. Explore the evidence. Methods and retained scope notes.
Continue the technical conversation
Where could this help your work?
Bring a research question, a database workload or an application you want to build. Let’s connect the ideas in this article to an evaluation that matters to your team.
Discuss this work