Skip to content

Idea

The Dataset Doesn't Have to Change Hands

A data owner could sell twenty executions of an approved analysis without distributing the underlying dataset. The useful product is a permitted use, with clear limits on what leaves.

Published
Updated
Reading time5 minutes
Data governanceAI agentsPrivacyDigital Fabric

Imagine a manufacturer with five years of equipment failures, maintenance records and operating conditions. Another company has an analysis that might help it predict which parts will fail next.

There is a useful transaction here. There is also a difficult request: send over the dataset.

Those records may contain production volumes, supplier details and information about customers. The manufacturer might be willing to let someone answer a particular question while being unwilling to distribute everything needed to answer it. A confidentiality agreement matters, but it does not remove the practical consequences of creating another copy.

I think there is a product worth building around that distinction: a permitted analysis, sold with a defined allowance and a clear rule about what can leave.

The offer could be quite ordinary. Twenty executions of an approved analysis, against a specified dataset version, within thirty days. The buyer receives an agreed report. The computation takes place in an environment controlled by the data holder.

The buyer pays for an answer under agreed terms. The software needs to enforce those terms as precisely as the contract describes them.

At 8Braid, we have been examining this as a possible application of the technology behind Digital Fabric. Digital Fabric is our implementation of governed transactions using 8DB. The underlying components include signed entitlements with scope, budget and expiry, agreement checks before redemption, and persistent records of what has been consumed. These provide a foundation for a programmable transaction. Turning them into a data-analysis service would require an integration and a measured pilot.

For this application, I would want the permission to identify the dataset revision, approved analysis, available allowance and output policy. A request should be checked against the current agreement before it is admitted. If two workers arrive for the final available execution, the system needs an authoritative answer about which one received it.

Proposed service · illustrative terms

Sell the analysis. Keep the source records.

20 executions · 30 days · one approved analysis · one specified dataset revision

Data holder’s controlled environment

1 · Admit the request

Check the permission

The analyst submits a request. Check the agreement, dataset revision, allowance and expiry.

Record which request receives an allowance reservation.
2 · Run the analysis

Bring the work to the data

The approved analysis runs beside the source records. The proposed workflow keeps those records here.

Link the execution to its request and reservation.
3 · Review the output

Apply the release rule

Check the report against the output policy, including what repeated queries could reveal.

Permission to run is separate from permission to release.
What the analyst receives: the permitted report

The proposed service links the permission, execution, allowance outcome and released result in one traceable record.

If a request retries or a worker restarts

Reconcile the existing reservation and execution before deciding whether to rerun, release a result or charge. The pilot must define and test that recovery policy.

Conceptual workflow, not a deployed service or execution trace. 8DB’s entitlement and consumption components provide foundations; worker recovery and output-disclosure controls require integration and evaluation. A released report can contain sensitive information, and later revocation cannot recall it.

When the transaction is interrupted

That sounds like bookkeeping until something interrupts the transaction.

The buyer retries because a response was lost. A worker restarts after admission but before producing its report. The dataset owner changes the approved version. The allowance expires while a job is running. Each case needs a defined outcome, including what happens to a reservation, whether another execution is allowed and what the buyer is owed.

A database accepting a request does not, by itself, prove that an external worker executed it exactly once. The worker and transaction records must be connected through a recovery protocol. I would rather settle that question in a small experiment than discover it after someone has been charged twice.

Permission and privacy are different promises

There is an equally important distinction between authority and privacy.

A signature can authenticate an entitlement. It does not establish that the person issuing it had the right to license the underlying records. A purpose written into a policy does not prove that a buyer will respect that purpose after receiving an output. The application still needs trusted identities, verified authority and rules that its execution environment can actually enforce.

Keeping the raw files in place also does not make every answer safe to release. A report can reveal sensitive information. Several individually acceptable queries can reveal more in combination. The output policy may therefore need restrictions on the analysis itself, review of results, or a separately evaluated privacy mechanism.

That is why I see a collaboration opportunity here. OpenMined builds tools for governed computation across data silos. We could contribute the entitlement, consumption and connected evidence side of a specific workflow, while working with people who understand the execution and disclosure requirements. We should test the complete boundary together. We should not describe the database as supplying every privacy property the service needs.

Revocation deserves the same precision.

An owner could withdraw permission for subsequent controlled operations. The next request would encounter that changed state. But revocation cannot make a downloaded report disappear, undo what a person learned or guarantee that information is removed from a trained model.

It also should not silently erase a legitimate obligation. If work was commissioned under agreed terms, a later access change does not automatically cancel the payment owed for that work. Historical authorization, current permission and settlement need to remain distinguishable.

One data holder, one useful analysis

The first pilot I would propose is deliberately small: one data holder, one analyst, one analysis and public or synthetic records. We would agree on the output rule before running anything. Then we would test expiry, concurrent requests, changed permissions, duplicate submissions and recovery after interruption.

The measurements would include incorrect admissions, incorrect refusals, allowance reconciliation, recovery behavior and integration effort. Useful work must still get through. A service that protects its allowance by refusing everything has not solved the problem.

What interests me is the access this could make practical. A dataset that is too sensitive to distribute may still support a valuable, carefully bounded service. A smaller organization might buy one useful analysis without negotiating an unrestricted data transfer. The owner could retain a record of the permission, the execution and the result that was released.

The next test is whether one useful workflow can enforce those limits through interruption, retries and changing permission.

If you manage a valuable dataset or approve analytical workflows, I would like to hear one concrete requirement: what output rule would have to hold before you would let another organization run an analysis on your data? Send that requirement to ashley@8braid.com. It would give us something useful to build and test together.

Please use a hypothetical or non-confidential example. I will ask before attributing or quoting a private reply.

Part of AI With Authority. Start with Would You Give an AI $100 and Walk Away?.

Evidence scope Proposed application · built on implemented 8DB components. Explore the evidence.

Continue the technical conversation

Where could this help your work?

Bring a research question, a database workload or an application you want to build. Let’s connect the ideas in this article to an evaluation that matters to your team.

Discuss this work

Continue reading

The next layer of the argument.

Idea

AI With Authority

Would You Give an AI $100 and Walk Away?

An AI that can act for you needs more than a spending limit. It needs authority tied to current rules and evidence. That opens useful possibilities, and an uncomfortable question about who controls the controls.

9 min readThought experiment · implemented foundations, proposed integrationRead article ↗
Idea

AI With Authority

I Would Pay for a Good Counterexample

What if an AI earned its reward by passing an agreed check, including a check that proves your original idea wrong? A thought experiment about commissioning useful, inspectable research.

5 min readProposed application · built on implemented 8DB componentsRead article ↗
Idea

Trustworthy AI Data

Your AI Found the Right Passage. Can It Tell Whether to Trust It?

A relevant answer can come from an expired policy or hide a conflicting source. We propose testing the evidence a retrieval system hands to an agent, so teams can see what survives the journey from stored record to answer.

6 min readEvaluation proposal · Illustrative fixture · No system scores reportedRead article ↗