Skip to content

Idea

I Would Pay for a Good Counterexample

What if an AI earned its reward by passing an agreed check, including a check that proves your original idea wrong? A thought experiment about commissioning useful, inspectable research.

Published
Updated
Reading time5 minutes
AI agentsScientific verificationResearch workflowsDigital Fabric

Suppose I ask an AI agent to investigate a mathematical identity that an engineering model relies on. There are two useful outcomes: establish the identity under the stated assumptions, or produce an example that satisfies those assumptions and breaks it.

I would be happy to pay for either.

A convincing paragraph would not be enough. I would want an artifact that another program or person could inspect: the derivation and its checker, or the precise counterexample and the calculation showing where the identity fails.

That is a different way to commission AI work. The reward follows an acceptance condition agreed before the investigation. It does not depend on whether the agent brings back the answer I hoped to hear.

At 8Braid, our recent scientific work has made me interested in this possibility. We have spent time separating what an artifact establishes from the larger conclusion someone might want to attach to it. That distinction matters when a checked algebraic statement depends on physical assumptions, or a reproducible numerical calculation still needs a convergence study.

I think those habits could become part of how research work is bought.

Consider three possible commissions. Reproduce a specified calculation using pinned inputs and return the required artifacts. Find a valid counterexample to a declared claim. Produce a certificate accepted by a particular exact checker.

Each has a relatively clear boundary. An open-ended request such as “discover the correct theory” does not. It would still need scientific judgment, and I would not pretend that attaching a payment rule makes that judgment automatic.

Illustrative commission · proposed integration

Pay for a checked answer, whichever way it goes.

Agree on the exact claim, assumptions, artifact, checker and reward before the investigation starts.

Outcome A · Establish it

A valid proof

The submitted derivation establishes the exact claim under the agreed assumptions and passes its acceptance check.

Eligible for the agreed reward.
Outcome B · Refute it

A valid counterexample

The example satisfies the assumptions and demonstrates that the exact claim fails. Its calculation is checked.

Also eligible for the agreed reward.
Outcome C · Still open

Inconclusive work

Finding neither a proof nor a counterexample leaves the question open.

Reward only if the contract separately defines payment for that work.
A tiny example of Outcome B

Claim: for every real x, x² = x. Submit x = 2. It is real, and 2² = 4 differs from 2. This refutes the claim. Submitting x = 1 would not: 1² = 1.

The reward follows the agreed acceptance rule.An invalid artifact does not qualify as a proof or refutation. Later changes to scientific support remain separate from the recorded acceptance and any dispute process.
Conceptual commission, not a live payment or an 8DB execution trace. Proof and refutation are alternative outcomes for a claim, not two simultaneously valid answers. The arithmetic example is illustrative; a research service still needs a suitable checker, submission identity, recovery and settlement integration.

Agree on what counts before the work begins

For a checkable commission, the job description should carry at least four things:

  • The exact input versions and assumptions.
  • The artifact the worker must submit.
  • The acceptance rule and checker version.
  • The reward, submission rules and procedure for disputed or inconclusive results.

Those details are the agreement. If a checker accepts one representation and rejects another, the worker needs to know that before starting. If a numerical result is acceptable only with a particular error bound, that requirement belongs in the job. If an inconclusive investigation has value, define what evidence of the investigation earns a reward.

Failure to find a counterexample is not a proof. A candidate counterexample outside the stated assumptions is not a valid refutation. Paying for both positive and negative results only works when those distinctions survive the payment process.

This is where I see a possible role for the underlying 8DB capabilities behind Digital Fabric.

8DB implements signed entitlements, agreement admission and authoritative consumption records, alongside its evidence structures. A proposed integration could connect the commissioned task to its input revisions, submitted artifact, checker result and reward allowance. The configured acceptance path would check that relationship before authorizing the next transaction.

The complete workflow is something we would have to build and qualify. Existing entitlement checks do not establish that an external payment happens exactly once, or that every worker used the inputs it claimed to use. Submission identity, execution evidence, concurrency, retries and recovery all need explicit treatment.

This builds on an active research direction. The TessPay preprint proposes evidence-gated settlement for agentic commerce. The RAILS preprint describes a clearing protocol organized around obligations and admissible evidence. The useful question for us is what improves when the work, governing records, current allowance and evidence dependencies live in a connected data environment.

Accepted yesterday, reassessed today

One improvement I would want to test is how the system handles a result that was accepted yesterday but deserves a different assessment today.

Perhaps a checker defect is discovered. Perhaps an input calibration is withdrawn. Perhaps a premise behind the commissioned calculation was never established. The original submission, checker version and acceptance decision should remain available. Future uses of the result should also be able to see the changed support.

That does not mean automatically taking back a payment. The contract might say that a worker earns the reward by passing the agreed checker. A later discovery does not justify quietly substituting a new agreement. Historical acceptance, current scientific support and any dispute or refund process are different records with different meanings.

We also do not need to invent every part of the research packaging. Workflow Run RO-Crate already describes workflow execution provenance and associated inputs, outputs and code. A useful integration should exchange that information with existing tools, while making its additional acceptance and authorization semantics explicit.

Start with an answer we can inspect

My first experiment would involve a small set of deterministic tasks and simulated rewards. We would include a valid positive artifact, a valid counterexample to a deliberately false claim, a plausible but invalid submission, a duplicate request and an interruption during acceptance. Independent reviewers would agree on the expected outcomes before the system ran.

We would measure incorrect payments, incorrect refusals, allowance conservation and whether a reviewer could reconstruct each decision. Then we would change one evidence dependency and check that the historical record remained intact while the current assessment changed as specified.

A successful experiment would give us a small, useful mechanism for commissioning work. Its result would be a tested acceptance workflow; scientific validity would still depend on each task's assumptions, evidence and checker.

If you manage research workflows, computational services or reproducibility work, send me one task you already check and what a result must do to pass. A sentence or a small example is enough. It would help us choose a useful first commission. Send it to ashley@8braid.com.

Please use a hypothetical or non-confidential example. I will ask before attributing or quoting a private reply.

Part of AI With Authority. Start with Would You Give an AI $100 and Walk Away?.

Evidence scope Proposed application · built on implemented 8DB components. Explore the evidence.

Continue the technical conversation

Where could this help your work?

Bring a research question, a database workload or an application you want to build. Let’s connect the ideas in this article to an evaluation that matters to your team.

Discuss this work

Continue reading

The next layer of the argument.

Idea

AI With Authority

The Dataset Doesn't Have to Change Hands

A data owner could sell twenty executions of an approved analysis without distributing the underlying dataset. The useful product is a permitted use, with clear limits on what leaves.

5 min readProposed application · built on implemented 8DB componentsRead article ↗
Idea

AI With Authority

Would You Give an AI $100 and Walk Away?

An AI that can act for you needs more than a spending limit. It needs authority tied to current rules and evidence. That opens useful possibilities, and an uncomfortable question about who controls the controls.

9 min readThought experiment · implemented foundations, proposed integrationRead article ↗