The Chain of Custody Problem in AI

The law has a centuries-old test for whether an object can be believed. Run your ML pipeline through it and count the broken links.

By Dr. Anandkumar Prakasam · The Six Dimensions series, No. 2 · (publishes 12 July 2026)

Picture the most boring object in any courtroom, a sealed evidence bag. A knife, a hard drive, a vial. It doesn’t matter. What matters is the paper trail stapled to its existence. Who collected it, at what time, who sealed it, who transported it, who signed for it at the lab, who returned it to the locker, every hand accounted for from the scene to the witness stand. That trail has a name, chain of custody, and it is the law’s oldest answer to its oldest evidentiary question. Why should I believe this object is what you say it is?

Note precisely how the doctrine works, because the mechanism is the lesson. Opposing counsel does not need to prove the evidence was tampered with. They only need to find a gap, such as an unaccounted interval, an undocumented transfer, a missing signature, and the possibility of tampering does the rest. The evidence isn’t necessarily excluded. Something subtler and more fatal happens. Its weight collapses. The jury is invited to wonder, and wondering is enough. In custody doctrine, the gap is the verdict.

Last week I opened this series by arguing that “provenance” must answer a six-dimensional family of questions about data. Where it came from, what it was for, what it became, what it still touches. Today comes the evidentiary half of that argument. When an AI system’s decision is challenged, whether a credit denial, a claim rejection, or a flagged transaction, that decision is, in the law’s eyes, an object produced by a process. And the inquiry will want what inquiries have always wanted, the object’s journey. Which data, through which training run, produced which model, deployed in which configuration, that generated this decision, on this day.

Now hold a production ML pipeline against the custody standard, with every link recorded, every transfer accounted, contemporaneously, by identifiable hands, and start counting gaps.

The broken links, in order

Data to training set. The chain’s first link, and usually its first casualty. What was the provenance of each training record, its source, its consent basis, its version at ingestion? In most pipelines, training corpora were assembled by scripts that filtered, deduplicated, and merged sources, scripts since modified, run on data since updated, producing a corpus that exists today only as its own output. Ask “which records, under which lawful basis, entered this training set?” and the honest answer at item-level is archaeology. The first hand that touched the evidence is unidentified.

Training run to model. Which exact data state, code version, and configuration produced the weights now in production? Reproducibility is the ML equivalent of the unbroken seal, and most organizations cannot re-create their own production model from records. Checkpoints were cleaned up to save storage. Seeds went unrecorded. The run that made the artifact your business depends on often cannot be reproduced internally, let alone demonstrated to a third party.

Fine-tunes, merges, and the transformation cascade. Base model, instruction tuning, preference optimization, adapter merges, distillations, each is a transfer of custody through a transformation, and each typically records less than the one before. Readers of this series’ opening will recognize the sixth dimension here in evidentiary costume. The industry’s habit of treating each transformation as a fresh start, new artifact, our artifact, is precisely a custody break, asserted as a feature.

The vendor handoff. Then the model crosses an organizational boundary, and the chain crosses with it, or rather, doesn’t. The deployer can account for nothing upstream of the API, and the provider discloses little of what was accepted on the deployer’s behalf. And the break is not one-time but continuous, because the silent model update means the system in production this quarter is not the system you validated last quarter, and no transfer record marks the swap. (This is the custody-shaped argument for the risk-inheritance disclosure I’ve proposed to regulators, a handover document for exactly this boundary. The freight metaphor I used there belongs to the same legal family, because a bill of lading is a chain-of-custody instrument.)

The ephemeral context. A modern decision depended on more than weights, such as retrieved documents, cached state, the prompt and policy in force, the tool outputs in context. Was what the model actually saw recorded at decision time? Almost never, and the context window is the most consequential and least preserved crime scene in computing. Reconstructing it later from logs and git history is precisely the kind of after-the-fact reassembly that custody doctrine exists to distrust. Courts credit records made at the time of the act, by the process performing it, because contemporaneity is the currency, not reconstructions assembled once the dispute began.

Six links, five habitual breaks. And remember the mechanism. Nobody has to prove your model was trained on unlawful data or that the update changed the decision. They only have to establish that you cannot account for it, and your own engineering practices will stipulate to that in the first deposition.

Why this stops being theoretical now

Three converging pressures are turning custody from doctrine into deadline.

The burden has shifted, and readers of my legal series know the argument. Accountability regimes make the processor the party who must demonstrate lawful operation, continuously. A custody chain is what demonstrability looks like for artifacts, and a broken chain, under a reversed burden, is not a neutral gap but a failed proof. Discovery is learning to ask pipeline questions. AI litigation is young, but the requests are maturing fast, covering training-data composition, model lineage, version-at-decision-time. The first generation of cases was about outputs. The next is about journeys. And the statutes are custody-shaped, whether the EU AI Act’s logging and technical-documentation duties or the erasure rights that this series’ fourth dimension makes so uncomfortable. You cannot prove deletion of what you cannot trace, and you cannot trace what you never chained. The law, in short, assumes objects have journeys. ML practice produces artifacts with vibes. That collision now has a docket schedule.

What continuity actually requires

The repair is architectural, and its requirements can be stated concept-level in five lines, each one an engineering program, none of them optional.

Identity before chains. Every artifact in the pipeline, whether dataset states, training runs, model versions, configurations, or decisions, needs a stable, versioned identity. You cannot chain what you cannot name, and most pipelines cannot name their own past.

Links that are checkable, not asserted. Each transformation recorded such that the input–output relationship can be verified later by someone else, made contemporaneously, by the process performing the act, not reconstructed afterward from memory and repository forensics.

Operator-independence somewhere in the chain. A custody record maintained solely by the party whose conduct is in question is not custody. It is a diary. Courts have never accepted “I logged my own innocence” from any other kind of custodian, and some element of the chain must be verifiable without trusting the operator. (The engineering for this, tamper-evident records with independent anchoring and attestation, exists and is maturing, and the point here is the requirement, which no tooling purchase satisfies unless the architecture is designed for it.)

Records at the boundaries. Organizational handoffs, whether provider to deployer or deployer to integrator, need transfer documents covering what was delivered, which version, what known residual risks crossed with it, and notification when the upstream artifact changes. The handoff is where chains die today. It is also the easiest link to fix, because it’s a document, not a platform.

Survival through transformation. The hardest requirement and the one this book series exists to name. The chain must cross the transformations, whether training, tuning, derivation, or inference, rather than ending politely at each one. A chain that stops where the data becomes a model has preserved custody of everything except the thing that decides.

The audit: walk one chain backward

The artifact this week is twenty minutes of honest archaeology. Pick one consequential decision your AI system made last quarter, a real denial, a real flag. Now walk backward and score each link as recorded / reconstructable / broken.

The decision → the exact configuration, prompt, and policy in force at that moment → the model version that ran → the training/tuning run that produced it → the data states that fed the run → the sources and lawful bases of that data.

Five links. Most organizations get two links back before the trail goes cold, usually at the model-version boundary, almost always by the training run. Wherever your trail dies, mark the spot. That is where opposing counsel will pitch their tent, because that is where your account of your own system ends and your hopes begin.

One closing turn, because the chain has a second act. Suppose you do the work, with every link recorded, every transfer documented, the journey complete. A final question remains, and it is the cross-examiner’s favorite. Who made these records, who held them, and who could have changed them? If every link in your beautiful chain was written by you, stored by you, and editable by you, what you have built is a very thorough diary, and the difference between a diary and evidence is the difference between this essay and the next one.

Your AI audit trail won’t survive cross-examination. Next in this series, why, and what would.

Dr. Anandkumar Prakasam works at the intersection of computer science and law. This is No. 2 in The Six Dimensions series, from the forthcoming book Provenance: How the Six Dimensions of Data Will Rewrite Privacy, Power, and Accountability. Previously, “Provenance Is Not a Watermark.”