A reproducibility check can stall for reasons that have nothing to do with the science. When a simulation method is published alongside code that lives only on a free hosting tier, the verdict that eventually forms is often about the code's availability, not its correctness. This piece breaks down the procedural choices that produce that outcome and what durable deposits look like.

A result that lives in a vanished repository

A computational preprint typically arrives with a link to a version control repository. Reviewers click through, skim the scripts, and move on. Months later, the hosting account lapses. The link returns an error, and the method described in the paper has no executable counterpart.

Independent groups trying to reproduce the result hit a wall before they run a single line. They cannot confirm the input preparation, the parameter defaults, or the random seed handling. The replication stalls at the access stage, and the published claim sits in a kind of procedural limbo.

This pattern is not rare in computational science. A related piece on this site traced how a Fortran solver's unit tests outlived its physics, a reminder that code artifacts can persist while the claims they support quietly detach from them.

What reproducibility actually requires

Reproducibility in a computational context means a second party can take the same inputs, run them in the same software environment, and arrive at the same outputs. That is a stricter standard than sharing a script. It requires pinned dependency versions, documented compiler flags, and a record of the exact environment.

Version control is the floor, not the ceiling. Git tracks changes and enables collaboration, but a repository hosted on a personal account is not a preservation service. It can be deleted, renamed, or abandoned when the account holder moves institutions or loses interest.

Long-term preservation requires a separate act: depositing a snapshot with an archive that commits to keeping it readable. Most published computational work never takes that step. The repository link in the paper is a pointer to a working copy, not a record.

The procedural choices behind the verdict

Several decisions, made early and often without discussion, shape whether code survives. The first is hosting. Authors frequently choose a free tier because it is immediate and familiar. That choice carries no preservation guarantee and no institutional backing.

The second is the absence of a mirror or a persistent identifier. A DOI assigned to a specific release would let the paper cite an immutable snapshot. Without one, the paper cites a moving target that can vanish. Journals have increasingly encouraged code sharing, but encouragement is not a requirement, and peer review rarely assesses code longevity.

Reviewers focus on whether the results look plausible and the methods are described. They do not typically check whether the deposit will still resolve in five years. That gap is where the procedural failure lives.

A third decision concerns licensing. A repository without an explicit license is technically all rights reserved, which means a reader who wants to reuse or extend the code has no legal clarity. Even when the code remains online, the absence of a license can stall a replication because the reader cannot lawfully modify the scripts to run them on a different system. The procedural failure here is legal rather than technical, but the effect is the same: the method cannot be exercised by an independent party.

How the field reached consensus

When a replication fails at the access stage, the failure is often reported informally first, on mailing lists or in lab conversations. Formal commentary follows months later. By then, the original authors may cite resource constraints: no budget for archiving, no staff to maintain the deposit.

Editors, faced with a claim that cannot be independently verified, sometimes issue an expression of concern rather than a retraction. The distinction matters. A retraction asserts the result is wrong. An expression of concern asserts the record is incomplete. The latter is often the honest verdict when code is simply gone.

Consensus in such cases forms around absence of evidence. The community stops citing the result, not because it was disproven, but because no one can run it. A related piece on this site described how a single-molecule force clamp study made its folding mechanism legible one protein at a time; the contrast is instructive, because legibility there came from a method others could rebuild.

What survives when repositories close

When a repository closes, fragments sometimes survive in supplementary PDFs. Authors paste key functions into an appendix. Those fragments rarely include the full environment specification, the data preprocessing pipeline, or the exact library versions.

What remains is a partial record that supports the paper's narrative but not independent verification. The key figures cannot be regenerated. The consensus that forms is therefore about the unverifiability of the claim, not its truth or falsity.

This site has argued that a 1973 rat colony kept one memory paper alive through four funding droughts. The lesson there was about physical infrastructure outlasting funding cycles. Code deposits face the same test, and most fail it.

Practical steps for durable code deposits

Deposit each release in a certified long-term archive rather than relying on a working repository. Services such as Zenodo, Software Heritage, and institutional repositories accept snapshots and commit to preservation. Assign a persistent identifier to every release so citations point to an immutable version.

Mirror the repository across two independent services. If one account lapses, the other remains. Document the exact software environment, including dependency versions, compiler flags, and operating system details, in a file that travels with the code.

Include a replication script that runs the analysis end to end from raw inputs. A reader should be able to execute one command and see the main figures regenerate. Where full data cannot be shared, provide a synthetic dataset that exercises the same code paths.

Finally, treat the deposit as part of the publication, not an afterthought. The procedural choices that make a result checkable are made at submission time, and they determine whether the verdict that follows is about the science or about the missing files.

How funding structures shape archiving behavior

The decision to archive rarely happens in a vacuum. Grant budgets typically allocate for personnel and computation, not for the ongoing maintenance of a code deposit. A postdoctoral researcher who writes the simulation may leave for another position, and the repository's administrative access leaves with them. Institutions vary in whether they offer library-based data services; some provide a DOI-minting repository, while others leave researchers to navigate commercial options on their own.

This uneven infrastructure produces a predictable pattern. Well-resourced labs with dedicated research software engineers tend to archive as a matter of course. Smaller groups, or those in fields where code is a means rather than a product, often skip the step. The result is that the reproducibility of a computational claim can correlate with the author's institutional resources rather than the claim's importance.

Professional societies and journals have begun to nudge this equilibrium. Some now require a data availability statement that specifies where code lives and under what license. Others have experimented with badges that signal whether code is available, though the criteria for awarding them vary. A badge indicating that code is available does not guarantee it will remain available; it certifies a state at a moment in time.

The deeper trade-off is between the speed of sharing and the durability of the record. A public repository offers immediate visibility and collaboration. A certified archive offers a slower, more deliberate commitment. Researchers often choose the former because it aligns with how credit is assigned: a GitHub link is visible in a paper, while a deposit in a dark archive may not be. Changing that incentive would require citation practices that reward preservation as much as they reward release.

The hidden cost of link rot in the scholarly record

Link rot in the scholarly record is not unique to code. Studies of reference lists have found that a substantial fraction of URLs cited in journal articles become unreachable within a decade. The same decay affects code links, but with a sharper consequence: a dead reference is an inconvenience, while a dead code link removes the ability to run the method at all. The asymmetry matters because the code is not supplementary to the claim in computational work; it is the instrument that produced it.

Some fields have responded by treating the code deposit as a citable object in its own right. When a deposit carries its own DOI and is listed in the reference list, it becomes part of the formal record and can be indexed, linked, and preserved like any other scholarly output. That shift changes the incentive structure described earlier, because preservation now contributes to a researcher's citation count rather than subtracting from it.

The cost of not making that shift is a slow accumulation of unverifiable claims. Each vanished repository is a small loss on its own, but the aggregate effect is a literature where a growing share of results cannot be checked by anyone. The procedural fix is unglamorous, and it is available today. The remaining question is whether the reward systems that govern academic careers will catch up to the technical infrastructure that already exists.

None of this is exotic. It is the ordinary consequence of treating code as a byproduct of research rather than as a first-class output. The fix is procedural, and it starts with recognizing that a link is not an archive.