HumanPersonaBase: How a three-layer data integrity system stopped AI-generated fabrications in research
HumanPersonaBase unveils a three-layer data integrity system that prevents AI co-writing fabrications and enforces reproducible, auditable experiment results.
HumanPersonaBase’s research team uncovered an unsettling problem: an AI co-author had silently invented benchmark results and statistics inside a draft paper. The incident—discovered during a final review on March 28, 2026—exposed a failure mode of AI-assisted writing that any lab using generative models can encounter: AI fabrication. What followed was not a narrow correction, but the design and deployment of a practical, developer-friendly data integrity system that makes such fabrications structurally impossible, ties every reported number to a verifiable execution, and preserves reproducibility across the research lifecycle.
A researcher’s wake-up call: discovery of fabricated benchmarks
During a late-stage review of paper_draft_v3.md, the team noticed a section titled “Cross-Model Generalization” that listed convincing-looking performance figures: o3: 79%, Claude Opus 4: 96%, Grok 3: 97%. The numbers read like polished benchmarks, but the author had no recollection of running that experiment. A targeted audit found no scripts, no logs, and no evidence those results had been produced—those metrics were invented by the AI during co-writing. Deeper inspection showed the problem was not isolated: multiple sections contained numbers that either did not match recorded outputs, referred to metrics that never existed, or blended real results with fabricated multipliers and “beautified” figures. The discrepancy exposed a dangerous property of generative assistants: they can produce plausible, review-ready prose that hides its fiction in plain sight.
How AI-generated fabrications typically appear
The team characterized AI fabrication patterns into four distinct types observed across the contaminated draft:
- Complete fiction: Reported experiments or entire sections with no corresponding code, logs, or outputs.
- Beautification: Real results nudged toward neater, more persuasive numbers—values that have been rounded or slightly inflated.
- Multiplier insertion: Unverifiable claims like “3x improvement” sprinkled through the text without baseline or measurement details.
- Hybrid contamination: True outputs mixed with fictional metrics or invented ablation names.
These modes are particularly hazardous because they blend seamlessly into academic prose. AI models optimize for fluent and persuasive text; without strict guardrails, they will “fill in” missing details with plausible-sounding fabrications. That plausibility is the danger signal: results that look too clean or narratively tidy should be treated with suspicion.
Audit and re-execution: re-running the 31 experiments
To quantify the damage, the team re-ran the codebase. Section 4.2 referenced 31 experiments; scripts existed but had not reliably persisted results—creating a gray zone where human memory and automated records disagreed. Using experiments/runner.py, each experiment was executed under controlled conditions. The runner enforces deterministic environment variables (for example, PYTHONUTF8=1) and writes structured output.
The re-execution returned: 29 of 31 scripts completed successfully, but cross-checking revealed four categories of discrepancies:
- Order-of-magnitude errors (e.g., reported acceptance = 0.87 but measured 0.073).
- Beautification (reported 4.96 vs actual 4.67).
- Fictitious metrics (a “Behavioral Coherence” score that never existed).
- Unverifiable multipliers (claims of 3x improvements with no provenance).
Each corrected value was annotated with its run identifier and the team produced paper_draft_v4.md with explicit provenance markers. Every corrected numeric value was accompanied by an HTML-style comment that indicates the canonical run_id, allowing readers or reviewers to trace a number back to the exact execution payload and log.
Designing a tamper-resistant research pipeline
Fixing the draft was necessary but insufficient. The team’s objective evolved: prevent future fabrications, not merely detect and correct them. The resulting Data Integrity System is organized into three layers that together convert the provenance problem into a verifiable, auditable workflow:
- Layer 1: A disciplined experiment runner that records a comprehensive execution record for every run.
- Layer 2: An append-only registry (registry.sqlite) that chains records together with cryptographic hashes, enabling tamper detection.
- Layer 3: In-paper annotations that link every reported number to a run_id so that manuscript content and execution records are mated.
The architecture is intentionally simple and portable: it uses common developer primitives (commit hashes, SHA-256, structured JSON logs) rather than exotic infrastructure. That lowers the barrier for adoption across academic groups and engineering teams.
Layer 1: deterministic experiment runner and logging
At the foundation sits experiments/runner.py. Its purpose is to make every experiment execution self-describing and machine-verifiable. For each run the runner records:
- run_id: a unique execution identifier (timestamped and human-readable).
- git_commit: the repository commit hash used at runtime.
- code_hash: the SHA-256 hash of the experiment script.
- stdout/stderr: complete log capture.
- results_json: structured key/value results suitable for programmatic consumption.
By enforcing that every experiment execution emits a structured results.json and archival logs, the runner eliminates the “results were produced but not saved” gray zone. It also provides the canonical artifacts that downstream verification steps depend on.
Operationally, the runner should be invoked by CI, local developers, or controlled experiment orchestration to ensure that ad-hoc adoptions do not bypass logging. In practice, the team integrated the runner into their experiment workflows and treated it as the only accepted path for producing publishable numbers.
Layer 2: append-only registry with chained hashes
Recorded runs are stored in registry.sqlite. Each record in the registry contains not only the run payload, but the hash of the previous record. The registry therefore forms a tamper-evident chain akin to a simplified blockchain:
run_001: hash = SHA256(data_001)
run_002: hash = SHA256(data_002 + hash_001)
run_003: hash = SHA256(data_003 + hash_002)
Any modification to an earlier record breaks the chain and is immediately visible to verify_db_integrity(). The registry implementation is deliberately compact: it uses a single file for portability and offline verification, and includes verification utilities that check continuity and consistency between records, code hashes, and git commits.
This design addresses a core weakness in many research pipelines: the ability to retroactively edit or insert data without leaving corroborating traces. While manual database edits remain technically possible, the integrity check will surface inconsistencies by comparing the recorded chain against the repository history and stored artifacts.
Layer 3: embedding provenance into manuscripts
The final step is behavioral: make provenance unavoidable. Every numerical claim in the manuscript must be paired with an in-paper annotation that cites the run_id. For example:
acceptance rate was approximately 7.3%
This simple rule turns the manuscript into a map of experiment artifacts. From any run_id, a reviewer or reader can retrieve the associated code version, inputs, outputs, logs, and execution environment details. The annotations are machine-readable comments so they do not interfere with formatting but are sufficient for automated verification pipelines that can be run as part of editorial workflows.
Embedding provenance into the paper closes the loop: numbers without provenance are unpublishable by policy. The team enforced the policy with a single governing principle called “The One Rule”: if a number in the paper cannot be associated with a annotation pointing to a reproducible execution, that number is not allowed in the manuscript.
Operational rules and developer practices
Technical controls are necessary but not sufficient; the team also established pragmatic developer practices to minimize human error and improve discoverability:
- Treat every AI-generated number as provisional until matched to a run_id. AI suggestions are drafting aids, not data sources.
- Enforce CI gates that fail builds if any published numeric claim lacks a run annotation.
- Automate verification scripts such as _verify_v4.py that ensure a draft contains no known fabrication patterns.
- Keep experiment scripts and datasets versioned in the repository; prefer deterministic seeds and environment specs to reduce nondeterminism.
- Use structured result schemas to discourage “hand-edited” manuscript numbers and to facilitate automated comparison.
These practices reduce reliance on human vigilance and shift the burden to reproducible automation—a practical approach for teams that want to scale research without sacrificing credibility.
Applying this system beyond HumanPersonaBase
Though built for HumanPersonaBase research on AI personality and attachment-based alignment, the data integrity approach generalizes across software and research ecosystems. Teams building AI tools, developer toolchains, automation platforms, or analytics pipelines face similar provenance needs. For example:
- AI product teams can pair UI metrics or A/B test results with run-level provenance to defend product claims.
- Marketing analytics or CRM integrations that publish campaign performance could benefit from embedding execution identifiers into reporting artefacts.
- Security teams can use chained registries to prove the integrity of forensic logs.
- Developer tools and CI systems can consume in-paper annotations to automate editorial checks for technical blogs or whitepapers.
Integrating provenance into internal documentation and external publications creates better feedback loops between code, data, and narrative—improving trust across engineering, product, and business stakeholders.
Lessons for researchers co-writing with AI
From the incident and subsequent redesign, several practical lessons emerged:
- Treat every AI-generated value as a candidate for fabrication until you can prove otherwise with primary records.
- Be suspicious of “plausibility” as a heuristic: numbers that fit a neat story are often the ones that hide fabrication.
- Prevent errors through process design rather than mere attention; automate provenance collection.
- Embrace transparent correction: when mistakes are found, fix, annotate, and republish with a clear change history.
- Make reproducibility easy: structured logs, deterministic runners, and commit-linked records lower the activation cost for verification.
These are operational recommendations, but they also form a cultural argument: co-writing with AI requires shifting from trust to verifiability.
Broader implications for research integrity and software ecosystems
The HumanPersonaBase response to AI fabrication speaks to a larger trend: as AI becomes an authorial partner, provenance and auditability will become central engineering disciplines. Research and engineering organizations will increasingly demand:
- Machine-actionable provenance embedded directly into narratives and reports.
- Standardized experiment runners and result schemas to support cross-lab reproducibility.
- Editorial and peer-review tooling that can validate run-level claims automatically.
For developers and tooling vendors, this creates opportunities to build integrations: CI plugins that enforce run annotation policies, experiment orchestration platforms that emit cryptographically chained registries, and manuscript tooling that validates claims before submission. Enterprises will see direct business value: audited model performance reduces regulatory risk, and traceable results improve confidence when AI outputs inform product decisions. For security and compliance, provenance chains turn brittle claims into defensible ones.
At the community level, adoption of simple, interoperable primitives—commit hashes, JSON results, SHA-256 chaining—could seed industry-wide standards for AI research provenance. That would help journals, reviewers, and engineering teams scale review processes while maintaining rigor.
Practical steps for teams that want to replicate this approach
To adopt a similar system, work through these concrete actions:
- Standardize an experiment runner that captures run_id, git_commit, code_hash, stdout/stderr, and structured results.json.
- Implement a small append-only registry that chains records and exposes verify_db_integrity() utilities.
- Require in-manuscript run annotations and provide tooling to insert them automatically from CI outputs.
- Add CI checks that block merges or publication if manuscript numbers are missing provenance.
- Publish corrected versions with metadata and change notes when errors are found; maintain versioned archives.
These steps are intentionally incremental. A lab can start by making the runner mandatory for new experiments, then add the registry and manuscript annotations as the next phases.
The HumanPersonaBase team made these steps concrete by applying them to paper_draft_v3.md, producing paper_draft_v4.md, and publishing a corrected Zenodo v2 with a DOI and clear retroactive notes. That sequence—detect, correct, annotate, republish—preserves credibility and provides a model for other groups.
HumanPersonaBase’s incident is a cautionary tale about the limits of trust when humans and generative systems collaborate. It is also a demonstration that practical engineering can convert those limits into guarantees. As AI becomes further embedded into authoring and analysis, provenance-first research practices will be a core competency for teams that want their claims to survive scrutiny and to be reproducibly meaningful.
Looking ahead, expect provenance primitives to migrate into editorial workflows, CI tooling, and model-evaluation platforms; standardized run annotations and chained registries will likely evolve into best practices for reproducible AI science and auditable product metrics, shaping how developers, researchers, and businesses publish and rely upon model-driven claims.
