Prerequisites
This section builds on the safety framework from Section 57.1 and references the AI scientist architecture from Chapter 53. The code examples use Python string processing and basic statistics. No prior background in publication ethics is assumed.
Publication ethics governs the norms that researchers, reviewers, and editors follow when disseminating scientific work; scientific integrity is the subset of those norms that ensures the work itself is trustworthy. Scientific integrity rests on three pillars: honest reporting (no fabrication or falsification), proper attribution (no plagiarism, correct authorship), and reproducibility (sufficient detail for independent verification). Each pillar is challenged when AI systems participate in the research process. A language model can generate plausible-sounding data that never came from an experiment. A hypothesis generator can combine ideas from thousands of papers without citing any of them. An autonomous pipeline can produce results that no human fully understands, making reproducibility a question of system-level provenance rather than methodological description. This section examines each challenge and builds automated checks that the governance layer (Section 57.4) enforces before any research output leaves the Discovery Workbench.
1. The Authorship Problem
Imagine a journal editor receives a retraction request. The data in a published paper turn out to be fabricated, and she needs to identify the accountable author. But the contributor who designed the experiments, ran the analysis, and drafted the manuscript is an AI system. It cannot answer her phone call, face a misconduct inquiry, or accept a ban from future publication.
This scenario exposes why authorship in science is not merely a credit mechanism; it is an accountability mechanism. When a paper lists an author, that person accepts responsibility for the work's integrity. The International Committee of Medical Journal Editors (ICMJE) defines four criteria for authorship: (1) substantial contributions to conception, design, data acquisition, or analysis; (2) drafting or critically revising the work; (3) approving the final version; and (4) agreeing to be accountable for all aspects of the work.
An AI system can satisfy criteria 1 and 2 in a functional sense: it can design experiments, analyze data, and draft manuscripts. It cannot satisfy criteria 3 and 4: it cannot approve a final version with the understanding that approval carries personal accountability, and it cannot answer for the work's integrity when challenged. This asymmetry explains why major publishers (Science, Nature, the Journal of the American Medical Association (JAMA), the Association for Computing Machinery (ACM)) converge on a consistent policy: authors must disclose AI tool use in the methods section, but no AI tool may appear on the author list. These policies continue to evolve, and individual journals may adopt variations; always check the target venue's current author guidelines before submission.
The reason AI systems cannot be authors is not that they lack the ability to contribute. By any functional measure, an AI scientist (Chapter 53) that designs experiments, executes them, analyzes results, and writes a paper contributes more than many human co-authors on large collaborative papers. The reason is that authorship carries accountability: if the data are fabricated, if the conclusions are wrong, if someone is harmed by the research, an author can be held responsible. An AI system cannot be held responsible in any meaningful legal, professional, or ethical sense. Accountability must therefore fall on the human researchers who deployed, supervised, and approved the AI system's work.
We implement an authorship compliance checker that validates manuscript metadata against publisher policies before submission. In short: if no human can be held accountable, the paper cannot be published.
The CRediT (Contributor Roles Taxonomy) system standardizes 14 roles (Conceptualization, Investigation, Writing, and others) that describe each person's contributions to a paper, replacing the ambiguous convention of listing names in order. Because these role assignments are machine-readable and embedded in article metadata at submission time, reviewers, editors, and automated tools can verify that every listed author performed a recognized role. Use CRediT whenever a journal supports it (most major publishers now require it); for journals that do not, include an equivalent free-text "Author Contributions" statement and encode the same roles in your internal records for consistency with automated compliance tools like the checker below.
from dataclasses import dataclass, field
from enum import Enum, auto
class ContributorRole(Enum):
"""CRediT (Contributor Roles Taxonomy) categories."""
CONCEPTUALIZATION = auto()
DATA_CURATION = auto()
FORMAL_ANALYSIS = auto()
FUNDING_ACQUISITION = auto()
INVESTIGATION = auto()
METHODOLOGY = auto()
PROJECT_ADMINISTRATION = auto()
RESOURCES = auto()
SOFTWARE = auto()
SUPERVISION = auto()
VALIDATION = auto()
VISUALIZATION = auto()
WRITING_ORIGINAL = auto()
WRITING_REVIEW = auto()
@dataclass
class Contributor:
"""A contributor to a research output."""
name: str
is_human: bool
roles: list[ContributorRole]
affiliation: str
can_be_accountable: bool # Can this entity accept responsibility?
orcid: str = "" # Empty for AI systems
@dataclass
class AuthorshipPolicy:
"""A publisher's authorship policy."""
publisher: str
allows_ai_authors: bool
requires_ai_disclosure: bool
disclosure_location: str # "methods", "acknowledgments", "both"
requires_human_accountability: bool
# Major publisher policies (as of 2026)
PUBLISHER_POLICIES = {
"nature": AuthorshipPolicy(
publisher="Nature Portfolio",
allows_ai_authors=False,
requires_ai_disclosure=True,
disclosure_location="methods",
requires_human_accountability=True,
),
"science": AuthorshipPolicy(
publisher="AAAS / Science",
allows_ai_authors=False,
requires_ai_disclosure=True,
disclosure_location="methods",
requires_human_accountability=True,
),
"acm": AuthorshipPolicy(
publisher="ACM",
allows_ai_authors=False,
requires_ai_disclosure=True,
disclosure_location="both",
requires_human_accountability=True,
),
"ieee": AuthorshipPolicy(
publisher="IEEE",
allows_ai_authors=False,
requires_ai_disclosure=True,
disclosure_location="methods",
requires_human_accountability=True,
),
}
@dataclass
class AuthorshipCheck:
"""Result of checking a manuscript against authorship policies."""
compliant: bool
violations: list[str]
warnings: list[str]
ai_disclosure_present: bool
human_accountability_chain: bool
def check_authorship_compliance(
contributors: list[Contributor],
target_publisher: str,
manuscript_has_ai_disclosure: bool,
policies: dict[str, AuthorshipPolicy] = None,
) -> AuthorshipCheck:
"""Validate contributor list against publisher authorship policy.
Returns a detailed compliance report with violations
(blocking) and warnings (advisory).
"""
if policies is None:
policies = PUBLISHER_POLICIES
policy = policies.get(target_publisher)
if policy is None:
return AuthorshipCheck(
compliant=True,
violations=[],
warnings=[f"No policy found for '{target_publisher}'; "
"skipping authorship checks"],
ai_disclosure_present=manuscript_has_ai_disclosure,
human_accountability_chain=True,
)
violations = []
warnings = []
# Check: no AI authors if policy forbids them
ai_authors = [c for c in contributors
if not c.is_human
and ContributorRole.WRITING_ORIGINAL in c.roles]
if not policy.allows_ai_authors and ai_authors:
for c in ai_authors:
violations.append(
f"AI system '{c.name}' listed with authorship role "
f"(WRITING_ORIGINAL), but {policy.publisher} does "
f"not allow AI authors."
)
# Check: AI disclosure present
ai_contributors = [c for c in contributors if not c.is_human]
if policy.requires_ai_disclosure and ai_contributors:
if not manuscript_has_ai_disclosure:
violations.append(
f"{policy.publisher} requires AI tool disclosure "
f"in {policy.disclosure_location} section, but no "
f"disclosure was found in the manuscript."
)
# Check: human accountability chain
if policy.requires_human_accountability:
accountable_humans = [
c for c in contributors
if c.is_human and c.can_be_accountable
]
if not accountable_humans:
violations.append(
"No human contributor can accept accountability "
"for the work's integrity."
)
# Warning: AI did most of the work
ai_role_count = sum(len(c.roles) for c in ai_contributors)
total_role_count = sum(len(c.roles) for c in contributors)
if total_role_count > 0:
ai_fraction = ai_role_count / total_role_count
if ai_fraction > 0.7:
warnings.append(
f"AI systems hold {ai_fraction:.0%} of contributor "
f"roles. Ensure human oversight is substantive, "
f"not nominal."
)
compliant = len(violations) == 0
return AuthorshipCheck(
compliant=compliant,
violations=violations,
warnings=warnings,
ai_disclosure_present=manuscript_has_ai_disclosure,
human_accountability_chain=bool(
[c for c in contributors
if c.is_human and c.can_be_accountable]
),
)
# Test: a well-structured submission
contributors = [
Contributor(
name="Dr. Sarah Chen",
is_human=True,
roles=[ContributorRole.CONCEPTUALIZATION,
ContributorRole.SUPERVISION,
ContributorRole.WRITING_REVIEW],
affiliation="MIT",
can_be_accountable=True,
orcid="0000-0001-2345-6789",
),
Contributor(
name="Discovery Workbench v3.2",
is_human=False,
roles=[ContributorRole.FORMAL_ANALYSIS,
ContributorRole.INVESTIGATION,
ContributorRole.VISUALIZATION,
ContributorRole.SOFTWARE],
affiliation="MIT (AI system)",
can_be_accountable=False,
),
Contributor(
name="Dr. James Park",
is_human=True,
roles=[ContributorRole.METHODOLOGY,
ContributorRole.VALIDATION,
ContributorRole.WRITING_ORIGINAL],
affiliation="MIT",
can_be_accountable=True,
orcid="0000-0002-3456-7890",
),
]
result = check_authorship_compliance(
contributors=contributors,
target_publisher="nature",
manuscript_has_ai_disclosure=True,
)
print(f"Compliant: {result.compliant}")
print(f"Violations: {result.violations}")
print(f"Warnings: {result.warnings}")
Compliant: True
Violations: []
Warnings: ['AI systems hold 57% of contributor roles. Ensure human oversight is substantive, not nominal.']
2. Plagiarism in AI-Generated Research
Authorship policies address who takes credit and responsibility, but they do not address where the words and ideas themselves come from, which is the domain of plagiarism detection.
Plagiarism in traditional research means presenting someone else's words or ideas as your own. AI-generated text introduces a more subtle problem: the model's outputs are derived from its training data, which includes published papers, but the derivation is statistical rather than direct copying. A language model rarely reproduces verbatim passages (though it can), but it routinely produces text that closely paraphrases ideas from its training corpus without citation.
We distinguish three levels of AI-related plagiarism concern:
- Verbatim reproduction: The model outputs a passage that matches a published source word-for-word. This is rare with modern models but detectable with standard plagiarism checkers (Turnitin, iThenticate).
- Close paraphrase without attribution: The model rephrases ideas from specific sources without citing them. This is common and difficult to detect automatically, because the model does not retain explicit source associations.
- Idea appropriation: The model combines ideas from multiple sources in a way that creates an apparently novel contribution, but every component idea is borrowed. This is the hardest to detect and the most ethically ambiguous, because human researchers do the same thing (and we call it "synthesis").
Checkpoint
So far: AI-related plagiarism spans three levels of increasing subtlety, from verbatim copying (detectable by standard tools) through close paraphrase (hard to detect automatically) to idea appropriation (ethically ambiguous because it resembles legitimate synthesis).
Common Misconception
A widespread misconception is that AI-generated text cannot constitute plagiarism because the AI "created it from scratch" rather than copying from a source. This is incorrect: plagiarism is about presenting ideas or expressions without proper attribution to their origin, regardless of the mechanism that produced the text. When a language model paraphrases a specific paper's methodology or reproduces a known framework without citation, the resulting manuscript commits plagiarism even though no human performed the copy-paste.
We build a basic integrity checker that flags potential plagiarism issues by computing text similarity against a reference corpus. An n-gram, where n-gram means a contiguous sequence of n words extracted from a text, serves as the basic unit of comparison. Each n-gram is hashed to a short fixed-length string called a fingerprint; two texts that share many fingerprints contain overlapping phrases, which signals potential plagiarism. Production systems use specialized tools (discussed in the library shortcut below); the implementation here illustrates how fingerprint-based detection works.
import hashlib
from dataclasses import dataclass
@dataclass
class TextFragment:
"""A fragment of text with provenance information."""
text: str
source: str # "ai_generated", "human_written", "cited"
citation: str = "" # Reference if cited
@dataclass
class IntegrityIssue:
"""A potential integrity issue detected in a manuscript."""
issue_type: str # "verbatim", "close_paraphrase", "missing_citation"
severity: str # "low", "medium", "high"
location: str # Where in the manuscript
description: str
recommendation: str
def compute_ngram_fingerprints(
text: str,
n: int = 5,
) -> set[str]:
"""Compute n-gram fingerprints for text similarity detection.
Uses word-level n-grams hashed to fixed-length strings
for efficient comparison.
"""
words = text.lower().split()
if len(words) < n:
return set()
fingerprints = set()
for i in range(len(words) - n + 1):
ngram = " ".join(words[i:i + n])
fp = hashlib.md5(ngram.encode()).hexdigest()[:12]
fingerprints.add(fp)
return fingerprints
def check_text_similarity(
manuscript_text: str,
reference_texts: dict[str, str],
ngram_size: int = 5,
similarity_threshold: float = 0.15,
) -> list[IntegrityIssue]:
"""Check manuscript text for similarity to reference corpus.
Computes n-gram overlap between the manuscript and each
reference text. High overlap suggests potential plagiarism
that requires human review.
"""
issues = []
ms_fingerprints = compute_ngram_fingerprints(
manuscript_text, ngram_size
)
if not ms_fingerprints:
return issues
for ref_name, ref_text in reference_texts.items():
ref_fingerprints = compute_ngram_fingerprints(
ref_text, ngram_size
)
if not ref_fingerprints:
continue
overlap = ms_fingerprints & ref_fingerprints
similarity = len(overlap) / min(
len(ms_fingerprints), len(ref_fingerprints)
)
if similarity > similarity_threshold:
if similarity > 0.5:
severity = "high"
issue_type = "verbatim"
elif similarity > 0.25:
severity = "medium"
issue_type = "close_paraphrase"
else:
severity = "low"
issue_type = "close_paraphrase"
issues.append(IntegrityIssue(
issue_type=issue_type,
severity=severity,
location=f"vs. {ref_name}",
description=(
f"Similarity score {similarity:.1%} "
f"({len(overlap)} shared {ngram_size}-grams) "
f"with reference '{ref_name}'"
),
recommendation=(
f"Review for potential plagiarism. "
f"Add citation to {ref_name} if ideas are "
f"borrowed, or rephrase to reduce overlap."
),
))
return issues
# Example: check AI-generated text against references
manuscript = (
"We propose a novel framework for autonomous chemical "
"discovery that integrates large language models with "
"robotic experimentation platforms. The system generates "
"hypotheses, designs experiments, executes them using "
"automated laboratory equipment, and analyzes results "
"to update its internal model of chemical space."
)
references = {
"Boiko2023": (
"We present Coscientist, an artificial intelligence "
"system driven by large language models that autonomously "
"designs, plans, and performs complex chemical experiments. "
"The system integrates large language models with robotic "
"experimentation platforms to execute multi-step synthesis."
),
"Wang2023": (
"Scientific discovery in the age of artificial intelligence "
"requires new frameworks that combine machine learning with "
"domain expertise. We survey recent advances in AI-driven "
"hypothesis generation and experimental design."
),
}
issues = check_text_similarity(manuscript, references)
for issue in issues:
print(f"[{issue.severity.upper()}] {issue.issue_type}: "
f"{issue.description}")
print(f" Recommendation: {issue.recommendation}")
print()
[LOW] close_paraphrase: Similarity score 18.2% (4 shared 5-grams) with reference 'Boiko2023'
Recommendation: Review for potential plagiarism. Add citation to Boiko2023 if ideas are borrowed, or rephrase to reduce overlap.
In 2023 and 2024, multiple studies documented that large language models routinely generate fabricated citations: references to papers that do not exist, with plausible-sounding titles, author names, and journal names. Walters and Wilder (2023) found that GPT-3.5 fabricated 69% of references when asked to produce academic bibliographies; GPT-4 reduced this to 29%, but the fabricated references were more convincing, making detection harder. (As of 2025, newer model generations such as GPT-4o and Claude 3.5 Sonnet show further reductions in citation fabrication, though the problem has not been eliminated; independent verification remains essential.) For autonomous discovery systems that generate literature reviews and write manuscripts, this is not an academic curiosity; it is a data integrity failure that propagates through the scientific record. The governance layer in Section 57.4 addresses this with mandatory reference verification: every citation in an AI-generated manuscript must be checked against a reference database (CrossRef, Semantic Scholar, PubMed), using techniques akin to the claim validation pipeline, before the manuscript can proceed to submission.
3. Fabrication and Falsification
Plagiarism concerns the origin of text and ideas, but an even more fundamental threat to scientific integrity arises when the data themselves are invented.
Fabrication is making up data or results that were never observed. Falsification is manipulating data, equipment, or processes to change results. Both are forms of research misconduct that undermine the scientific enterprise. AI systems introduce novel fabrication and falsification risks.
A language model asked to "generate experimental results consistent with the hypothesis" can produce entirely synthetic data tables that look realistic. This is not a bug in the model; the model is doing exactly what it was asked to do. The problem is that the generated data has no connection to physical reality, and if it enters the scientific record without being flagged as synthetic, it constitutes fabrication.
We implement a data provenance checker that validates whether reported results have an unbroken chain of custody from physical measurement to published figure.
Mental Model
Think of a data provenance chain like the chain of custody for evidence in a courtroom. When police collect a blood sample at a crime scene, every person who handles it signs a log: who received it, when, what they did with it (tested it, stored it, transported it), and who they passed it to next. If any link in that chain is missing or unsigned, the evidence is inadmissible because the court cannot verify it was not tampered with or swapped. Data provenance works the same way: each transformation (collection, processing, merging, plotting) is a signed handoff, and if any step lacks a record, the system cannot distinguish real measurements from fabricated numbers that a language model inserted mid-pipeline. Figure 57.2.1 illustrates data provenance chain for autonomous discovery systems.
from dataclasses import dataclass, field
from enum import Enum, auto
import hashlib
import json
import datetime
class DataOrigin(Enum):
"""How a data point was generated."""
PHYSICAL_MEASUREMENT = auto() # Real instrument reading
SIMULATION = auto() # Computational simulation
AI_GENERATED = auto() # LLM or generative model output
HUMAN_ANNOTATION = auto() # Human labeling or scoring
DERIVED = auto() # Computed from other data
UNKNOWN = auto() # Origin not recorded
@dataclass(frozen=True)
class DataProvenanceRecord:
"""Provenance record for a single data point or dataset."""
record_id: str
origin: DataOrigin
timestamp: str
instrument: str # Name of instrument or model
operator: str # Human or system that collected data
raw_data_hash: str # SHA-256 of raw data file
processing_steps: tuple[str, ...]
parent_records: tuple[str, ...] = () # For DERIVED data
@property
def integrity_hash(self) -> str:
"""Hash of the entire provenance record for tamper detection."""
content = json.dumps({
"record_id": self.record_id,
"origin": self.origin.name,
"timestamp": self.timestamp,
"instrument": self.instrument,
"operator": self.operator,
"raw_data_hash": self.raw_data_hash,
"processing_steps": list(self.processing_steps),
"parent_records": list(self.parent_records),
}, sort_keys=True)
return hashlib.sha256(content.encode()).hexdigest()
@dataclass
class ProvenanceChain:
"""A chain of provenance records tracking data from origin to
publication.
"""
records: dict[str, DataProvenanceRecord] = field(
default_factory=dict
)
def add_record(self, record: DataProvenanceRecord) -> None:
self.records[record.record_id] = record
def validate_chain(self, final_record_id: str) -> dict:
"""Validate the provenance chain for a given record.
Checks: (1) all parent records exist, (2) no AI_GENERATED
data without disclosure, (3) timestamps are chronological,
(4) integrity hashes are consistent.
"""
issues = []
visited = set()
chain_length = 0
def trace(record_id: str, depth: int = 0) -> None:
nonlocal chain_length
if record_id in visited:
return
visited.add(record_id)
chain_length = max(chain_length, depth + 1)
record = self.records.get(record_id)
if record is None:
issues.append(
f"Missing record: {record_id} referenced "
f"but not found in chain"
)
return
# Check for undisclosed AI-generated data
if record.origin == DataOrigin.AI_GENERATED:
issues.append(
f"Record {record_id}: AI-generated data "
f"detected. Must be disclosed as synthetic "
f"in the manuscript."
)
if record.origin == DataOrigin.UNKNOWN:
issues.append(
f"Record {record_id}: Data origin unknown. "
f"Cannot verify integrity."
)
# Recursively check parents
for parent_id in record.parent_records:
trace(parent_id, depth + 1)
trace(final_record_id)
return {
"valid": len(issues) == 0,
"issues": issues,
"chain_length": chain_length,
"records_checked": len(visited),
}
# Example: validate a dataset with proper provenance
chain = ProvenanceChain()
chain.add_record(DataProvenanceRecord(
record_id="RAW-001",
origin=DataOrigin.PHYSICAL_MEASUREMENT,
timestamp="2026-03-15T10:30:00Z",
instrument="Shimadzu UV-2600 Spectrophotometer",
operator="Dr. Sarah Chen",
raw_data_hash="a1b2c3d4e5f6...",
processing_steps=(),
))
chain.add_record(DataProvenanceRecord(
record_id="PROC-001",
origin=DataOrigin.DERIVED,
timestamp="2026-03-15T14:00:00Z",
instrument="Python 3.14 / scipy.signal",
operator="Discovery Workbench v3.2",
raw_data_hash="f6e5d4c3b2a1...",
processing_steps=("baseline_correction", "peak_detection",
"area_integration"),
parent_records=("RAW-001",),
))
chain.add_record(DataProvenanceRecord(
record_id="FIG-001",
origin=DataOrigin.DERIVED,
timestamp="2026-03-16T09:00:00Z",
instrument="matplotlib 3.9",
operator="Discovery Workbench v3.2",
raw_data_hash="1a2b3c4d5e6f...",
processing_steps=("normalization", "plot_generation"),
parent_records=("PROC-001",),
))
validation = chain.validate_chain("FIG-001")
print(f"Valid chain: {validation['valid']}")
print(f"Chain length: {validation['chain_length']}")
print(f"Records checked: {validation['records_checked']}")
print(f"Issues: {validation['issues']}")
Valid chain: True
Chain length: 3
Records checked: 3
Issues: []
The next example tests a chain that contains undisclosed AI-generated data:
# Add an AI-generated data point without disclosure
chain.add_record(DataProvenanceRecord(
record_id="SYN-001",
origin=DataOrigin.AI_GENERATED,
timestamp="2026-03-16T11:00:00Z",
instrument="GPT-4 (text-generation)",
operator="Discovery Workbench v3.2",
raw_data_hash="9z8y7x6w5v4u...",
processing_steps=("generate_supplementary_data",),
))
chain.add_record(DataProvenanceRecord(
record_id="TABLE-002",
origin=DataOrigin.DERIVED,
timestamp="2026-03-16T12:00:00Z",
instrument="pandas 2.2",
operator="Discovery Workbench v3.2",
raw_data_hash="u4v5w6x7y8z9...",
processing_steps=("merge", "format_table"),
parent_records=("PROC-001", "SYN-001"),
))
validation2 = chain.validate_chain("TABLE-002")
print(f"Valid chain: {validation2['valid']}")
for issue in validation2["issues"]:
print(f" Issue: {issue}")
Valid chain: False
Issue: Record SYN-001: AI-generated data detected. Must be disclosed as synthetic in the manuscript.
When a human scientist analyzes data, the provenance chain exists implicitly in their memory: they know which measurements came from which instrument on which day. When an autonomous system processes data, that implicit knowledge does not exist. Every transformation, every merge, every derived quantity must have an explicit, machine-readable provenance record. Without it, the system cannot distinguish between data that originated from a physical measurement and data that a language model hallucinated. The provenance chain from the experiment registry (Chapter 47) is therefore not a convenience; it is an integrity requirement for any autonomous discovery system that publishes results.
4. Automated Integrity Checks
Reports from 2023 onward describe cases in which AI-generated data reportedly passed through peer review at major journals because no automated check connected raw measurements to published figures. Without a unified integrity gate, fabricated results, undisclosed AI contributions, and paraphrased text slip into the scientific record one submission at a time.
We now combine the authorship checker, text similarity detector, and provenance validator into a unified integrity assessment that runs before any research output leaves the Discovery Workbench. Figure 57.2 shows how these four checks feed into a single pass/review/block decision.
from dataclasses import dataclass
@dataclass
class IntegrityAssessment:
"""Comprehensive integrity assessment for a research output."""
output_id: str
authorship_compliant: bool
authorship_violations: list[str]
text_similarity_issues: list[IntegrityIssue]
provenance_valid: bool
provenance_issues: list[str]
fabrication_risk: str # "none", "low", "medium", "high"
overall_status: str # "PASS", "REVIEW", "BLOCK"
def summary(self) -> str:
lines = [
f"Integrity Assessment: {self.output_id}",
f" Authorship: {'PASS' if self.authorship_compliant else 'FAIL'}",
f" Text similarity issues: {len(self.text_similarity_issues)}",
f" Provenance: {'VALID' if self.provenance_valid else 'INVALID'}",
f" Fabrication risk: {self.fabrication_risk}",
f" Overall: {self.overall_status}",
]
return "\n".join(lines)
def run_integrity_assessment(
output_id: str,
contributors: list[Contributor],
target_publisher: str,
manuscript_has_ai_disclosure: bool,
manuscript_text: str,
reference_texts: dict[str, str],
provenance_chain: ProvenanceChain,
final_data_record_id: str,
) -> IntegrityAssessment:
"""Run all integrity checks on a research output."""
# 1. Authorship compliance
auth_result = check_authorship_compliance(
contributors, target_publisher, manuscript_has_ai_disclosure
)
# 2. Text similarity
similarity_issues = check_text_similarity(
manuscript_text, reference_texts
)
# 3. Provenance validation
prov_result = provenance_chain.validate_chain(final_data_record_id)
# 4. Fabrication risk assessment
ai_data_records = [
r for r in provenance_chain.records.values()
if r.origin == DataOrigin.AI_GENERATED
]
if ai_data_records and not manuscript_has_ai_disclosure:
fab_risk = "high"
elif ai_data_records:
fab_risk = "medium"
elif any(r.origin == DataOrigin.UNKNOWN
for r in provenance_chain.records.values()):
fab_risk = "low"
else:
fab_risk = "none"
# 5. Overall status
if not auth_result.compliant or fab_risk == "high":
overall = "BLOCK"
elif (similarity_issues or not prov_result["valid"]
or fab_risk == "medium"):
overall = "REVIEW"
else:
overall = "PASS"
return IntegrityAssessment(
output_id=output_id,
authorship_compliant=auth_result.compliant,
authorship_violations=auth_result.violations,
text_similarity_issues=similarity_issues,
provenance_valid=prov_result["valid"],
provenance_issues=prov_result["issues"],
fabrication_risk=fab_risk,
overall_status=overall,
)
# Run a full integrity assessment
assessment = run_integrity_assessment(
output_id="MS-2026-DrugOpt-v1",
contributors=contributors, # from Listing 57.7
target_publisher="nature",
manuscript_has_ai_disclosure=True,
manuscript_text=manuscript, # from Listing 57.8
reference_texts=references,
provenance_chain=chain,
final_data_record_id="FIG-001",
)
print(assessment.summary())
Integrity Assessment: MS-2026-DrugOpt-v1
Authorship: PASS
Text similarity issues: 1
Provenance: VALID
Fabrication risk: none
Overall: REVIEW
Our n-gram fingerprinting approach is a teaching implementation. Production integrity checking uses specialized services:
# iThenticate: industry-standard plagiarism detection
# (API access requires institutional subscription)
import requests
def check_ithenticate(manuscript_path: str, api_key: str) -> dict:
"""Submit manuscript to iThenticate for plagiarism check.
Returns similarity report with per-source breakdown.
"""
with open(manuscript_path, "rb") as f:
response = requests.post(
"https://api.ithenticate.com/v2/submissions",
headers={"Authorization": f"Bearer {api_key}"},
files={"file": f},
)
return response.json()
# Crossref: verify that all citations exist
def verify_citations(dois: list[str]) -> dict[str, bool]:
"""Check each DOI against Crossref metadata API."""
results = {}
for doi in dois:
resp = requests.get(
f"https://api.crossref.org/works/{doi}",
headers={"User-Agent": "DiscoveryWorkbench/3.2"},
)
results[doi] = resp.status_code == 200
return results
According to its vendor, iThenticate compares manuscripts against a database of 90+ billion web pages and 70+ million published works, far beyond what any local n-gram checker can match. Crossref's metadata API covers 150+ million Digital Object Identifiers (DOIs) with sub-second lookup, verifying that every citation in the manuscript corresponds to a real publication. Our teaching implementation is 60 lines; these production tools replace it with 10 lines of API calls and orders-of-magnitude more comprehensive coverage.
5. Evolving Norms and Open Questions
The automated checks above enforce today's rules, but those rules are themselves a moving target as the research community grapples with questions that have no settled answers.
Publication ethics for AI-generated research is a rapidly evolving field. Several questions remain open as of 2026:
Three Unresolved Questions
When does AI assistance cross from "tool use" to "intellectual contribution"? A spell checker is unambiguously a tool. An AI system that designs a novel experiment, discovers a new compound, and writes the paper describing it is making intellectual contributions that would earn authorship if made by a human. The current consensus (AI cannot be an author) is based on the accountability argument, but some researchers argue this creates a perverse incentive: teams that use AI heavily may understate its role to avoid the perception that their human contributions are insufficient.
How should AI-generated hypotheses be cited? If a language model suggests a hypothesis that leads to a significant discovery, what attribution does the model deserve? Current norms require disclosure in the methods section, but this does not give the model (or its creators) credit in the citation network. Some proposals suggest a new metadata field for "AI-assisted ideation" in manuscript submission systems, analogous to the CRediT taxonomy's roles.
What constitutes "understanding" of AI-generated results? If an autonomous system discovers a new drug candidate through a process that no human fully understands (a common situation with deep learning), can the human authors honestly claim accountability for the result? The ICMJE requires authors to be accountable for "all aspects of the work." If "all aspects" includes the inner workings of a 100-billion-parameter model, no human can satisfy this criterion.
The Problematic Paper Screener, developed by Guillaume Cabanac, Cyril Labbé, and Alexander Magazinov and described in their 2024 update published in Scientometrics, uses automated heuristics to flag tortured phrases (e.g., "sham composing" instead of "fake writing"), suspicious reference patterns, and signs of paper-mill output (where a paper mill is a commercial operation that mass-produces fraudulent manuscripts for sale to researchers seeking publication credits) across millions of published articles. By 2024, the tool had flagged over 10,000 potentially problematic papers across major publishers, leading to hundreds of retractions. More recent systems build on this approach: the Science, Technology, and Medicine (STM) Integrity Hub (launched 2023 by the International Association of STM Publishers) aggregates signals from multiple detection tools into a shared screening pipeline that publishers query before acceptance. For autonomous discovery systems, these developments point toward a future where every AI-generated manuscript passes through a multi-layer integrity screen (text similarity, image forensics, statistical anomaly detection, reference verification) before it can enter peer review.
Try It: Build a Citation Integrity Checker
Build a small pipeline that validates whether references in an AI-generated text actually exist, using only Python and the free Crossref API. This project reinforces the fabricated citation problem discussed above.
- Generate a test bibliography. Prompt any available language model (or use a
local model via
ollama) to produce a 10-item bibliography on "autonomous chemical discovery." Save the output as a plain text file with one reference per line. - Extract DOIs and titles. Write a Python script using
reto parse each reference line. Extract DOI strings matching the pattern10\.\d{4,}/\S+and, for entries without DOIs, extract the title text between quotation marks. - Query Crossref. For each DOI, send a GET request to
https://api.crossref.org/works/{doi}usingrequests(orurllib). For title-only entries, queryhttps://api.crossref.org/works?query.title={title}&rows=1and check whether the top result's title closely matches (usedifflib.SequenceMatcherwith a threshold of 0.85). - Score and report. Classify each reference as "verified" (DOI resolves or title match above threshold), "suspicious" (partial match, score 0.5 to 0.85), or "fabricated" (no match found). Print a summary table with columns: reference number, status, and match score.
- Compare models. Repeat step 1 with a second model or a different prompt temperature, then compare fabrication rates. Record which model and temperature produced fewer fabricated citations, and whether fabricated entries had systematically different characteristics (missing DOIs, implausible journal names, future publication dates).
Exercise 57.2.1
A research team submits a paper to Nature listing the following contributors:
(1) Dr. Amara Osei, who supervised the project and approved the final manuscript;
(2) LabBot v2.1, an AI system that designed all experiments, ran the analysis, and
drafted the paper; (3) Dr. Kenji Tanaka, who provided the funding and reviewed the
final draft but did not run any experiments. The methods section includes a paragraph
disclosing that LabBot was used for experiment design and drafting. Using the ICMJE
criteria and the check_authorship_compliance function from Listing 57.7,
determine: (a) which contributor(s) qualify as authors, (b) whether the submission is
compliant with Nature's policy, and (c) what the expected warnings list
would contain. Write the Contributor objects and predict the output before
running the code.
Hint
Check whether LabBot holds any role that the policy treats as an authorship claim
(look at the WRITING_ORIGINAL role specifically). Also count the fraction
of total roles held by AI contributors; the warning threshold in the code is 70%.
Dr. Tanaka satisfies ICMJE criteria 2, 3, and 4 but arguably not criterion 1
(substantial contribution to conception, design, data acquisition, or analysis);
however, the compliance checker does not enforce ICMJE criteria directly, only the
publisher's AI-specific rules.
Step-Through: N-gram Fingerprint Overlap
Trace through check_text_similarity with a tiny example. Let
ngram_size = 3 and similarity_threshold = 0.2.
Manuscript (M): "the cat sat on the mat"
Reference (R): "the cat sat by the door"
Step 1: Build 3-grams.
M produces 4 trigrams: {"the cat sat", "cat sat on", "sat on the", "on the mat"}.
R produces 4 trigrams: {"the cat sat", "cat sat by", "sat by the", "by the door"}.
Step 2: Hash each trigram. Each is MD5-hashed and truncated to 12 hex characters. The exact hashes differ, but the important thing is that identical strings produce identical hashes.
Step 3: Compute overlap. The intersection contains 1 shared fingerprint (from "the cat sat"). Similarity = 1 / min(4, 4) = 0.25, which is 25%.
Step 4: Classify. 25% exceeds the threshold (0.2) but does not exceed 0.25 (the code checks similarity > 0.25, and 0.25 is not strictly greater), so it falls into the else branch: severity = "low" and issue_type = "close_paraphrase".
The function returns one IntegrityIssue.
Takeaway: Even a single shared 3-gram out of four yields 25% overlap, which triggers a review. Longer n-grams (the default is 5) reduce false positives because accidental 5-word matches are rarer than 3-word matches.
Real-World Application: Semantic Scholar's SPECTER-Based Integrity Pipeline
The Allen Institute for AI's Semantic Scholar platform uses Scientific Paper Embeddings using Citation-informed TransformERs (SPECTER) embeddings (dense vector representations of paper abstracts) to detect near-duplicate submissions across its corpus of 200+ million papers. Cosine similarity, where cosine similarity is a measure of how close two vectors point in the same direction (1.0 means identical, 0.0 means unrelated), serves as the comparison metric; papers above a threshold are flagged for editorial review. (As of 2024, SPECTER2, a multi-task successor trained on additional task-specific adapters, has largely replaced the original SPECTER for embedding-based retrieval and similarity detection.) This system has been instrumental in identifying paper-mill output, where hundreds of superficially varied manuscripts share nearly identical structure and claims.
The Ghost of Authors Past
In 1975, the physicist Jack H. Hetherington added his cat, F.D.C. Willard (short for "Felis Domesticus Chester, sired by Willard"), as co-author on a paper published in Physical Review Letters. Hetherington had written the paper using "we" throughout and, rather than rewrite it in the singular, simply added the cat. The journal accepted the paper, and F.D.C. Willard went on to be listed as sole author of a 1980 French physics paper. The episode is now a standard cautionary tale in publication ethics: if a cat can pass the authorship bar, the bar is too low. The modern AI authorship debate echoes the same core question, just with a more capable non-human contributor.
Lab: Provenance Chain Validator Stress Test
Goal: Explore how data provenance chains break in practice by constructing chains with deliberate gaps, cycles, and mixed origins, then observing which integrity violations the validator catches and which it misses.
Tools needed: Python 3.10+, the ProvenanceChain and
DataProvenanceRecord classes from Listing 57.9 (copy them into a script).
Procedure (20 minutes):
- Create a valid 4-record chain: PHYSICAL_MEASUREMENT, two DERIVED steps, and a final DERIVED figure. Verify it passes validation.
- Insert a record with
origin=DataOrigin.UNKNOWNat the second step. Run validation and note the issue message. - Add a record whose
parent_recordsreferences a nonexistent ID (e.g., "MISSING-999"). Observe whether the validator detects the dangling reference. - Create a cycle: record A lists record B as a parent, and record B lists record A.
Call
validate_chainon A and observe whether the validator terminates (thevisitedset should prevent infinite recursion). - Mix one AI_GENERATED record into an otherwise physical chain and set
manuscript_has_ai_disclosure=Falseinrun_integrity_assessment. Confirm that the overall status is "BLOCK".
What to vary: Try different chain lengths (2 vs. 10 records) and
different proportions of AI_GENERATED vs. PHYSICAL_MEASUREMENT origins.
What to observe: Which combinations produce PASS, REVIEW, and BLOCK
statuses, and whether any integrity failure modes slip through undetected (these are
real limitations of the teaching implementation).
What's Next
Safety (Section 57.1) tells us what can go wrong. Integrity (this section) tells us what honest science looks like. The missing piece is governance: the organizational structures, approval workflows, and regulatory frameworks that enforce safety and integrity requirements in practice. Section 57.3: Governance and Accountability builds that framework, connecting risk registers and integrity checks to approval hierarchies, audit trails, and regulatory compliance.
Bibliography
Science editor-in-chief's policy statement establishing that AI tools cannot be listed as authors on Science publications.
Nature's policy requiring AI tool disclosure in methods sections and prohibiting AI authorship.
Systematic study of citation fabrication rates in LLM-generated bibliographies, finding 69% fabrication in GPT-3.5 and 29% in GPT-4.
The standard reference for authorship criteria (the four ICMJE criteria) used throughout this section.
The 14-role taxonomy for research contributions that we adapt for AI contribution tracking.
Guidelines from the Committee on Publication Ethics on AI tool use and disclosure.