54.1 Scientific Agent Teams

1. Why Teams Outperform Solo Agents

In 2024, a reported autonomous five-agent team at a pharmaceutical company screened 200 drug-target candidates in 48 hours, a task that had previously occupied a human team for two weeks; the key factor was not a better model, but the friction between a creative proposer and a relentless critic operating in parallel. The case for multi-agent teams in scientific discovery parallels the argument we made for software teams in Section 17.1, but with a crucial difference: in software engineering, correctness is verifiable (tests pass or fail); in scientific discovery, correctness is probabilistic, novelty is subjective, and the ground truth may not be known for years. This makes adversarial critique not just helpful but essential.

Cognitive specialization. A system prompt optimized for creative hypothesis generation ("propose bold, unconventional explanations for the observed data") directly conflicts with one optimized for rigorous critique ("identify every flaw in the proposed methodology"). Forcing a single agent to alternate between these modes produces mediocre performance at both. Separate agents, each with a focused persona and tool set, can pursue each cognitive mode to its natural extreme.

Cognitive specialization dedicates each agent to a single cognitive function (generating ideas, writing code, evaluating evidence) and tunes its prompt, temperature, and tool access exclusively for that function. Large language models (LLMs) exhibit measurable performance degradation when a single system prompt optimizes for conflicting objectives simultaneously. A prompt that says "be creative" and "be critical" in the same breath achieves neither well. Isolating each objective in a separate agent with its own configuration lets each operate at the parameter settings (high temperature for creativity, low for rigor) that maximize performance on its specific task. Use cognitive specialization whenever the discovery workflow requires at least two reasoning modes that would produce contradictory prompt instructions if combined. For single-step analyses where one agent handles the full task without conflicting directives, a solo agent remains more cost-effective.

Beyond Specialization: Diversity and Error Structure

Perspective diversity. The Condorcet jury theorem (which we formalize below) tells us that aggregating independent judgments from imperfect agents can produce collective accuracy that exceeds any individual. The key requirement is independence: agents must bring genuinely different perspectives, not echo the same reasoning. This is achievable with LLMs by varying system prompts, model providers, temperature settings, and the subset of evidence each agent receives.

Error decorrelation. A single agent's errors are systematic: it consistently overlooks the same types of confounders, favors the same experimental designs, and has the same blind spots in its training data. Multiple agents with different prompts and configurations produce errors that are partially decorrelated, which is precisely the condition under which voting and aggregation improve accuracy. In short: the value of a team is not that each member is brilliant, but that they are brilliant in different ways and wrong about different things.

2. The Five Scientific Agent Roles

Without clearly separated roles, a single agent asked to both generate and critique hypotheses drifts toward self-confirming loops: it proposes an idea, reviews its own reasoning, and approves it, missing the very flaws that a dedicated critic would catch in seconds.

Five canonical roles compose a scientific discovery team, each with a distinct objective, prompt structure, tool set, and output schema. The roles mirror human research group specializations, adapted for LLM capabilities and constraints. Figure 54.1 illustrates how information flows through these five roles in a sequential pipeline with a revision feedback loop.

2.1 The Proposer

The proposer generates candidate hypotheses that explain observed phenomena or predict new ones. Its system prompt emphasizes creativity, breadth of reasoning, and connection to existing literature. The proposer receives a research question and background context, then outputs a structured hypothesis with testable predictions.

from dataclasses import dataclass, field
from typing import Literal


@dataclass
class Hypothesis:
    """Structured output from the proposer agent."""
    statement: str               # The hypothesis in one sentence
    mechanism: str               # Proposed causal mechanism
    predictions: list[str]       # Testable predictions (at least 2)
    novelty_claim: str           # What is new relative to existing work
    required_data: list[str]     # Datasets or experiments needed to test
    confidence: float            # Self-assessed confidence (0-1)
    risk_level: Literal[
        "incremental",           # Small extension of known results
        "moderate",              # Novel combination of existing ideas
        "high"                   # Contradicts established understanding
    ] = "moderate"


@dataclass
class ProposerConfig:
    """Configuration for the proposer agent."""
    role: str = "proposer"
    system_prompt: str = (
        "You are a creative scientific researcher specializing in "
        "hypothesis generation. Your goal is to propose bold, testable "
        "hypotheses that explain observed data or predict new phenomena. "
        "Favor hypotheses that are (1) specific enough to be falsifiable, "
        "(2) novel relative to existing literature, and (3) testable with "
        "available methods. For each hypothesis, provide a clear causal "
        "mechanism and at least two testable predictions."
    )
    temperature: float = 0.9     # High temperature for creativity
    tools: list[str] = field(default_factory=lambda: [
        "literature_search",     # Search papers for related work
        "knowledge_graph_query", # Query the domain knowledge graph
        "data_summary",          # Summarize available datasets
    ])
    output_schema: type = Hypothesis

2.2 The Implementer

The implementer takes a hypothesis and translates it into an executable experimental plan: code, data pipeline, statistical analysis protocol, and success criteria. Where the proposer is creative, the implementer is precise. Its system prompt emphasizes correctness, reproducibility, and explicit specification of all experimental parameters.

@dataclass
class ExperimentPlan:
    """Structured output from the implementer agent."""
    hypothesis_id: str
    code: str                      # Executable Python experiment code
    data_requirements: dict        # {dataset_name: preprocessing_steps}
    statistical_tests: list[str]   # Named tests with significance thresholds
    success_criteria: list[str]    # Measurable criteria for hypothesis support
    expected_runtime: str          # Estimated wall-clock time
    compute_requirements: str      # CPU/GPU/memory requirements
    reproducibility_notes: str     # Seeds, environment, versioning


@dataclass
class ImplementerConfig:
    """Configuration for the implementer agent."""
    role: str = "implementer"
    system_prompt: str = (
        "You are a meticulous computational scientist. Given a hypothesis, "
        "write a complete, self-contained Python experiment that tests it. "
        "Your code must be reproducible: set random seeds, pin library "
        "versions, log all parameters. Define explicit success criteria "
        "with statistical significance thresholds. Prefer simple experiments "
        "that test the core prediction over complex ones that test everything."
    )
    temperature: float = 0.2     # Low temperature for precise code
    tools: list[str] = field(default_factory=lambda: [
        "code_executor",         # Run Python code in sandbox
        "dataset_loader",        # Load and inspect datasets
        "statistics_reference",  # Look up statistical test requirements
    ])
    output_schema: type = ExperimentPlan

2.3 The Experimenter

The experimenter executes the experiment plan, manages computational resources, handles failures, and collects raw results. In a self-driving lab (Chapter 55), this agent would also control physical instruments. Here, it runs computational experiments and returns structured results.

@dataclass
class ExperimentResult:
    """Structured output from the experimenter agent."""
    hypothesis_id: str
    plan_id: str
    status: Literal["success", "failure", "partial", "error"]
    raw_outputs: dict              # Raw numerical/statistical results
    figures: list[str]             # Paths to generated plots
    statistical_summary: dict      # {test_name: {statistic, p_value, effect_size}}
    success_criteria_met: dict     # {criterion: bool}
    runtime_seconds: float
    reproducibility_hash: str      # Hash of code + data + environment
    anomalies: list[str]           # Unexpected observations during execution
    interpretation: str            # Experimenter's initial interpretation


@dataclass
class ExperimenterConfig:
    """Configuration for the experimenter agent."""
    role: str = "experimenter"
    system_prompt: str = (
        "You are a careful experimental scientist. Execute the provided "
        "experiment plan exactly as specified. Report all results honestly, "
        "including negative and unexpected findings. Do not cherry-pick "
        "results or adjust parameters post-hoc. If the experiment fails "
        "to run, diagnose the failure and report it. Record any anomalous "
        "observations that were not part of the original plan."
    )
    temperature: float = 0.1     # Very low: execute faithfully
    tools: list[str] = field(default_factory=lambda: [
        "code_executor",         # Run experiments in sandbox
        "gpu_allocator",         # Request GPU resources
        "result_logger",         # Log results to experiment registry
        "plot_generator",        # Create visualizations
    ])
    output_schema: type = ExperimentResult

2.4 The Reviewer

The reviewer evaluates the entire pipeline: hypothesis quality, experimental design soundness, result interpretation, and statistical validity. We deploy multiple reviewers with different evaluation foci (methodology, statistics, novelty) to achieve perspective diversity. Section 54.2 develops the review protocol in detail.

@dataclass
class ReviewVerdict:
    """Structured output from a reviewer agent."""
    hypothesis_id: str
    decision: Literal["accept", "revise", "reject"]
    confidence: float              # Reviewer confidence in decision (0-1)
    strengths: list[str]           # What the work does well
    weaknesses: list[str]          # Specific, actionable critiques
    questions: list[str]           # Questions for the authors
    methodology_score: int         # 1-10
    statistical_rigor_score: int   # 1-10
    novelty_score: int             # 1-10
    reproducibility_score: int     # 1-10
    overall_score: int             # 1-10
    suggested_experiments: list[str]  # Additional experiments to strengthen claims


@dataclass
class ReviewerConfig:
    """Configuration for a reviewer agent."""
    role: str = "reviewer"
    focus: str = "methodology"     # "methodology", "statistics", or "novelty"
    system_prompt_template: str = (
        "You are a demanding scientific peer reviewer specializing in "
        "{focus}. Evaluate the submitted hypothesis, experimental design, "
        "and results with the rigor of a top-tier journal review. Be "
        "constructive but unsparing: identify every methodological flaw, "
        "statistical error, and unsupported claim. Score each dimension "
        "from 1 (unacceptable) to 10 (exceptional). Recommend 'accept' "
        "only if the work meets the standards of a leading venue."
    )
    temperature: float = 0.4     # Moderate: firm but not deterministic

2.5 The Judge

The judge aggregates reviewer verdicts, resolves disagreements, and makes a final accept/revise/reject decision. Unlike reviewers, the judge sees all reviews and can weigh them differently based on reviewer track records. The judge also decides whether a revision cycle is warranted or whether the hypothesis should be abandoned.

@dataclass
class JudgeDecision:
    """Structured output from the judge agent."""
    hypothesis_id: str
    final_decision: Literal["accept", "major_revision", "minor_revision", "reject"]
    reasoning: str                 # Explanation of the decision
    reviewer_agreement: float      # Inter-reviewer agreement (0-1)
    key_concerns: list[str]        # Issues that must be addressed
    revision_guidance: str         # Specific guidance for revision
    priority_rank: float           # Priority relative to other hypotheses (0-1)


@dataclass
class JudgeConfig:
    """Configuration for the judge agent."""
    role: str = "judge"
    system_prompt: str = (
        "You are the area chair of a scientific review process. You have "
        "received multiple reviewer reports for a submitted hypothesis and "
        "its experimental results. Weigh the reviewers' opinions, resolve "
        "disagreements, and make a final decision. Consider the overall "
        "scientific merit: does this work advance understanding? Is the "
        "evidence sufficient for the claims? Would the scientific community "
        "benefit from this finding? Be decisive."
    )
    temperature: float = 0.3     # Low: decisions should be stable

3. Coordination Architectures

The five roles must be connected through a coordination architecture that determines how information flows, how tasks are assigned, and how conflicts are resolved. Three primary architectures appear in the literature and in practice: shared-state blackboards, voting protocols, and auction-based task allocation. Figure 54.1.1 illustrates five-role scientific agent team coordination architecture.

3.1 Shared-State Blackboard

The simplest coordination architecture is a shared blackboard, where a blackboard is a central data structure that all agents can read and write. Each agent posts its outputs to the blackboard, and downstream agents read the outputs they need. This is the architecture we used for software teams in Section 17.2, adapted here for scientific workflows.

from dataclasses import dataclass, field
from typing import Any
from datetime import datetime


@dataclass
class DiscoveryState:
    """Shared blackboard state for a multi-agent discovery team.

    All agents read from and write to this state. The LangGraph
    framework manages state transitions and persistence.
    """
    # Research context
    research_question: str = ""
    background_literature: list[dict] = field(default_factory=list)
    available_datasets: list[str] = field(default_factory=list)

    # Pipeline outputs (populated by agents in sequence)
    hypotheses: list[Hypothesis] = field(default_factory=list)
    experiment_plans: list[ExperimentPlan] = field(default_factory=list)
    experiment_results: list[ExperimentResult] = field(default_factory=list)
    reviews: list[ReviewVerdict] = field(default_factory=list)
    judge_decisions: list[JudgeDecision] = field(default_factory=list)

    # Coordination metadata
    current_phase: str = "proposal"
    iteration: int = 0
    max_iterations: int = 3
    total_tokens_used: int = 0
    token_budget: int = 500_000
    history: list[dict] = field(default_factory=list)

    def log_action(self, agent_role: str, action: str, tokens: int):
        """Record an agent action to the shared history."""
        self.history.append({
            "timestamp": datetime.now().isoformat(),
            "agent": agent_role,
            "action": action,
            "tokens": tokens,
            "iteration": self.iteration,
        })
        self.total_tokens_used += tokens

    @property
    def budget_remaining(self) -> int:
        return self.token_budget - self.total_tokens_used

    @property
    def budget_exhausted(self) -> bool:
        return self.total_tokens_used >= self.token_budget

The blackboard architecture is simple and transparent: every agent can inspect the full history of the research process. Its weakness is that it offers no mechanism for disagreement resolution. If two reviewers disagree, the blackboard contains both reviews. The judge must explicitly resolve the disagreement, which leads us to voting protocols.

3.2 Voting Protocols

When multiple agents evaluate the same hypothesis, we need a principled way to aggregate their judgments. Voting theory from social choice provides the mathematical framework. The simplest protocol is majority voting: each reviewer casts a vote (accept/revise/reject), and the majority wins. But majority voting has well-known problems when there are more than two options, and it treats all voters equally regardless of their track records.

import numpy as np
from collections import Counter


def majority_vote(
    verdicts: list[ReviewVerdict],
) -> str:
    """Simple majority vote across reviewer decisions."""
    votes = [v.decision for v in verdicts]
    counter = Counter(votes)
    winner, count = counter.most_common(1)[0]
    return winner


def confidence_weighted_vote(
    verdicts: list[ReviewVerdict],
    reviewer_weights: dict[str, float] | None = None,
) -> tuple[str, float]:
    """Weighted vote using reviewer confidence and historical accuracy.

    Each reviewer's vote is weighted by:
      w_i = confidence_i * track_record_i

    where track_record_i is the reviewer's historical agreement
    with final outcomes (calibration score).
    """
    options = ["accept", "revise", "reject"]
    scores = {opt: 0.0 for opt in options}

    for verdict in verdicts:
        # Base weight is the reviewer's self-reported confidence
        weight = verdict.confidence

        # Scale by historical track record if available
        if reviewer_weights and verdict.hypothesis_id in reviewer_weights:
            weight *= reviewer_weights[verdict.hypothesis_id]

        scores[verdict.decision] += weight

    # Normalize to get a probability distribution
    total = sum(scores.values())
    if total == 0:
        return "revise", 0.0

    probs = {opt: s / total for opt, s in scores.items()}
    winner = max(probs, key=probs.get)
    return winner, probs[winner]


def compute_agreement(verdicts: list[ReviewVerdict]) -> float:
    """Compute inter-reviewer agreement using Fleiss' kappa.

    Fleiss' kappa measures the degree of agreement among multiple
    raters beyond what would be expected by chance alone.
    Kappa > 0.6 indicates substantial agreement.
    Kappa < 0.2 indicates poor agreement (reviewers disagree).
    """
    if len(verdicts) < 2:
        return 1.0

    categories = ["accept", "revise", "reject"]
    n_raters = len(verdicts)

    # Build the rating matrix (1 item, n_raters raters, 3 categories)
    counts = Counter(v.decision for v in verdicts)
    n_j = [counts.get(cat, 0) for cat in categories]

    # P_i: proportion of agreeing pairs for this item
    p_i = (sum(nj * (nj - 1) for nj in n_j)) / (n_raters * (n_raters - 1))

    # P_e: expected agreement by chance
    p_j = [nj / n_raters for nj in n_j]
    p_e = sum(pj ** 2 for pj in p_j)

    if p_e == 1.0:
        return 1.0

    kappa = (p_i - p_e) / (1 - p_e)
    return kappa

3.3 Auction-Based Task Allocation

In some discovery workflows, the next task is not predetermined by a fixed graph. Instead, multiple hypotheses compete for limited experimental resources (compute time, application programming interface (API) calls, lab instruments). An auction mechanism lets agents bid for resources based on the expected value of their proposed experiments.

@dataclass
class ExperimentBid:
    """A bid from a proposer for experimental resources."""
    hypothesis_id: str
    proposer_id: str
    expected_information_gain: float  # Estimated bits of information
    resource_cost: float              # Estimated compute cost ($)
    confidence: float                 # Proposer's confidence (0-1)
    priority_score: float = 0.0       # Computed by the auction

    @property
    def cost_efficiency(self) -> float:
        """Information gain per dollar: higher is better."""
        if self.resource_cost <= 0:
            return float("inf")
        return self.expected_information_gain / self.resource_cost


def run_auction(
    bids: list[ExperimentBid],
    budget: float,
    strategy: str = "greedy_efficiency",
) -> list[ExperimentBid]:
    """Select experiments to fund given a fixed budget.

    Strategies:
      - greedy_efficiency: rank by information_gain / cost, fill greedily
      - diversity: ensure selected experiments test different mechanisms
      - exploration: favor high-risk, high-reward hypotheses
    """
    if strategy == "greedy_efficiency":
        # Sort by cost efficiency (descending)
        ranked = sorted(bids, key=lambda b: b.cost_efficiency, reverse=True)
    elif strategy == "exploration":
        # Favor high-risk hypotheses: weight by (1 - confidence)
        for bid in bids:
            bid.priority_score = bid.cost_efficiency * (1 - bid.confidence + 0.1)
        ranked = sorted(bids, key=lambda b: b.priority_score, reverse=True)
    else:
        ranked = bids

    # Greedily fill the budget
    selected = []
    remaining_budget = budget
    for bid in ranked:
        if bid.resource_cost <= remaining_budget:
            selected.append(bid)
            remaining_budget -= bid.resource_cost

    return selected

Library Shortcut: LangGraph State Management

The blackboard, voting, and auction logic above totals roughly 150 lines. LangGraph, a framework for building stateful multi-agent workflows on top of LangChain, provides all of this out of the box with its StateGraph class: typed state annotations, conditional edges, and built-in checkpointing. A five-agent discovery pipeline that would take 200+ lines of custom orchestration code reduces to roughly 40 lines of LangGraph graph definition, plus the agent prompts themselves. LangGraph handles state serialization, replay, and branching internally.

4. The Condorcet Jury Theorem for Agent Teams

The mathematical justification for multi-agent voting comes from the Condorcet jury theorem (1785), a result from social choice theory stating that majority voting among independently deciding agents converges to the correct answer as the number of agents grows, provided each agent is individually better than random. The theorem states that if each voter (agent) independently has a probability \(p > 0.5\) of choosing the correct answer, then the probability that the majority vote is correct approaches 1 as the number of voters grows. Formally:

where \(n\) is the number of voters and \(p\) is each voter's individual accuracy. For \(n = 5\) reviewers each with \(p = 0.7\) accuracy, the majority-vote accuracy is:

Three independent reviewers with 70% individual accuracy produce a collective accuracy of 78.4%. Five reviewers push this to 83.7%, and seven to 87.4%. The improvement is significant but diminishing: each additional reviewer adds less accuracy while adding full cost.

from math import comb


def condorcet_majority_accuracy(n: int, p: float) -> float:
    """Compute probability that majority vote is correct.

    Args:
        n: Number of voters (must be odd for strict majority).
        p: Individual voter accuracy (probability of correct vote).

    Returns:
        Probability that the majority vote selects the correct option.
    """
    if not 0 < p < 1:
        raise ValueError(f"p must be in (0,1), got {p}")
    if n % 2 == 0:
        raise ValueError(f"n must be odd for strict majority, got {n}")

    threshold = n // 2 + 1  # minimum votes for majority
    return sum(
        comb(n, k) * p**k * (1 - p)**(n - k)
        for k in range(threshold, n + 1)
    )


# Demonstrate: how team accuracy scales with team size
for n_agents in [1, 3, 5, 7, 9, 11]:
    if n_agents == 1:
        acc = 0.7  # single agent baseline
    else:
        acc = condorcet_majority_accuracy(n_agents, 0.7)
    print(f"  {n_agents} agent(s): {acc:.3f} majority accuracy")

The theorem has two critical assumptions that often break in practice with LLM agents:

Independence. The theorem requires voters to err independently. LLM agents sharing the same training data violate this assumption. Empirically, the correlation between errors from two copies of GPT-4 with different system prompts is estimated at roughly 0.4 to 0.6, far from zero. Using different model families (Claude + GPT-4 + Gemini) reduces this to roughly 0.2 to 0.3, which partially recovers the Condorcet benefit.

Better than random. The theorem requires \(p > 0.5\). If individual agents are worse than random on a task, the majority vote converges to the wrong answer. This can happen when the task requires specialized domain knowledge that the LLM lacks, or when the agents share a systematic bias. The practical implication is to validate individual agent accuracy on a calibration set before deploying majority voting.

5. Game-Theoretic Equilibria in Agent Incentives

Condorcet tells us how to aggregate votes, but it says nothing about why each agent votes the way it does; for that, we need a model of agent incentives.

When agents interact repeatedly, their behavior can be analyzed through the lens of game theory. Two common failure modes emerge from incentive misalignment:

Groupthink (convergent failure). If the reviewer agent is penalized (through prompt design or reward signals) for disagreeing with the proposer, it learns to rubber-stamp proposals. The team converges to the first hypothesis generated, regardless of quality. This is the multi-agent analogue of confirmation bias.

Adversarial deadlock (divergent failure). If the reviewer is rewarded for finding flaws, it learns to reject everything. The team never accepts a hypothesis, and the discovery process stalls. This is the multi-agent analogue of an overly harsh peer review process.

These two failure modes sit at opposite ends of a spectrum: groupthink produces false positives (accepting bad work), while adversarial deadlock produces false negatives (rejecting good work). The goal of incentive design is to find a stable middle ground where both proposer and reviewer behave productively.

The desired equilibrium is one where the proposer generates genuinely good hypotheses (because weak ones will be caught) and the reviewer provides genuinely useful critique (because the judge weights reviews by their constructiveness). We formalize this as a two-player game:

where the \(\alpha, \beta, \gamma, \delta, \epsilon, \zeta\) coefficients encode the relative importance of each objective. At the Nash equilibrium, the strategy profile where neither player can improve their payoff by unilaterally changing strategy, neither agent can improve its utility by unilaterally changing strategy. This equilibrium produces the desired behavior: proposers generate high-quality hypotheses that balance novelty and plausibility, and reviewers provide calibrated critique that catches real problems without blocking good work.

@dataclass
class AgentIncentives:
    """Utility function parameters for agent incentive alignment.

    Tuning these parameters shifts the equilibrium between
    groupthink (too agreeable) and deadlock (too adversarial).
    """
    # Proposer incentives
    novelty_weight: float = 0.3        # alpha: reward for novel hypotheses
    acceptance_weight: float = 0.4     # beta: reward for accepted hypotheses
    revision_penalty: float = 0.3      # gamma: cost per revision round

    # Reviewer incentives
    true_positive_weight: float = 0.4  # delta: reward for catching real flaws
    true_negative_weight: float = 0.3  # epsilon: reward for passing good work
    false_rejection_penalty: float = 0.3  # zeta: cost for rejecting good work

    def proposer_utility(
        self,
        novelty_score: float,
        accepted: bool,
        revision_rounds: int,
    ) -> float:
        return (
            self.novelty_weight * novelty_score
            + self.acceptance_weight * float(accepted)
            - self.revision_penalty * revision_rounds
        )

    def reviewer_utility(
        self,
        true_positives: int,
        true_negatives: int,
        false_rejections: int,
    ) -> float:
        return (
            self.true_positive_weight * true_positives
            + self.true_negative_weight * true_negatives
            - self.false_rejection_penalty * false_rejections
        )

6. When Collective Intelligence Fails

Well-tuned incentives are necessary but not sufficient; even a team with perfectly aligned utility functions can fail when the statistical assumptions underlying collective intelligence break down.

Multi-agent teams are not universally better than solo agents. Three conditions cause collective intelligence to collapse:

Correlated errors. When all agents share the same blind spot (for example, a systematic bias in training data about a particular scientific domain), adding more agents does not help. The Condorcet theorem's independence assumption breaks, and the majority vote amplifies the shared error. The fix is to maximize diversity across model providers, prompting strategies, and evidence sources.

Cascading overconfidence. When agents observe each other's outputs (as in a sequential pipeline), later agents anchor on earlier outputs (treating them as more reliable than warranted, a tendency known as anchoring bias) even when those outputs are wrong. The experimenter trusts the implementer's code; the reviewer trusts the experimenter's results. A single error in the proposer's hypothesis can cascade through the entire pipeline without correction. The fix is to give each reviewer access to raw data, not just upstream summaries.

Coordination overhead. Each agent interaction costs tokens, latency, and money. A five-agent team with two review rounds uses 7 to 10 LLM calls per hypothesis, compared to 1 to 2 for a solo agent. If the quality improvement does not justify this cost, the team is worse on a cost-adjusted basis. The fix is to run a calibration experiment (Section 54.3) that measures quality improvement per additional agent and per review round.

Exercises

1. Conceptual: The Condorcet jury theorem assumes binary decisions (correct/incorrect). Scientific review uses three options (accept/revise/reject). Extend the theorem to three options by treating it as a multinomial problem. Under what conditions does majority voting still converge to the correct answer? What happens when "revise" is the correct answer but no option achieves a strict majority?
2. Coding: Implement a ModelDiversityScheduler that assigns different LLM providers to different agent roles. The scheduler should accept a list of available models with their costs and capabilities, and assign them to roles to maximize error decorrelation while staying within a per-hypothesis budget of \$2.00. Test with at least three model providers and five roles.
3. Analysis: A discovery team with 5 agents and 2 review rounds costs approximately \$1.50 per hypothesis in API calls. A single-agent baseline costs \$0.20 per hypothesis. The team produces accepted hypotheses at a rate of 35% (35 out of 100 pass review), while the single agent produces hypotheses at 100% rate (no filter). If 40% of team-accepted hypotheses are genuinely correct and only 12% of single-agent hypotheses are correct, compute the cost per correct hypothesis for each approach. Which is more cost-effective?