6.1 Workbench Architecture Overview

"I am seven microservices in a trench coat pretending to be a scientist. The trench coat is YAML."

A Microservice That Achieved Consciousness

1. The Workbench as a Formal System

Three AI agents run parallel experiments overnight: one tests a drug combination, another re-analyzes last month's genomics data, and a third fine-tunes a model on freshly collected spectra. By morning each claims a breakthrough, but can you tell which results are trustworthy, which code produced them, and whether any agent silently reused corrupted data? A Discovery Workbench answers that question. It supports the full cycle introduced in Chapter 1 (formulating hypotheses, designing experiments, executing computations, analyzing results, and recording what happened) while keeping every piece traceable. We can formalize this as a seven-tuple:

A Discovery Workbench decomposes the scientific workflow into isolated, interchangeable layers. Each layer carries a typed contract, so you can swap, upgrade, or scale it independently. This matters because monolithic scripts that blend data loading, computation, and result storage become untraceable and unreproducible the moment a second person or a second machine joins the project. Each layer exposes a small set of abstract methods (register, retrieve, version, lineage) through Python protocols, where a protocol is a structural interface that defines a set of methods a class must implement without requiring explicit inheritance. Layers communicate only through these interfaces, never by sharing mutable state. You should reach for this layered design over a flat script whenever your work involves collaboration, automation, or any result you may need to reproduce later. For a quick one-off analysis you will never revisit, a single notebook remains perfectly adequate.

where \(D\) is the Data Layer (raw observations, curated datasets, and feature stores), \(C\) is the Code Layer (versioned scripts, notebooks, and pipelines), \(M\) is the Model Layer (trained weights, hyperparameters, and evaluation metrics), \(T\) is the Tool Layer (callable functions exposed to agents), \(A\) is the Agent Layer (autonomous decision-makers that plan and execute multi-step workflows), \(\Gamma\) is the Memory Layer (accumulated knowledge from past experiments, stored as embeddings and structured logs), and \(P\) is the Provenance Layer (the immutable record of who did what, when, with which inputs, producing which outputs).

What. Each symbol in the tuple represents a bounded responsibility: a set of artifacts, a set of operations on those artifacts, and a contract governing how this layer communicates with the others.

Why. Without explicit boundaries, responsibilities bleed together. A Jupyter notebook that loads data, trains a model, and writes results to a shared folder is doing the work of four layers at once. That works for a single experiment on a Tuesday afternoon. It does not work when three agents are running experiments in parallel and a reviewer asks which version of the training data produced Figure 7.

How. We define each layer as a Python protocol (or abstract base class) with typed inputs and outputs. Layers communicate through well-defined interfaces, never by reaching into each other's internal state.

When. You need this formalism the moment your discovery process involves more than one person, more than one machine, or more than one day. In other words, always. In short: seven layers, four universal methods, one guarantee: anything that happened can be traced, repeated, and explained.

Let us define the base contract that every layer must satisfy:

from __future__ import annotations
from abc import ABC, abstractmethod
from dataclasses import dataclass, field
from datetime import datetime
from typing import Any, Generic, TypeVar
from uuid import UUID, uuid4

T = TypeVar("T")


@dataclass(frozen=True)  # frozen: instances are immutable after creation
class LayerArtifact(Generic[T]):
    """Base class for any object managed by a workbench layer."""
    id: UUID = field(default_factory=uuid4)
    created_at: datetime = field(default_factory=datetime.utcnow)
    version: int = 1
    metadata: dict[str, Any] = field(default_factory=dict)


class WorkbenchLayer(ABC):
    """Contract that every layer in the workbench must satisfy."""

    @abstractmethod
    def register(self, artifact: LayerArtifact) -> UUID:
        """Store an artifact and return its unique identifier."""
        ...

    @abstractmethod
    def retrieve(self, artifact_id: UUID) -> LayerArtifact:
        """Fetch an artifact by its identifier."""
        ...

    @abstractmethod
    def list_versions(self, artifact_id: UUID) -> list[int]:
        """Return all available versions of an artifact."""
        ...

    @abstractmethod
    def lineage(self, artifact_id: UUID) -> list[UUID]:
        """Return the chain of artifact IDs that produced this one."""
        ...

The four methods in Listing 6.1 are intentionally minimal. register and retrieve give you a key-value store. list_versions gives you history. lineage gives you provenance. Every layer extends this contract with domain-specific methods, but these four are the universal connective tissue.

The foundation every experiment rests on is the data itself.

2. The Data Layer

What. The Data Layer manages raw observations, curated datasets, feature matrices, and any other input that experiments consume. Each dataset is an immutable, versioned artifact with a schema describing its columns, types, and units.

Why. Data is the empirical anchor of the entire discovery process. If your data changes silently (a CSV gets overwritten, a database row gets updated, a sensor stream drifts), every downstream result becomes suspect. The Data Layer enforces immutability: once a dataset version is registered, it never changes. New data creates a new version.

How. The Data Layer wraps storage backends (local files, cloud object stores, databases) with a registry that tracks content hashes, schemas, and access timestamps. The content hash \(h(d)\) for a dataset \(d\) is computed at registration time and verified at every retrieval, guaranteeing bit-for-bit integrity:

When. You interact with the Data Layer at the start of every experiment (to load inputs) and at the end (to store derived datasets for downstream use).

import hashlib
from dataclasses import dataclass, field
from pathlib import Path
from uuid import UUID, uuid4


@dataclass(frozen=True)
class Dataset(LayerArtifact):
    """An immutable, versioned dataset in the Data Layer."""
    name: str = ""
    path: Path = Path(".")
    content_hash: str = ""
    schema: dict[str, str] = field(default_factory=dict)
    n_rows: int = 0
    n_cols: int = 0

    @staticmethod
    def compute_hash(filepath: Path) -> str:
        """Compute SHA-256 hash of file contents."""
        sha = hashlib.sha256()
        with open(filepath, "rb") as f:
            for chunk in iter(lambda: f.read(8192), b""):
                sha.update(chunk)
        return sha.hexdigest()


class DataLayer(WorkbenchLayer):
    """Manages immutable, versioned datasets."""

    def __init__(self, storage_root: Path):
        self._root = storage_root
        self._registry: dict[UUID, Dataset] = {}

    def register(self, artifact: Dataset) -> UUID:
        content_hash = Dataset.compute_hash(artifact.path)
        registered = Dataset(
            id=artifact.id,
            name=artifact.name,
            path=artifact.path,
            content_hash=content_hash,
            schema=artifact.schema,
            n_rows=artifact.n_rows,
            n_cols=artifact.n_cols,
        )
        self._registry[registered.id] = registered
        return registered.id

    def retrieve(self, artifact_id: UUID) -> Dataset:
        ds = self._registry[artifact_id]
        # Verify integrity on every read
        current_hash = Dataset.compute_hash(ds.path)
        if current_hash != ds.content_hash:
            raise ValueError(
                f"Dataset {ds.name} has been modified since registration. "
                f"Expected hash {ds.content_hash[:12]}..., "
                f"got {current_hash[:12]}..."
            )
        return ds

    def list_versions(self, artifact_id: UUID) -> list[int]:
        return [self._registry[artifact_id].version]

    def lineage(self, artifact_id: UUID) -> list[UUID]:
        return [artifact_id]  # Raw data has no upstream lineage

3. The Code Layer

What. The Code Layer tracks every script, notebook, pipeline definition, and configuration file that participates in the discovery process. Each code artifact is pinned to a specific Git commit, so you can always recover the exact program that produced a given result.

Why. Data without code is a spreadsheet. Code without version control is a prayer. When a discovery agent runs an experiment, the Code Layer records which commit was checked out, which branch was active, and whether the working tree was clean. If the tree was dirty (uncommitted changes), the layer refuses to register the run, forcing the researcher or agent to commit first.

How. The Code Layer wraps Git (or another version control system) with a registry that associates code snapshots with experiment runs. The key invariant: no registered run may reference uncommitted code.

When. Every time you start an experiment, the Code Layer captures a snapshot. Every time you review a result, the Code Layer tells you exactly which code produced it.

4. The Model Layer

What. The Model Layer stores trained model weights, hyperparameters, training metrics, and evaluation results. Each model artifact is linked to the code that trained it, the data it was trained on, and the configuration that controlled the training process.

Why. A trained model is the most expensive artifact in most discovery pipelines. Training a large language model can cost millions of dollars (depending on model size and training duration); training even a modest neural network for a materials science application might take hours on a GPU cluster. The Model Layer ensures that once you have a trained model, you never lose it, and you always know how it was produced.

How. The Model Layer records model artifacts as tuples of \((w, \theta, \mu)\): weights \(w\), hyperparameters \(\theta\), and metrics \(\mu\). Each tuple is linked to the code commit and dataset version that produced it:

When. You write to the Model Layer after training completes. You read from it when loading a model for inference, evaluation, or fine-tuning.

5. The Tool Layer

What. The Tool Layer defines callable functions that agents can invoke during discovery workflows. A tool might run a simulation, query a database, call an external API, submit a compute job, or parse a PDF. Each tool has a typed signature (name, parameters, return type) and a human-readable description that agents use to decide when to call it.

Why. Agents cannot do useful work without tools. A large language model (LLM) can reason about chemistry, but it cannot run a density functional theory (DFT) calculation by thinking hard. The Tool Layer bridges the gap between reasoning and action. By making tools explicit, typed, and registered, we get three critical properties: (1) agents can discover available tools programmatically, (2) the provenance system can record which tools were invoked during a run, and (3) administrators can control which tools are available in which contexts (sandboxing).

How. Each tool is registered with a JSON Schema (a declarative format for describing the structure, types, and constraints of JSON data) describing its inputs and outputs. The Tool Layer validates inputs before execution and captures outputs (including errors) for the provenance record.

When. Agents query the Tool Layer's catalog at planning time to understand their capabilities. They invoke tools at execution time. The provenance system queries the Tool Layer at audit time to reconstruct what happened.

from dataclasses import dataclass, field
from typing import Any, Callable
from uuid import UUID


@dataclass
class ToolSpec:
    """Specification for a callable tool in the discovery workbench."""
    name: str
    description: str
    parameters_schema: dict[str, Any]   # JSON Schema for inputs
    return_schema: dict[str, Any]       # JSON Schema for output
    callable: Callable[..., Any] = lambda **kw: None
    requires_approval: bool = False     # human-in-the-loop gate
    max_runtime_seconds: int = 300
    cost_estimate_usd: float = 0.0


class ToolLayer:
    """Registry and executor for discovery tools."""

    def __init__(self):
        self._tools: dict[str, ToolSpec] = {}

    def register_tool(self, spec: ToolSpec) -> None:
        self._tools[spec.name] = spec

    def list_tools(self) -> list[dict[str, str]]:
        """Return tool catalog for agent consumption."""
        return [
            {"name": t.name, "description": t.description}
            for t in self._tools.values()
        ]

    def invoke(self, name: str, params: dict[str, Any]) -> dict[str, Any]:
        """Execute a tool, returning result and execution metadata."""
        spec = self._tools[name]
        if spec.requires_approval:
            raise PermissionError(
                f"Tool '{name}' requires human approval before execution."
            )
        # In production: validate params against parameters_schema
        result = spec.callable(**params)
        return {
            "tool": name,
            "params": params,
            "result": result,
            "cost_usd": spec.cost_estimate_usd,
        }


# Example: register a simulation tool
sim_tool = ToolSpec(
    name="run_dft_calculation",
    description="Run a density functional theory calculation on a molecular geometry.",
    parameters_schema={
        "type": "object",
        "properties": {
            "geometry_xyz": {"type": "string"},
            "functional": {"type": "string", "default": "B3LYP"},
            "basis_set": {"type": "string", "default": "6-31G*"},
        },
        "required": ["geometry_xyz"],
    },
    return_schema={
        "type": "object",
        "properties": {
            "energy_hartree": {"type": "number"},
            "converged": {"type": "boolean"},
        },
    },
    requires_approval=True,  # DFT is expensive
    max_runtime_seconds=3600,
    cost_estimate_usd=2.50,
)

6. The Agent Layer

What. The Agent Layer hosts autonomous decision-makers that plan and execute multi-step discovery workflows. An agent might formulate a hypothesis, design an experiment to test it, invoke tools to run the experiment, analyze the results, and decide whether to revise the hypothesis or move on. Each agent operates within a defined scope, budget, and set of permissions.

Why. The preceding four layers provide the raw materials (data, code, models, tools). The Agent Layer provides the intelligence that orchestrates them. Without agents, a human must manually wire together every step of the discovery pipeline. With agents, the system can explore hypothesis space autonomously, bounded by safety constraints defined in the Tool and Provenance Layers.

How. Agents are implemented as stateful loops that alternate between reasoning (choosing the next action) and acting (invoking tools). The Agent Layer provides scheduling, resource allocation, and isolation. Each agent has an identity, a set of permitted tools, a budget (compute time, API cost, number of tool invocations), and an audit trail.

When. You deploy agents when the discovery process is well-enough understood that you can define clear objectives, safety constraints, and success criteria. In early exploration, agents assist human researchers. In mature pipelines, agents run autonomously with periodic human review.

Agent-Tool Information Flow

The relationship between agents and tools follows a clear information flow. Let \(a \in A\) be an agent, \(t \in T\) be a tool, and \(\gamma \in \Gamma\) be a memory entry. At each step \(k\), the agent selects an action based on its current state and accumulated memory:

Here \(\pi\) is the agent's policy (the decision function that maps the agent's current observations and memory to the next action), \(s_k\) is the current state (including all prior observations), and \(t_{a_k}\) is the tool selected by action \(a_k\). The memory \(\Gamma_k\) refers to the Memory Layer defined in the next subsection; it grows with each step, accumulating observations that inform future decisions.

Autonomous agents gain speed, but without a record of past successes and failures they gain no wisdom; that accumulated experience is exactly what the next layer provides.

7. The Memory Layer

What. The Memory Layer stores accumulated knowledge from past experiments: which hypotheses were tested, which succeeded, which failed, what the agent learned, and what the system should remember for future runs. Memory is stored as both structured records (key-value pairs, experiment summaries) and dense embeddings (vector representations for semantic retrieval).

Why. Without memory, every experiment starts from scratch. An agent that does not remember its previous failures will repeat them. A system that does not record which regions of hypothesis space have already been explored will waste resources re-exploring them. The Memory Layer provides the "experience" that makes a discovery system improve over time, distinguishing it from a stateless script runner.

How. The Memory Layer combines a vector store (for semantic similarity search over past experiments) with a structured store (for exact retrieval of parameters, metrics, and outcomes). When an agent begins planning, it queries the Memory Layer with its current hypothesis to retrieve relevant prior experience. The retrieval function maps a query \(q\) to the \(k\) most relevant memories:

where \(\text{sim}\) is cosine similarity (the cosine of the angle between two vectors, ranging from -1 to 1, where higher values indicate greater semantic relatedness) and \(\text{embed}\) maps text to a dense vector representation.

When. The Memory Layer is written to after every experiment (success or failure). It is read from at the start of every planning phase. Over time, it becomes the most valuable artifact in the system, encoding institutional knowledge that survives personnel changes and hardware migrations.

8. The Provenance Layer

What. The Provenance Layer records the complete history of every action taken in the workbench: which data was loaded, which code was executed, which model was trained, which tools were invoked, which agent made the decision, and what the outcome was. Every record is immutable and append-only.

Why. Provenance is the layer that makes everything else trustworthy. Without it, you have results but no evidence. A reviewer asks: "How did you arrive at this structure?" Without provenance, you answer: "I think I used the B3LYP functional with a 6-31G* basis set, but I might have switched to PBE0 at some point." With provenance, you answer with a query that returns the exact pipeline, the exact inputs, the exact code commit, and the exact intermediate results.

How. The Provenance Layer implements the W3C PROV data model (a W3C standard that defines a vocabulary of entities, activities, and agents for representing provenance information in a machine-readable form), recording entities (artifacts), activities (computations), and agents (human or automated). Every run creates a provenance record linking inputs to outputs through an activity:

When. The Provenance Layer records data continuously during execution. It is queried during review, auditing, debugging, and reproduction.

Each layer handles one responsibility in isolation. The architecture's real power emerges when a single experiment flows through all seven layers together.

9. How the Layers Compose

When composition breaks, the consequences are concrete: one drug-discovery team lost three months of screening results because their pipeline wrote model outputs directly to the data store, bypassing provenance and making it impossible to determine which model version produced which predictions. Understanding how layers connect is what prevents that kind of silent, expensive failure.

The seven layers are not independent silos. They compose through typed interfaces, forming a dependency structure. Every experiment touches at least five of the seven layers. Figure 6.1 illustrates the information flow through the full stack:

1. The Agent consults the Memory Layer and formulates a plan.
2. The plan specifies which Data to load and which Code to execute.
3. Execution invokes Tools, which may train or evaluate Models.
4. Results flow back to the Agent, which updates Memory.
5. Every step is recorded in the Provenance Layer.

We can express the composition as a function. Given an experiment specification \(e\), the workbench produces a result \(r\) and a provenance trace \(p\):

The \(\circ\) symbol denotes function composition: \(f \circ g\) means "apply \(g\) first, then feed its output to \(f\)." Reading right to left: the Memory Layer provides context, the Data, Code, and Model Layers supply materials, the Tool Layer executes computations, the Agent Layer orchestrates the process, and the Provenance Layer wraps everything in an auditable record. Figure 6.1.1 illustrates the seven-layer workbench architecture stack.

10. Putting It Together: The Workbench Registry

The complete workbench is a composite object that holds references to all seven layers and provides a unified interface for running experiments. Here is a minimal but functional implementation:

from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
from uuid import UUID


@dataclass
class ExperimentSpec:
    """What an agent submits to run an experiment."""
    hypothesis: str
    dataset_id: UUID
    code_commit: str
    tool_sequence: list[str]
    parameters: dict[str, Any] = field(default_factory=dict)
    budget_usd: float = 10.0
    max_steps: int = 100


@dataclass
class ExperimentResult:
    """What the workbench returns after an experiment completes."""
    run_id: UUID
    status: str                         # "completed" | "failed" | "budget_exceeded"
    outputs: dict[str, Any] = field(default_factory=dict)
    metrics: dict[str, float] = field(default_factory=dict)
    provenance_ids: list[UUID] = field(default_factory=list)
    cost_usd: float = 0.0


class DiscoveryWorkbench:
    """Top-level orchestrator that composes all seven layers."""

    def __init__(
        self,
        data: DataLayer,
        tool: ToolLayer,
        storage_root: Path = Path("./workbench"),
    ):
        self.data = data
        self.tool = tool
        self._storage = storage_root
        self._runs: dict[UUID, ExperimentResult] = {}

    def run_experiment(self, spec: ExperimentSpec) -> ExperimentResult:
        """Execute a full experiment through all seven layers."""
        from uuid import uuid4

        run_id = uuid4()
        provenance_chain: list[UUID] = []
        total_cost = 0.0

        # 1. Retrieve dataset (Data Layer)
        dataset = self.data.retrieve(spec.dataset_id)
        provenance_chain.append(dataset.id)

        # 2. Verify code commit (Code Layer, simplified)
        # In production: check git status, refuse dirty trees
        assert len(spec.code_commit) == 40, "Provide a full SHA-1 commit hash."
        provenance_chain.append(uuid4())  # code snapshot ID

        # 3. Execute tool sequence (Tool + Model Layers)
        outputs: dict[str, Any] = {}
        for tool_name in spec.tool_sequence:
            if total_cost >= spec.budget_usd:
                return ExperimentResult(
                    run_id=run_id,
                    status="budget_exceeded",
                    cost_usd=total_cost,
                    provenance_ids=provenance_chain,
                )
            result = self.tool.invoke(tool_name, spec.parameters)
            outputs[tool_name] = result["result"]
            total_cost += result.get("cost_usd", 0.0)
            provenance_chain.append(uuid4())

        # 4. Record result (Provenance Layer)
        experiment_result = ExperimentResult(
            run_id=run_id,
            status="completed",
            outputs=outputs,
            provenance_ids=provenance_chain,
            cost_usd=total_cost,
        )
        self._runs[run_id] = experiment_result
        return experiment_result

Listing 6.4 is deliberately incomplete: it covers four layers (Data, Code, Tool, Provenance) and leaves Agent and Memory for later chapters. The architecture supports incremental extension. Adding the Memory Layer should not require changes to the Data Layer; adding a new agent type should not require changes to the Tool Layer. If modifying layer \(X\) forces a change in layer \(Y\), the boundary between them is wrong.