"I had a perfectly reproducible experiment once. Then I upgraded pip."
A Docker Container That Survived Three Migrations
Overview
Every chapter so far has equipped you with a concept: discovery as search (Chapter 1), the structure of scientific knowledge (Chapter 2), knowledge representation (Chapter 3), reasoning (Chapter 4), and the interplay of data, models, and simulation (Chapter 5). These concepts are powerful, but they live in isolation until something connects them. That something is architecture: the skeleton that holds data, code, models, agents, and provenance together into a system that can actually do discovery.
This chapter introduces the Discovery Workbench, the platform that will grow through every subsequent chapter of this book. By the end of Chapter 58, the Workbench will be a full autonomous discovery system. Here, we build its first bones: a typed artifact graph, a task log, an API layer, and a reproducibility harness. The design is deliberately minimal. A good scaffold constrains just enough to prevent chaos, and leaves everything else open for the chapters that will extend it.
Prerequisites
You should be comfortable with the ideas from Chapter 1 (discovery as search through state spaces), Chapter 3 (knowledge representation and graphs), and Chapter 5 (data, models, and simulation as first-class objects). Familiarity with Python, REST APIs (we use FastAPI), and basic SQL will serve you well. If you have used Git and Docker, the reproducibility sections will feel natural; if not, we introduce just enough to get started.
Learning Outcomes
After completing this chapter, you will be able to:
- Describe the layered architecture of a discovery system and explain why each layer exists.
- Model scientific artifacts as typed, versioned nodes in a directed acyclic graph (DAG) with provenance edges.
- Design run records that capture inputs, outputs, parameters, and human approval gates.
- Implement safety boundaries, cost controls, and observability for agent-driven discovery.
- Bootstrap a working Discovery Workbench v0 with an artifact registry, task log, API, and reproducibility CLI.
Sections
6.1 Workbench Architecture Overview
The seven layers of a discovery system: data, code, models, tools, agents, memory, and provenance. How they compose, why the boundaries exist, and what happens when you skip one.
6.2 Artifact Graphs and Provenance
Typed nodes, versioned edges, and evidence links. The run record schema. Human-in-the-loop approval gates. How to query "where did this result come from?" in constant time.
6.3 Safety, Cost, and Observability
Sandboxes for agent execution. Safety boundaries that prevent runaway experiments. Cost and latency budgets. Observability with OpenTelemetry, structured logging, and dashboards.
6.4 Bootstrapping the Discovery Workbench
Recipe: build Workbench v0 with SQLite-backed artifact registry, task log, FastAPI layer, Git-based reproducibility, and a CLI that ties it all together.
What's Next
With the Discovery Workbench scaffold in place, we cross from foundations into practice. Chapter 7: Software Development as a Discovery Process reframes the act of writing software as itself a form of scientific discovery, and begins extending the Workbench with the tools of modern AI-assisted engineering. The architecture you build here will carry every extension from vibe coding (Chapter 9) to autonomous software organizations (Chapter 24) and beyond.
Bibliography
Foundational Papers
Demonstrates a full autonomous discovery loop with LLM agents, highlighting the architecture decisions that make agent-driven science reproducible.
Tool-augmented LLM architecture for chemistry; the tool registry pattern maps directly onto our artifact graph design.
The W3C standard for provenance; our run record schema draws on its Entity-Activity-Agent triple.
Books
The definitive reference on data system architecture; our layered design owes much to its treatment of storage, encoding, and distributed processing.
Pipeline architecture patterns for ML systems; its treatment of artifact stores and metadata tracking directly informs Section 6.2.
Tools & Libraries
The experiment tracking and model registry system we wrap in Section 6.4; its run/artifact/metric abstraction is the industry standard.
The API framework used throughout the Workbench; its automatic OpenAPI schema generation simplifies agent-to-Workbench communication.
The observability standard we adopt in Section 6.3 for traces, metrics, and logs across discovery system components.
Git-compatible data and model versioning; complements our artifact graph with large-file tracking.
Containerization for reproducible environments; our Workbench v0 uses Docker to freeze the execution context of each run.
Tutorials & Guides
Argues that modern AI capabilities come from systems of many components, not single models; the architectural mindset behind this chapter.
Comprehensive survey of LLM agent architectures; its taxonomy of memory, planning, and tool-use modules maps onto our layered design.