Overview
Software development is not manufacturing. It is discovery. Every feature begins as an uncertain guess about what users need, how the system should behave, and which technical approach will survive contact with production. The software development life cycle (SDLC) is, at its core, an iterative process of forming hypotheses, building experiments (code), running those experiments (tests, deployments, user feedback), and updating beliefs. This chapter makes that analogy rigorous.
We formalize the SDLC using the search framework from Chapter 1, where the state space $S$ consists of possible software systems, actions $A$ are development activities (writing code, refactoring, deploying), and the objective $f$ is a composite of user satisfaction, reliability, and maintainability. We show that agile methodologies, lean startup principles, and design thinking are all instances of hypothesis-driven search with different exploration strategies. We then quantify discovery velocity, the rate at which a team reduces uncertainty about what to build and how, and derive the expected regret of greedy development (building the first idea that seems plausible without exploring alternatives).
The tools of this chapter are Git, GitHub, and pytest: the instruments that let you version your hypotheses, collaborate on experiments, and automate falsification. By the end, you will have a complete recipe for transforming a vague product idea into falsifiable hypotheses, user stories, and skeleton tests, all tracked in a repository that serves as both a lab notebook and a production system.
Prerequisites
This chapter assumes you have read Chapter 1: Discovery As Search (for the search framework and regret analysis), Chapter 2: Scientific Discovery and Knowledge Creation (for the hypothesis testing paradigm), and Chapter 6: Discovery System Architecture (for the Discovery Workbench scaffold). Familiarity with basic Git operations (commit, branch, merge) and Python testing (assert statements, running pytest) is helpful but not strictly required; we build from first principles.
Learning Outcomes
- Model the software development life cycle as iterative hypothesis testing within the $(S, A, T, f, C)$ search framework.
- Map agile, lean, and design thinking methodologies to exploration strategies with distinct regret profiles.
- Define and measure discovery velocity: the rate at which a team reduces uncertainty about requirements, architecture, and implementation.
- Derive the expected regret of greedy (waterfall) development versus exploratory (iterative) development.
- Use Git branching as a mechanism for parallel hypothesis exploration and pytest as an automated falsification engine.
- Transform a vague product idea into falsifiable hypotheses, user stories, and executable skeleton tests.
Sections
7.1 SDLC as Iterative Hypothesis Testing
Formalizing the SDLC within the $(S, A, T, f, C)$ search framework. Requirements as prior distributions, code commits as state transitions, test suites as falsification oracles. Defining discovery velocity and deriving the expected regret of greedy (waterfall) versus exploratory (iterative) development.
7.2 Engineering Discovery and AI
Agile, lean, and design thinking as exploration strategies. How AI reshapes the economics of software discovery. LLM-assisted hypothesis generation, automated testing, and AI pair programming as uncertainty reducers.
7.3 Building a Hypothesis-Driven Dev Workflow
Recipe: from vague idea to falsifiable hypotheses, user stories, skeleton tests, and a living repository. Git as a lab notebook. pytest as an automated falsification engine. Extending the Discovery Workbench with a development tracker.
Bibliography
Foundational Papers
Articulates the value-driven perspective on software engineering that underpins the economic analysis of discovery velocity in Section 7.1.
The spiral model introduced risk-driven iteration into the SDLC, a direct precursor to hypothesis-driven development.
A comprehensive survey of LLM applications in software engineering, providing context for Section 7.2's discussion of AI-accelerated discovery.
Books
The canonical reference for build-measure-learn cycles, minimum viable products, and validated learning in product development.
Defines the deployment pipeline as an automated experiment runner, directly supporting the hypothesis-testing view of SDLC.
Tim Brown's articulation of design thinking as an iterative, human-centered discovery process for product innovation.
Tools & Libraries
The distributed version control system used throughout this chapter as a hypothesis versioning and branching mechanism.
Python's most popular testing framework, used here as an automated falsification engine for development hypotheses.
The collaboration platform whose pull requests, issues, and actions pipelines provide infrastructure for team-based hypothesis-driven development.
Tutorials & Surveys
Surveys the landscape of LLM-assisted software engineering, covering code generation, testing, debugging, and documentation.
A systematic review of empirical evidence for agile methods, relevant to our analysis of agile as a discovery strategy.
The most comprehensive recent review of LLMs in software engineering, covering 229 papers across code generation, testing, and maintenance.