"I spent centuries encoding all of human knowledge into first-order logic, only to discover that the real world prefers a vector space with 768 dimensions and a healthy disregard for the law of the excluded middle."
A Knowledge Graph With Trust Issues
Chapter Overview
Before an AI system can discover anything, it must represent what it already knows. This chapter surveys the major paradigms for encoding scientific knowledge in machine-readable form: formal logic and ontologies, knowledge graphs and their embedding models, and dense vector representations with the data structures that make them searchable at scale. Each paradigm answers a different question. Logic asks "what follows from what?" Graphs ask "what connects to what?" Vectors ask "what is close to what?" A discovery system needs all three.
We begin with the symbolic tradition (Section 3.1): propositional and first-order logic, description logics, and the ontology languages built on top of them (OWL, RDF Schema). We then move to knowledge graphs (Section 3.2), the practical workhorse of modern knowledge representation, and derive the loss functions that let neural networks learn low-dimensional embeddings of entities and relations (TransE, RotatE, ComplEx). Section 3.3 turns to dense vector representations and the approximate nearest-neighbor structures (HNSW) that make retrieval tractable in millions of dimensions. Finally, Section 3.4 brings everything together in a hands-on recipe: building a hybrid scientific search system that combines sparse keyword matching (BM25) with dense semantic search over a corpus of research papers.
The tools you will use in this chapter include NetworkX, RDFLib, Neo4j, Qdrant, and pgvector. By chapter's end you will have a working hybrid search index that you can point at any scientific corpus.
Prerequisites
This chapter assumes you have read Chapter 1: Discovery as Search (the state-space framing we reuse throughout) and Chapter 2: Scientific Discovery and Knowledge Creation (the epistemological context for what counts as "knowledge"). Familiarity with Python, basic linear algebra (dot products, matrix multiplication), and elementary probability will suffice. No prior experience with graph databases, RDF, or vector databases is assumed.
Learning Outcomes
- Translate informal scientific claims into propositional logic, first-order logic, and description logic, and explain the expressiveness/decidability trade-offs.
- Construct a knowledge graph from structured data, query it with Cypher and SPARQL, and explain the open-world assumption.
- Derive and implement the TransE, RotatE, and ComplEx scoring functions, and train entity/relation embeddings on a benchmark dataset.
- Build an HNSW index from scratch, explain its logarithmic search complexity, and compare it against brute-force and tree-based alternatives.
- Implement a hybrid BM25 + dense-embedding search pipeline and evaluate it on held-out scientific queries.
Sections
3.1 Logic, Ontologies, and Semantic Networks
Logic and formal languages; ontologies and schema languages; semantic networks and frames. From propositional logic to OWL, with the expressiveness ladder that connects them.
3.2 Knowledge Graphs
Triples, entities, relations; the graph $G = (V, E, L)$; embedding objectives: TransE, RotatE, ComplEx. Building and querying knowledge graphs with Neo4j and RDFLib.
3.3 Embeddings and Vector Search
From words to concepts; vector databases; HNSW index construction; cosine similarity. The geometry of meaning and the data structures that exploit it.
3.4 Building a Hybrid Scientific Search
Recipe: scientific concept graph from papers, hybrid BM25 + dense search, evaluation on held-out queries. A complete, deployable search pipeline.
What's Next
With knowledge represented in graphs, vectors, and formal structures, the next step is to reason over it. Chapter 4: Reasoning for Discovery picks up exactly where this chapter ends: given a knowledge base, how does an AI system draw conclusions, test hypotheses, and generate explanations? The logical foundations from Section 3.1 will ground deductive reasoning; the graph structures from Section 3.2 will support path-based inference; and the embedding spaces from Section 3.3 will enable analogical reasoning through vector arithmetic. Later in the book, Chapter 38: Knowledge Graph Discovery revisits knowledge graphs at production scale, and Chapter 37: Retrieval Augmented Discovery Systems extends the hybrid search recipe from Section 3.4 into a full RAG pipeline.
Bibliography
Foundational Papers
Bordes, A., Usunier, N., Garcia-Durán, A., Weston, J., & Yakhnenko, O. (2013). Translating Embeddings for Modeling Multi-relational Data. NeurIPS. The paper that introduced TransE and the translation-based paradigm for knowledge graph embeddings.
Sun, Z., Deng, Z.-H., Nie, J.-Y., & Tang, J. (2019). RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space. ICLR. Models relations as rotations in complex vector space, capturing symmetry, antisymmetry, inversion, and composition patterns.
Trouillon, T., Welbl, J., Riedel, S., Gaussier, E., & Bouchard, G. (2016). Complex Embeddings for Simple Link Prediction. ICML. ComplEx: bilinear scoring in complex space for antisymmetric relations.
Malkov, Y. A., & Yashunin, D. A. (2020). Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs. IEEE TPAMI. The HNSW algorithm that underpins most modern vector databases.
Books
Baader, F., Calvanese, D., McGuinness, D. L., Nardi, D., & Patel-Schneider, P. F. (Eds.) (2007). The Description Logic Handbook. Cambridge University Press. The comprehensive reference for description logics and their role in ontology languages.
Hogan, A., et al. (2021). Knowledge Graphs. ACM Computing Surveys. A 200-page survey covering data models, query languages, graph analytics, embeddings, and applications.
Tools & Libraries
Neo4j. Neo4j Graph Database Documentation. Property graph model and Cypher query language, the most widely deployed graph database.
RDFLib. RDFLib Documentation. Pure-Python library for RDF graphs, SPARQL queries, and ontology manipulation.
Qdrant. Qdrant Documentation. High-performance vector similarity search engine with filtering and payload support.
pgvector. pgvector GitHub. Open-source vector similarity search extension for PostgreSQL.
NetworkX. NetworkX Documentation. Python library for creation, manipulation, and study of complex networks.
Tutorials & Datasets
Toutanova, K., & Chen, D. (2015). Observed Versus Latent Features for Knowledge Base and Text Inference. ACL Workshop. Introduced the FB15k-237 benchmark, the standard for evaluating KG embedding models after filtering test leakage.
Robertson, S. E., & Zaragoza, H. (2009). The Probabilistic Relevance Framework: BM25 and Beyond. Foundations and Trends in Information Retrieval. The definitive treatment of BM25 and the probabilistic relevance model.