Scruffy Knowledge Representation
24 Feb 2026 - 28 Aug 2026
- To build on the ideas in Politics and Pragmatism in Scientific Ontology Construction. It's interesting looking back on that paper, which was critical of the logicist/objectivist model of KR, but didn't really have very good ideas for going beyond it. It vaguely suggested a kind of fuzzy, divergent, resolved-at-the-edges model, but didn't go into it in any depth and I didn't really build anything based on those ideas.
- However, now that we are in the LLM era and all software has this sort of magic soft matching ability at runtime, maybe the situation has changed. In a way I was just too early – now, rigid objectivist ontologies feel even more out of place and we have the tools to do all the soft fuzzy stuff you could ever imagine.
- Marc-Antoine Parent ideas might be relevant.
- Should credit Jeff Shrager and BioBIKE where this position was first articulated.
Some assertions / positions
Every person/group/domain has an ontological project
Rather than force conformity, support diversity
And, support processes for reconciliation (static, dynamic, whatever)
The logicist/objectivist/semweb approach has failed
- Duh. But I don't want to be negative.
- Failed at what, exactly? Providing a common knowledge infrastructure. The evidence is, we don't have one. Not readily available and widely used, that is. There are particular ontologies that have been succesful in particular niches, but formalized knowledge in general remains ridiculously ad hoc
KR in the age of LLMs
- Obviously important and there is a shit-ton of stuff going on that I don't know about.
- This is the only interesting topic, the rest is posturing or complaining.
- Alright, fuck it, give me a summary of interesting work in integrating LLMs and traditional knowledge representation / ontologies. I know there's a fuckton of stuff, esp around knowledge graphs. See if you can summarize Ask Claude
- Here's the lay of the land. The field has basically organized itself into a few major clusters:
1. The Core Framing: LLMs and KGs as Complements, Not Rivals
- The dominant frame is that
- neural models like LLMs are non-deterministic, opaque, and tend to confabulate, while KGs contain factual information in symbolic form — but creating and curating KGs is time-consuming and resource-demanding.
- So the question is how to get the best of both.
- Neural networks in the form of LLMs provide the missing ingredient for a hybrid that realizes long-standing visions for symbolic AI, while explicit knowledge representation in KGs complements LLMs and helps them transcend their limitations.
- From a high level,
- the integration work falls into three strategies: KG-enhanced LLMs (KEL), LLM-enhanced KGs (LEK), and collaborative LLMs+KGs (LKC).
- Everything lives somewhere in that taxonomy.
2. LLM-Enhanced KGs: Using LLMs to Build and Maintain Ontologies/KGs
- With the advent of LLMs, KG construction has shifted from rule-based and statistical pipelines to language-driven and generative frameworks.
- Concretely:
- Ontology learning: There's now a whole challenge series around this —
- the "LLMs4OL" work marked a notable direction toward employing LLMs in ontology learning, demonstrating their potential in automating knowledge acquisition and representation for the Semantic Web.
- Key techniques identified:
- prompt engineering, retrieval-augmented generation (RAG), and ensemble learning.
- End-to-end ontology construction:
- Systems like NeOn-GPT and LLMs4Life have introduced end-to-end, prompt-driven workflows that integrate ontology reuse and adaptive refinement to construct deep, coherent ontological structures in complex scientific domains like life sciences.
- Ontology matching/alignment: This is a big subfield — using LLMs to reconcile heterogeneous ontologies.
- Ontology alignment is essential for enabling semantic interoperability across heterogeneous knowledge systems.
- Tools like OntoAligner and LLMs4OM are doing this at scale, and there's active work on fine-tuning on synthetic data to improve it.
- Dynamic/enterprise KG construction:
- AutoSchemaKG integrates schema-based and schema-free paradigms within a unified architecture, supporting the real-time generation and evolution of enterprise-scale knowledge graphs.
3. KG-Enhanced LLMs: Using KGs to Ground and Fix LLMs
- This is the hallucination-reduction angle.
- By grounding the generation process in a textualized or symbolic knowledge graph, LLMs can produce responses that are more accurate and aligned with real-world facts — especially useful in question answering, logical reasoning, and dialogue systems where factual precision is crucial.
- Graphs can be used to inject knowledge into LLMs via structured prompting or retrieval-based augmentation, while LLMs can support graph-centered tasks by performing entity linking, relation extraction, or link prediction, largely improving the graph's coverage.
4. GraphRAG: The Hot Practical Thing
- This is where most of the energy is right now. Standard RAG retrieves text chunks; GraphRAG retrieves from graph structure.
- Microsoft's GraphRAG constructs a graph-based index with an LLM, derives a knowledge graph from source documents, pre-generates community summaries for clustered entities, and aggregates partial responses into a final answer.
- LightRAG is a lightweight version that extracts entities and relations from source documents and generates a short description of each entity for retrieval.
- There's an explosion of variants:
- ArchRAG, KET-RAG, DIGIMON, PIKE-RAG, E²GraphRAG, OG-RAG (ontology-grounded), and many others, all targeting different cost/quality tradeoffs.
- LEGO-GraphRAG decomposes retrieval into subgraph-extraction, path-filtering, and path-refinement modules to improve reasoning.
5. Neuro-Symbolic: The Deeper Integration
- Beyond RAG-style retrieval, there's work on tighter coupling.
- Translation from natural language to symbolic language is accomplished via in-context LLM prompting, with deterministic symbolic solvers like Prover9 (FOL), Pyke (logic programming), or Z3 (SAT/SMT) executing inference on the symbolic representation.
- LLMs act as a bridge between symbolic and natural language spaces in this process.
6. What's Still Broken
- Despite the volume of work,
- several key research gaps remain — many studies claim LLMs are useful for ontology development tasks but do not clearly distinguish the specific development phases where LLMs provide the most value.
- And
- open problems persist in benchmarks, reproducibility, ontology drift, and the surprisingly thin public record of SHACL adoption in production.
So: Does This Vindicate Your Position?
- Sort of. The field is essentially enacting your "scruffy" intuition from the bottom up — LLMs do the fuzzy matching and semantic bridging that formal systems were always terrible at, while structured representations (KGs, ontologies) provide the anchoring, auditability, and reasoning that LLMs are bad at. The "reconciliation" problem you gestured at — resolving divergent ontological commitments — is now the active subfield of ontology alignment, and LLMs are genuinely making it tractable in ways that pure logic-based matchers never did.
- The thing that's still missing: a principled framework that takes your position seriously (every group has its own ontological project, don't force conformity, support reconciliation). Most of this work still implicitly assumes there's a right answer to converge to. The diversity-respecting, reconciliation-as-first-class-citizen framing hasn't really landed yet.