KGpipe framework documentation

KGpipe is a framework to define pipelines for data integration into knowledge graphs. The framework enables you to compose existing tools and implementations into modular pipelines that integrate heterogeneous data sources into a unified knowledge graph.

The framework is organized into three main subpackages: kgpipe contains the core framework functionality including CLI, common utilities, execution backends, and evaluation components. kgpipe_tasks provides task implementations for cleaning, construction, entity resolution, schema alignment, and text processing. kgpipe_llm includes LLM-based task implementations and utilities.

Current version: 0.7.0
Python: >= 3.12

Quickstart

Start here: Quickstart guide

Minimal “happy path” (install + discover + inspect what’s available):

pip install -e .
kgpipe discover --all --show-results
kgpipe list --type tasks
kgpipe list --type metrics

Create a new experiment project (recommended):

cd experiments/examples
./init.sh

How to use KGpipe (docs map)

• Define tasks: Task specification
• Build and run pipelines: Pipelines
• Configure runs and task parameters: Configuration and Parameters
• Evaluate generated KGs: Evaluation and Metrics
• Understand the internal “PipeKG”: Meta KG

Other Links

• Reproducing the movie kg experiments for 15 pipelines (rdf, json, text)
• Adopting KGpipe (integrating existing pipelines/tools)
• UI / viewer

Docs backlog

Open items live in TODO.md (High/Medium/Low priority). Keep the landing page focused on user-facing docs.