Standard retrieval-augmented generation chatbots have long excelled at answering straightforward, isolated inquiries, such as retrieving a company’s return policy. However, when prompted with complex, multi-hop questions requiring the synthesis of disparate facts across a document collection—such as understanding how a return policy interacts with international warranty terms—traditional vector-based systems frequently falter. This fundamental limitation stems from vector search engines matching semantic similarities between queries and isolated text chunks without understanding the underlying relationships between those pieces of information.
To bridge this gap, AWS Labs has released an open-source, Apache 2.0-licensed Python toolkit known as graphrag-toolkit. Designed to facilitate graph-enhanced retrieval, the framework provides developers and enterprises with the tools necessary to move beyond simple semantic matching and build applications that reason over connected facts. The introduction of this toolkit arrives as major database providers increasingly highlight the advantages of combining the structural rigor of knowledge graphs with the generative fluency of large language models.
The core premise behind graph-enhanced RAG, often referred to as GraphRAG, is that information in the real world is inherently interconnected. Industry leaders like Neo4j and Oracle have independently emphasized that standard RAG treats every document chunk as an independent island, missing the broader context of how entities appear across multiple documents or how concepts relate across an entire corpus. While a traditional vector store evaluates semantic similarity for each chunk in isolation, it lacks any native representation of how chunks relate to one another, whether they discuss the same entity, or whether they share a cause-and-effect dependency. This structural bottleneck means that multi-hop questions often fail not because the underlying language model lacks reasoning capabilities, but because vector search fails to surface all the necessary pieces of information in the first place.
The newly introduced AWS Labs toolkit consists of two primary components: lexical-graph and BYOKG-RAG. The lexical-graph framework is designed to automatically build and query a hierarchical knowledge graph directly from unstructured documents. Meanwhile, BYOKG-RAG, or Bring Your Own Knowledge Graph, offers an alternative multi-strategy question-answering layer for organizations that already maintain a pre-existing knowledge graph and wish to avoid re-extracting data from scratch. The design principles behind the hierarchical lexical graph model are further detailed in a research paper titled "Hierarchical Lexical Graph for Enhanced Multi-Hop Retrieval," and the technology has already found traction in production environments, such as Deloitte’s Amazon Neptune-based GenAI Security Intelligence Center.
Under the hood, a lexical graph organizes extracted content into six distinct, interconnected levels ranging from the original source down to chunks, topics, statements, facts, and entities. This multi-layered hierarchy allows retrieval mechanisms to hop dynamically between levels rather than being restricted to the chunk-level retrieval typical of plain vector stores. The graph construction process is divided into two separate, resumable stages: extraction and building. During extraction, raw documents are ingested, chunked, and processed using language models to identify topics, statements, facts, and entities. The building stage then ingests these structured elements to populate the graph and vector databases.
To maintain high extraction quality, the pipeline employs a two-step language model approach for each chunk. The initial step refines raw text into self-contained propositions, while the subsequent step extracts topics, statements, facts, and entities from those cleaned propositions rather than messy source text. Once the graph is established, queries utilize a traversal-based search mechanism. A user’s query is embedded to find initial anchor points via vector similarity, which then seed graph traversal queries to pull in thematically connected content from across the wider graph. This retrieved information is subsequently supplied to a foundation model alongside the original prompt to generate a comprehensive answer.
The toolkit supports a variety of backend integrations for graph and vector storage, including Amazon Neptune, Neo4j, FalkorDB, Amazon OpenSearch Serverless, PostgreSQL with pgvector, and Amazon S3 Vectors. For developers looking to experiment locally without incurring cloud infrastructure costs, combinations such as Neo4j and PostgreSQL running via Docker containers provide a zero-infrastructure environment. For foundation models, the toolkit is configured by default to integrate with Amazon Bedrock, requiring an active AWS account with model access enabled, even when the graph and vector databases are hosted locally.
As organizations increasingly demand more reliable and sophisticated generative AI assistants capable of navigating intricate documentation, tools like the graphrag-toolkit highlight a significant shift in how retrieval systems are architected. By moving beyond isolated text chunks and embracing traversal-based graph search, developers can build applications capable of answering complex, multi-hop queries with greater accuracy and contextual awareness.
