Overview

Relevant source files

• README.md
• assets/aws-fargate-logo.svg
• assets/azure-logo.svg
• assets/gcp-logo.svg
• pyproject.toml

Purpose and Scope

The llm-app repository is a technical framework designed for creating responsive, production-ready AI applications. It leverages the Pathway Live Data Framework to provide high-accuracy Retrieval-Augmented Generation (RAG) and enterprise search at scale. The primary goal is to enable developers to build applications that stay synchronized with live data sources (files, databases, APIs) without the need for complex infrastructure like separate vector databases or cache layers.

The repository provides a set of LLM App Templates that can be deployed on-cloud (GCP, AWS, Azure) or on-premises, supporting Python versions >=3.10,<3.13.

Sources: README.md13-15 pyproject.toml2-4 pyproject.toml33

High-Level Architecture: The Live Data Framework

The core differentiator of this repository is the use of the Pathway reactive engine. Unlike traditional RAG stacks that require manual ETL (Extract, Transform, Load) processes to update a vector database, Pathway treats data as a live stream. When a document is added, updated, or deleted in a source like Google Drive or a local folder, the change propagates through the pipeline in real-time, updating the in-memory vector index.

Code Entity Space to Natural Language Space

The following diagram maps the logical components of the AI pipeline to the specific code entities and libraries used in the llm-app implementation.

Diagram: Component Mapping

Sources: README.md13-15 README.md38-39 pyproject.toml34

Available Templates

The repository contains several specialized templates located in the templates/ directory. These templates range from simple RAG implementations to complex multimodal and stateful alerting systems.

Template Name	Purpose	Key Features
Question-Answering RAG	Basic end-to-end RAG.	PDF/DOCX support, GPT integration, REST API.
Live Document Indexing	Standalone Vector Store service.	Real-time indexing, LangChain/LlamaIndex integration.
Multimodal RAG	Processing charts and tables.	GPT-4o vision parsing, unstructured document extraction.
Unstructured-to-SQL	NL2SQL for financial data.	PDF to PostgreSQL mapping, LLM-generated SQL queries.
Adaptive RAG	Token cost optimization.	4x cost reduction, geometric retrieval strategy.
Private RAG	Local-first, air-gapped AI.	Mistral, Ollama, local SentenceTransformers.
Slides AI Search	Presentation discovery.	Multi-modal slide indexing (PPTX/PDF), VLM support.
Video RAG	Video content search.	TwelveLabs Pegasus/Marengo integration, video-to-text.
Document Store MCP	Model Context Protocol server.	Exposes document store as tools for Claude Desktop.

Sources: README.md22-31

High-Level Data Flow

The data flow within an llm-app template is managed by a pathway reactive graph. The system maintains a "live" state, meaning the DocumentStore is always a reflection of the connected sources.

Diagram: Data Flow and Serving

Sources: README.md13-15 README.md33-38

Core System Architecture Overview

Every template in the repository follows a unified architectural pattern:

1. Configuration: Defined in app.yaml, specifying connectors, model parameters, and environment variables.
2. Entrypoint: Usually app.py, which initializes the Pathway components and calls pw.run().
3. Engine: The pathway0 library provides the Rust-based execution engine that handles parallel processing and incremental updates.
4. Indexing: Built-in vector search using the usearch library and hybrid search using tantivy.
5. Serving: Integrated HTTP server that exposes endpoints for querying (/v2/answer), retrieval (/v1/retrieve), and monitoring (/v1/statistics).

Sources: README.md33-38 pyproject.toml34

Next Steps

For a deep dive into the technical implementation of these components, refer to the following sections:

• Core System Architecture: Detailed breakdown of the reactive engine and shared components.
• RAG Pipelines: Deep dives into the specific RAG implementations like Adaptive and Private RAG.
• Specialized Pipelines: Documentation for unique use cases like Video RAG and Unstructured-to-SQL.
• Deployment: Instructions for Docker, persistence, and cloud scaling.