Haystack: the enterprise RAG framework from deepset in Berlin

What is Haystack?

Haystack is an open-source framework for RAG pipelines, developed since 2019 by deepset AG (Berlin, Germany). Apache 2.0 license, Python only. In May 2026 in version 2.x stable – a full rewrite over Haystack 1.x, completed in mid-2024 and the recommended variant since.

The commercial backing is deepset (founded 2018 by Milos Rusic, Malte Pietsch, Timo Moeller). deepset offers deepset Cloud as managed service with enterprise SLA, deepset Studio as visual builder for pipelines, and commercial support for Haystack installations. In May 2026 headquarters sits in Berlin and Boston, focused on European enterprise customers in finance, insurance, pharmaceutical, and government.

The core concept of Haystack 2.x is the pipeline as a directed graph of components. Components implement the Component protocol with defined inputs and outputs (type-safe via Python type hints). Pipelines connect components via explicit connect calls, which documents the data flow clearly. Conceptually cleaner than LangChain LCEL or LlamaIndex Settings – a pipeline configuration in Haystack can be serialised as YAML, versioned via Git, and deployed to production without code changes.

Component inventory in May 2026: Document Stores (Elasticsearch, Weaviate, Pinecone, Qdrant, ChromaDB, Postgres, OpenSearch). Embedders (SentenceTransformers, OpenAI, Cohere, Voyage, local models). Retrievers (BM25, Embedding, Hybrid). Generators (OpenAI, Anthropic, Mistral, AWS Bedrock, Azure OpenAI, Ollama, local HuggingFace). Plus specialised components for table extraction, image embedding, re-ranking, tool calling.

Production maturity is Haystack unique selling point. The library has been in productive use at deepset customers in finance since 2019 – DZ Bank, Airbus, Deutsche Telekom, several federal ministries. Stability, security reviews, documentation, and commercial support contracts lift Haystack clearly above hobby grade.

Why it matters

For Swiss fiduciaries and SMEs, Haystack in May 2026 is the right choice when enterprise requirements are in play. Three reasons speak for Haystack over LangChain or LlamaIndex.

First: stability and trust. A RAG pipeline for a bank, an insurer, or a hospital must run for years with minimal breaking changes and a well-documented audit trail. Haystack 2.x has been stable since mid-2024 – no breaking-change drama like LangChain. The deepset roadmap communicates transparently in May 2026 what is planned and what is being deprecated, with 12-month deprecation windows.

Second: German origin and EU compliance. deepset is a German AG with Berlin headquarters, a practical advantage for EU GDPR data-processing agreements. deepset Cloud runs in EU regions, a BDPA under German law is standard. For Swiss customers under revDSG demand, a meaningful difference from US cloud providers like LangSmith or LlamaCloud.

Third: pipeline clarity. A Haystack pipeline is explicitly documented in a YAML file. What comes in, what comes out, which components are connected, which parameters are set – all in one file, versionable in Git, reviewable by a compliance officer without Python knowledge. With LangChain and LlamaIndex the pipeline is scattered in code, making reviews harder.

The trade-off is reach. Haystack is in May 2026 less mainstream than LangChain – smaller community, fewer Stack Overflow answers, fewer tutorials. The deepset documentation is good, but external learning resources are thin. Anyone entering without commercial support must invest more own effort.

For the Swiss fiduciary market we recommend in May 2026 Haystack when the customer has high audit demands (e.g. FINMA-supervised, bank compliance), when a deepset support contract fits the budget (about EUR 30-60k/year for mid-size setups), or when the pipeline should be maintained via YAML versioning. Otherwise LlamaIndex is more pragmatic.

How it works

A Haystack pipeline consists of components and connections. Components have defined input slots and output slots; connections map one component output to another input.

Example indexing pipeline for a fiduciary knowledge base:

from haystack import Pipeline from haystack.components.converters import PyPDFToDocument from haystack.components.preprocessors import DocumentSplitter, DocumentCleaner from haystack.components.embedders import SentenceTransformersDocumentEmbedder from haystack_integrations.document_stores.qdrant import QdrantDocumentStore from haystack.components.writers import DocumentWriter

document_store = QdrantDocumentStore(url="http://localhost:6333", index="fiduciary_kb")

indexing = Pipeline() indexing.add_component("converter", PyPDFToDocument()) indexing.add_component("cleaner", DocumentCleaner()) indexing.add_component("splitter", DocumentSplitter(split_by="sentence", split_length=5)) indexing.add_component("embedder", SentenceTransformersDocumentEmbedder(model="BAAI/bge-m3")) indexing.add_component("writer", DocumentWriter(document_store=document_store))

indexing.connect("converter", "cleaner") indexing.connect("cleaner", "splitter") indexing.connect("splitter", "embedder") indexing.connect("embedder", "writer")

indexing.run({"converter": {"sources": ["./docs/handbook.pdf"]}})

The query pipeline is similarly structured: TextEmbedder -> QdrantEmbeddingRetriever -> PromptBuilder -> OpenAIGenerator. Connections explicit, inputs and outputs type-safe.

The YAML export feature is in May 2026 a Haystack highlight. pipeline.dumps() returns a full pipeline definition as YAML – components, parameters, connections. This YAML reads back via Pipeline.loads() without code changes. A pipeline can be read and approved by compliance officers or external auditors.

For tool calling and agentic workflows, Haystack 2.x offers the ToolInvoker component and ChatGenerator component. In May 2026 the agent functionality is less mature than LangGraph – for complex multi-step agents LangGraph remains recommended. For simple tool-use patterns Haystack suffices.

deepset Cloud offers three tiers: Free (pilot), Pro (USD 750/month), and Enterprise (contract). The cloud variant covers hosted pipelines, vector DB, monitoring dashboard, and Studio UI. EU region as default, US region optional.

deepset Studio (web UI) enables visual pipeline building without code – suitable for data analysts and non-technical domain experts. Exports the pipeline as Python code and YAML.

Haystack setup in 5 steps

1. 01Clarify compliance requirements: data residency (EU/CH), audit-trail demand, pipeline-review stakeholders, security level. If deepset support is desired: get a quote and plan it into the project budget.
2. 02Sketch the pipeline skeleton: indexing pipeline (Loader -> Cleaner -> Splitter -> Embedder -> Writer) and query pipeline (TextEmbedder -> Retriever -> PromptBuilder -> Generator). Enable YAML export from the start.
3. 03Pick components: Document Store (Qdrant in EU region as default), embedder (BGE-M3 for multilingual, OpenAI text-embedding-3-small for English-strong), generator (OpenAI gpt-4o-mini, Mistral EU, or local Llama 3.x).
4. 04YAML export as configuration artefact: version pipeline.dumps() in Git, add reviews by compliance officer, changes via pull request.
5. 05Evaluation and tracing: Haystack evaluation components for recall and answer-quality tests, OpenTelemetry export to own Grafana or Langfuse self-host. Only after green evaluation go to production.

When to use Haystack

Haystack is the right choice when (a) enterprise compliance is required, (b) pipeline reviews by non-technical stakeholders are needed, or (c) deepset support fits the budget.

Concrete cases: a Swiss bank builds an internal knowledge database for compliance staff with documented audit trail – Haystack with YAML-versioned pipelines plus deepset Cloud in EU region. An insurance IT department wants RAG for internal policy search, with security review by group IT – Haystack 2.x as open-source self-host, pipeline configuration in internal Git repository. A pharma company seeks a RAG system for regulatory documents – deepset Studio as visual builder for domain experts without Python knowledge.

For multilingual RAG pipelines (DE + EN + FR + IT), Haystack with BGE-M3 or Cohere Multilingual as embedder is well positioned – important for Swiss applications with confederation languages.

For table-heavy documents (balance sheets, Excel reports, regulatory tables), Haystack offers TableExtractor components comparable to LlamaParse.

When not to use

For small PoC projects without enterprise demand, Haystack is over-sized – LlamaIndex is faster productive and easier to learn.

For complex agentic workflows with many tool calls and state management, LangGraph is more mature in May 2026.

For solo developers or small teams without compliance pressure, the Haystack learning curve is disproportionate to the value – the YAML-pipeline advantage pays off only when several stakeholders must review the pipeline.

For JavaScript/TypeScript stacks, Haystack is not first choice – no JS port available in May 2026. LlamaIndex.TS or LangChain.js are the options.

For pure visual no-code setups without a code component, Flowise or Langflow fit better. deepset Studio is visual, but Haystack code stays in the background.

For extremely low latency demands (sub-200 ms), Haystack is slightly slower than a custom direct call through its pipeline component layer. At critical latency, a custom build with the raw vector-DB client and LLM SDK is faster.

Without budget for commercial support, one of the big Haystack advantages disappears. Pure open-source Haystack without deepset support remains usable, but then the extra effort over LlamaIndex rarely pays off.

Trade-offs

FAQ

Related topics

Sources

1. deepset-ai/haystack – GitHub repository and releases · 2026-05
2. Haystack 2.x documentation – pipelines and components · 2026-05
3. deepset Cloud – managed Haystack with EU residency · 2026-04
4. deepset blog – Haystack 2.x architecture and migration · 2026-03