Haystack

Haystack is an open-source AI orchestration framework for building production-ready LLM applications in Python, developed by the German company deepset. First published in 2019 — several years before the term "retrieval-augmented generation" was coined — it is released under the Apache 2.0 licence and organises work into typed components connected into pipelines, a design formalised in the Haystack 2.0 rewrite of March 2024.^[1][2][3]

History

Haystack was created by deepset, founded by Milos Rusic, Malte Pietsch and Timo Möller, as an extractive question-answering framework built on top of transformer models — years before RAG pipelines became the standard way to ground language models in private documents.^[3][4] As the field moved from "answer from the model's memory" to "retrieve the document, then answer", the framework's retrieval-first design fitted naturally and Haystack became one of the standard options alongside LangChain and LlamaIndex.^[4]

On 11 March 2024 deepset released Haystack 2.0, a structural rewrite: every operation — embedding, retrieval, prompt building, an LLM call, parsing — became a component with declaratively typed inputs and outputs, and a pipeline became a graph of components wired by matching those types.^[3] Because pipelines serialise to YAML, they can be versioned as text, promoted between environments and generated programmatically.^[3] Deepset also offers a commercial platform built on the same framework, funded by a Series A round in 2024.^[4]

Key Concepts and Technology

Three ideas define Haystack. First, typed components: a custom component is a Python class with a run method and type annotations, so mismatches surface at pipeline-build time rather than at runtime.^[3] Second, pipeline graphs: retrieval, reranking and generation stages are connected by attribute name, which makes multi-retriever and hybrid-search designs explicit rather than hidden in chain calls. Third, serialisation: a YAML pipeline is an artefact engineers can review in a pull request — the property that pushes teams toward it once applications leave the notebook. Haystack ships with integrations for common vector databases, retrievers and LLM providers, and its 2.x line added native agent support.^[3]

Applications and Impact

Haystack is used for document question-answering, knowledge-base chatbots, recommendation and extraction pipelines — anywhere an enterprise wants answers grounded in its own corpus with a retriever and generator it can swap independently. Its significance is less about any single feature than about the maturation of the RAG stack: by making retrieval pipelines declarative and testable, frameworks like Haystack moved LLM application building from ad-hoc prompting toward ordinary software engineering.

>See Also

• Retrieval-Augmented Generation
• LangChain
• LlamaIndex
• Vector Database
• Embedding
• Reranking