BGE-Reranker-v2-m3: open-source reranker for multilingual RAG setups

What is BGE-Reranker?

BGE-Reranker is a family of cross-encoder models from the Beijing Academy of Artificial Intelligence (BAAI), released under Apache 2.0 on HuggingFace. The family covers multiple sizes and language profiles; the dominant variant in May 2026 is bge-reranker-v2-m3. It uses the same XLM-RoBERTa base as the BGE-M3 embedding model, so it covers over 100 languages and can use the same preprocessing.

Unlike an embedding model, a reranker does not produce a vector per text. Instead it processes question and document passage together in a single forward pass and returns a relevance score. This cross-encoder architecture is markedly more accurate than pure vector similarity because the model can see word-level relationships between question and document directly.

bge-reranker-v2-m3 has about 568 million parameters, similar in size to its embedding sibling BGE-M3. Model files weigh about 2.3 GB. Inference runs comfortably on a single GPU with 8 GB VRAM; on CPU with ONNX runtime it is markedly slower (2-5 seconds per 50-document batch) but possible. On an RTX 3060 the model ranks about 50 candidates in 150-200 ms – comparable to the Cohere API.

Quality-wise bge-reranker-v2-m3 sits 2-4 points behind Cohere rerank-multilingual-v3.0 on MTEB reranking sub-tracks in May 2026, but clearly ahead of all other open-source rerankers (mxbai-rerank, sentence-transformers ms-marco, Jina-Reranker). On German and French it is the best freely licensed reranker we know.

Why it matters for Switzerland

Three reasons make BGE-Reranker the default for Swiss on-prem RAG setups. First, the Apache 2.0 licence. Unlike Cohere or Voyage Rerank there is no vendor contract, no DPA, no third-country transfer. The model runs in your own container and sees question-document pairs only internally. For mandates under SCC Art. 321 professional secrecy this is often the only legally clean option.

Second, the natural pairing with BGE-M3 as the embedding model. Using BGE-M3 in stage 1 gives you the same backbone architecture, the same tokenisation, the same language profile in the stage-2 reranker. This consistency avoids unintended semantic drift between stages and makes the pipeline easier to calibrate. In a concrete fiduciary setup we measured +18 percent Recall@5 over pure dense search – almost the same gain as with Cohere Rerank, without API costs.

Third, multilinguality at top level. On German- and French-centred reranking tasks bge-reranker-v2-m3 sits nearly level with Cohere. Italian is good, Romansh untrained – but that holds for all models. Anyone building a RAG stack across CH-relevant languages gets very-close-to-best-in-class quality with BGE-Reranker without the US vendor hop.

Also relevant for the Swiss market: BGE-Reranker has a mature community, many sample notebooks, good documentation in the FlagEmbedding library, and is built in as first-class integration in LangChain, LlamaIndex, Haystack. Operationalisation is no secret knowledge – a typical backend engineer can integrate the model into an existing RAG pipeline within a day.

How it works

BGE-Reranker-v2-m3 is a classic cross-encoder that sends a question-document pair through a 24-layer transformer network and emits a scalar score. Unlike a bi-encoder embedding model, each pair requires its own forward pass – making stage-2 latency linear in the number of candidates.

Integration via the FlagEmbedding library:

```python from FlagEmbedding import FlagReranker

reranker = FlagReranker("BAAI/bge-reranker-v2-m3", use_fp16=True)

pairs = [ [query, candidate_1], [query, candidate_2], # up to 50-100 candidates from stage 1 ]

scores = reranker.compute_score(pairs, normalize=True) ranked = sorted(zip(scores, pairs), reverse=True)[:5] ```

The normalize parameter applies a sigmoid to the logits and returns scores between 0 and 1 – interpretable as probability that the passage matches the question. Without normalisation the values are unscaled.

For production we recommend a FastAPI wrapper:

```python from fastapi import FastAPI from FlagEmbedding import FlagReranker

app = FastAPI() reranker = FlagReranker("BAAI/bge-reranker-v2-m3", use_fp16=True)

@app.post("/v1/rerank") async def rerank(query: str, documents: list[str], top_n: int = 5): pairs = [[query, d] for d in documents] scores = reranker.compute_score(pairs, normalize=True) ranked = sorted(zip(range(len(documents)), scores), key=lambda x: -x[1])[:top_n] return {"results": [{"index": i, "score": s} for i, s in ranked]} ```

The reranker is then a central HTTP service used by several applications. Model loaded once, GPU usage efficient.

For CPU setups the model runs via ONNX runtime. BAAI publishes ONNX conversions; performance on an 8-core VM is about 5-10 seconds per 50 candidates – acceptable for standard fiduciary loads, too slow for interactive apps with many queries per second.

The stage-2 pipeline: stage 1 Qdrant pulls 30-50 candidates via vector similarity (top-K=30 standard, top-K=50 when recall is priority). Stage 2 BGE-Reranker scores all candidates pairwise, returns top-K=3-5. Stage 3 the LLM receives top candidates as context and generates the answer. Latency profile typically: 30 ms stage 1, 200 ms stage 2, 800-1500 ms stage 3 – LLM latency dominates.

BGE-Reranker to production in 5 steps

1. 01Hardware: ideally GPU with 8 GB VRAM (RTX 3060 or better) for sub-200 ms latency; CPU with ONNX as fallback for smaller loads.
2. 02Pull the model via FlagEmbedding library (BAAI/bge-reranker-v2-m3) or directly via HuggingFace CLI. Separate model cache from container lifetime.
3. 03FastAPI wrapper: POST /v1/rerank with query, documents, top_n. Log calls without content persistence. Monitor GPU usage via nvidia-smi.
4. 04Pipeline integration: stage 1 Qdrant pulls 30-50 candidates, stage 2 calls the reranker endpoint, top 3-5 go to stage 3 (LLM).
5. 05Eval suite against baseline: measure Recall@5 before reranker, then with, document the delta. Expect +12-18 points. Less gain: raise stage-1 top_k or check the embedding model.

When to use BGE-Reranker

BGE-Reranker-v2-m3 is the right choice when (a) reranking must run on your own infrastructure (professional secrecy, nFADP strictness), (b) the corpus is multilingual with DE/FR/IT/EN mix, (c) the setup already uses BGE-M3 as embedding – stack consistency, or (d) no USD API cost is wanted.

Concrete cases: a law firm with Swiss clients strictly interpreting GDPR and professional secrecy – self-host is mandatory. A fiduciary with own GPU hardware (e.g. a Hetzner GPX130 with RTX 3060, rented for initial ingestion, then pure CPU inference for re-ranking). An SME with high RAG traffic (1000+ queries/day) where Cohere API cost adds up.

For the typical Swiss fiduciary setup with BGE-M3 as embedding and Qdrant as vector DB, BGE-Reranker is the natural complement. All three components are open-source (Apache 2.0), self-hostable, multilingual, EU-compliant. The entire RAG stack sits in your own data centre or at an EU host – architectural cleanliness that wins in a client pitch.

Combination with OpenAI or Anthropic LLMs is also possible. Embedding via BGE-M3, reranking via BGE-Reranker, LLM generation via GPT-4o or the current top Claude model – embedding and rerank layers self-host, LLM cloud. This markedly lowers the risk profile because question and document text can be filtered before the LLM call.

When not to use

When maximum recall is everything and EU hosting via AWS Bedrock is acceptable, Cohere Rerank is 2-4 points better on MTEB reranking. In high-precision setups (e.g. legal precedent research where the correct ruling may sit between positions 4 and 5) that difference matters.

If your team cannot or will not run GPU inference, BGE-Reranker is slow on CPU – 5-10 seconds per 50 candidates. Too slow for interactive applications. In that case Cohere Rerank (API) or FlashRank (CPU-optimised with smaller models) are better options.

If your corpus is almost entirely English and you want maximum quality, Voyage rerank-2 or Cohere rerank-english-v3.0 is slightly ahead. BGE-Reranker-v2-m3 as a multilingual model is not as English-specialised as English-focused rerankers.

If you rerank very long documents – passages above 4000 tokens – BGE-Reranker hits context limits. The model is limited to 8192 tokens total input (question + document); long documents require chunking before reranking, adding complexity.

Trade-offs

FAQ

Related topics

Sources

1. BAAI bge-reranker-v2-m3 – model card and benchmarks · 2026-05
2. FlagEmbedding GitHub – reference implementation and fine-tuning recipe · 2026-05
3. Chen et al., BGE-M3 paper (covers reranker training) · 2026-04
4. MTEB Leaderboard – reranking sub-track · 2026-05