text-embeddings-inference

EmbedCrate: self-hosted text embeddings and reranking server powered by Hugging Face TEI. OpenAI-compatible embeddings API. Supports BGE, Nomic, MiniLM, and other models. Optional cross-encoder reranking, API-key auth, model management, offline/air-gapped operation with cached models, and persistent model cache. Docker on CPU (amd64/arm64).

Repository of the project Efficient Edge Embeddings (E*3) project, subgrant 2dAI2OC07 from EU Horizon dAIEDGE

Docker Compose stack for scalable TEI embeddings (multi-GPU) fronted by a FastAPI proxy with a Qdrant cache. 🐳⛓️💾

RAG with hybrid search (BGE-M3 + Qdrant RRF), LangGraph agent via AG-UI protocol, CopilotKit chat frontend. 7 services, one docker compose up.

Text Embeddings Inference (TEI)'s unofficial python wrapper library for batch processing with asyncio

It will automatically batch inference requests from independent users in a single batch request for efficiency, so that for users the interface looks like individual requests, but internally it is handled as a batch request

Run HuggingFace TEI (Text Embeddings Inference) locally on NVIDIA Jetson Orin with GPU acceleration — embedding & reranker at 5-30ms latency.

An enterprise-grade RAG Chatbot built with AI-assisted development. Features local LLMs (Ollama), LlamaIndex integration, and Entra ID SSO. Designed for secure, air-gapped PDF analysis. 📄

Local inference engine

Same-host text embedding API for Russian/legal workloads: OpenAI-compatible /v1/embeddings, TEI backend, two-tier L1+Redis cache, Prometheus metrics.

Embeddings-backed semantic memory provider for opencode-hermes (GPU via text-embeddings-inference).