Running large language models on a single GPU for throughput-oriented scenarios.
-
Updated
Oct 28, 2024 - Python
#
high-throughput
Here are 236 public repositories matching this topic...
A thread-safe queue faster and more resource efficient than golang's native channels
go golang optimization concurrency ringbuffer low-latency lock-free fastest high-throughput memory-efficient thread-safe spsc-queue mpsc-queue highly-concurrent zero-allocations zenq
-
Updated
Mar 12, 2024 - Go
[NeurIPS'23] H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models.
-
Updated
Aug 1, 2024 - Python
Extremely Fast End-to-End Deep Multi-Agent Reinforcement Learning Framework on a GPU (JMLR 2022)
reinforcement-learning deep-learning gpu cuda pytorch numba high-throughput multiagent-reinforcement-learning
-
Updated
May 1, 2025 - Python
atomate2 is a library of computational materials science workflows
-
Updated
Oct 29, 2026 - Python
[ICML 2025 Spotlight] ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference
-
Updated
May 1, 2025 - Python
quacc is a flexible platform for computational materials science and quantum chemistry that is built for the big data era.
python workflow database dft chemistry hpc quantum-mechanics quantum-chemistry high-throughput computational-materials-science
-
Updated
Oct 28, 2026 - Python
Open-source red teaming framework for MLLMs with 42+ attack methods
-
Updated
Jul 17, 2026 - Python
High-performance scalable message broker for modern event-driven applications
go golang mqtt iot coap scalable messaging edge devices web-sockets high-throughput message-broker multiprotocol
-
Updated
Oct 26, 2026 - Go
One database for user-user (follows), user-item (likes), and item-item (related items) interactions — precomputed at write time, served as simple lookups
-
Updated
Oct 22, 2026 - Kotlin
Realtime distributed messaging platform built using Go and React (Fullstack) ❤️
-
Updated
Feb 21, 2025 - Go
⚡ Zero-Stall MoE Inference via Lookahead Prediction & Async DMA Prefetching. Optimized for SSD I/O with Hybrid MLA+Sliding Window Attention.
open-source artificial-intelligence lora high-throughput open-models mixture-of-experts llm generative-ai large-language-model streaming-llm predictive-inference sliding-window-attention io-latency-hiding async-dma ssd-offloading lookahead-routing mla-attention dual-layer-moe
-
Updated
Apr 26, 2026 - Python
A snakemake-based workflow for FEP and MM(PB/GB)SA calculations with GROMACS
-
Updated
May 9, 2026 - Python
extremly light uart library for AVR 8 bit microcontrollers
c avr transmission optimized flow-control uart rs232 rs485 avr-gcc prematured usart high-speed low-overhead arduino-less high-throughput avr-microcontroller avr-microcontrollers
-
Updated
Jun 14, 2026 - C
A high-performance, SIMD-optimized, pure C library for reading and writing Apache Parquet files.
-
Updated
Oct 30, 2026 - C
A low-latency LRU approximation cache in C++ using CLOCK second-chance algorithm. Multi level cache too. Up to 2.5 billion lookups per second.
fast caching benchmark memoization cpp high-performance cache lru data-structures low-latency fastest high-throughput lru-cache cache-simulator multi-level-cache low-latency-queries least-recently-used set-associative-cache direct-mapped-cache clock-second-chance
-
Updated
Jan 18, 2024 - C++
⚡ In-Memory Vector Search Engine & HNSW ANN Graph in Java 21 Loom. Sub-millisecond KNN retrieval, 8-bit scalar quantization (SQ8), 16,000+ QPS.
search-engine machine-learning concurrency embeddings ann high-throughput quantization knn in-memory-database similarity-search ai-search hnsw vector-search approximate-nearest-neighbors vector-database scalar-quantization project-loom virtual-threads java-21 vector-index
-
Updated
Oct 30, 2026 - Java
🇮🇳 High-Performance Geospatial Administrative Hierarchy & Proximity Query Engine for India's 650k+ Census Settlements (Java 21 Loom, 2D KD-Tree, 38k+ QPS).
spring-boot geospatial kd-tree gis in-memory reverse-geocoding spatial-index india census-data high-throughput high-concurrency proximity-search fast-search indic spatial-indexing trigram-search spatial-query project-loom virtual-threads java-21
-
Updated
Oct 30, 2026 - Java
An Intuitive, Lightweight, High Performance Full Stack Java Web Framework.
-
Updated
Oct 30, 2026 - Java
🔬🖥 Automated Blood Vasculature Analysis of 3D Light-Sheet Image Volumes
docker bioinformatics snakemake high-throughput graph-analysis vessel-segmentation 3d-imaging 3d-skeleton light-sheet-microscopy
-
Updated
Jun 6, 2024 - Python
Add this topic to your repo
To associate your repository with the high-throughput topic, visit your repo's landing page and select "manage topics."