Inference System · Rust
ROCmForge
Local AMD GPU LLM inference engine (HIP/ROCm) — quantized GGUF, Q4_0/K-quants, flash attention, KV cache. ~515 tok/s decode on Qwen2.5-0.5B Q4_0 (RX 7900 XT).
What it is
ROCmForge is an LLM inference engine for AMD GPUs that implements custom HIP kernels for operations like attention, quantization, and RoPE, featuring a GPU Decode Graph to reduce kernel launch overhead and an inference path router for dynamic optimization.
- amd
- gguf
- gpu
- hip
- inference
- llm
- local-llm
- quantization
- rocm
- rust
Where it counts
- NLP & LLMs95%
- Machine Learning90%
- MLOps20%
- Computer Vision10%
- Data Science10%
How much of the project is about each field, as scoring counts it. It ranks in the field it is most about.
