Skip to content

Inference System · C++

llama.cpp

llama.cpp fork with the fork-ple branch: PLE n-gram table overlay (.pleo) and llama-ple-lens for ENGRAFT (github.com/fulvian/engraft-ngram)

What it is

This repository is a fork of llama.cpp focused on optimizing LLM inference for AMD GPUs via Vulkan, featuring adaptive speculative decoding and support for multiple hardware backends. It provides a complete inference system with CLI, server, and tools for deployment.

Scored at9d9f9f918 Nov 2026Changes since

Merit describes the code at this commit. Later pushes count once the owner rescans. How merit is scored

Distribution based on AI work

1 / 2
NLP & LLMs: 35.2%Machine Learning: 33.3%MLOps: 14.8%Computer Vision: 11.1%Data Science: 3.7%AI Agents: 1.9%
  • NLP & LLMs35%
  • Machine Learning33%
  • MLOps15%
  • Computer Vision11%
  • Data Science4%
  • AI Agents2%

Each field's share of the project's AI work, as scoring counts it. It ranks in the field it is most about.

What the judge found

Read from the project's code at its scored commit.

Strengths5

  • High performance optimizations for specific hardware (AMD Strix Halo Vulkan).
  • Adaptive speculative decoding that improves throughput.
  • Wide hardware backend support (CUDA, Metal, Vulkan, etc.).
  • Comprehensive tooling (CLI, server, conversion, tuning).
  • Strong reproducibility with CI and tests.

Limitations

The owner keeps this project's limitations private.

Around it on the NLP & LLMs board

Whole board
  1. 1llama.cppInference System · C++80.2/100ExpertFulvio
  2. 2ROCmForgeInference System · Rust77.9/100ExpertLuiz Cesar Spies
  3. 3mini-llm-labAI Developer Tool · Python77.0/100AdvancedMohamed Khayat

More from Fulvio