Search results

gguf-py : add Numpy MXFP4 de/quantization support

ggmlchanges relating to the ggml tensor library for machine learningchanges relating to the ggml tensor library for machine learning

pythonpython script changespython script changes

Tensor Encoding Schemehttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemeshttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemes

Status: Merged (completed).

ggml-org/llama.cppnumber 15111

#15111 In ggml-org/llama.cpp;

· by compilade was closed on Aug 8, 2025

ggml-quants : weighted rounding algorithms with cumulative search

generation qualityQuality of model outputQuality of model output

ggmlchanges relating to the ggml tensor library for machine learningchanges relating to the ggml tensor library for machine learning

Less than 4 bitsEfforts related to viable quantized models using <4 bitsEfforts related to viable quantized models using <4 bits

research 🔬

Review Complexity : MediumGenerally require more time to grok but manageable by beginner to medium expertise levelGenerally require more time to grok but manageable by beginner to medium expertise level

Tensor Encoding Schemehttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemeshttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemes

Status: Draft (not ready).

ggml-org/llama.cppnumber 12557

#12557 In ggml-org/llama.cpp;

· compilade opened on Mar 25, 2025

Introduce New Lookup-Table(LUT)-Based Matrix Multiplication Method

ggmlchanges relating to the ggml tensor library for machine learningchanges relating to the ggml tensor library for machine learning

pythonpython script changespython script changes

Tensor Encoding Schemehttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemeshttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemes

Status: Open (in progress).

ggml-org/llama.cppnumber 10181

#10181 In ggml-org/llama.cpp;

· QingtaoLi1 opened on Nov 5, 2024

add FP8 support to gguf/llama:

buildCompilation issuesCompilation issues

examples

ggmlchanges relating to the ggml tensor library for machine learningchanges relating to the ggml tensor library for machine learning

scriptScript relatedScript related

Tensor Encoding Schemehttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemeshttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemes

testingEverything test relatedEverything test related

Status: Draft (not ready).

ggml-org/llama.cppnumber 10055

#10055 In ggml-org/llama.cpp;

· Djip007 opened on Oct 26, 2024

gguf-py : Numpy dequantization for most types

enhancementNew feature or requestNew feature or request

merge readyindicates that this may be ready to merge soon and is just holding out in case of objectionsindicates that this may be ready to merge soon and is just holding out in case of objections

pythonpython script changespython script changes

Review Complexity : MediumGenerally require more time to grok but manageable by beginner to medium expertise levelGenerally require more time to grok but manageable by beginner to medium expertise level

Tensor Encoding Schemehttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemeshttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemes

Status: Merged (completed).

ggml-org/llama.cppnumber 8939

#8939 In ggml-org/llama.cpp;

· by compilade was closed on Aug 11, 2024

ggml-quants : ternary packing for TriLMs and BitNet b1.58

enhancementNew feature or requestNew feature or request

examples

ggmlchanges relating to the ggml tensor library for machine learningchanges relating to the ggml tensor library for machine learning

merge readyindicates that this may be ready to merge soon and is just holding out in case of objectionsindicates that this may be ready to merge soon and is just holding out in case of objections

pythonpython script changespython script changes

Review Complexity : HighGenerally require indepth knowledge of LLMs or GPUsGenerally require indepth knowledge of LLMs or GPUs

Tensor Encoding Schemehttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemeshttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemes

testingEverything test relatedEverything test related

Status: Merged (completed).

ggml-org/llama.cppnumber 8151

#8151 In ggml-org/llama.cpp;

· by compilade was closed on Sep 6, 2024

Add support for BitnetForCausalLM (new model / new datatype)

examples

ggmlchanges relating to the ggml tensor library for machine learningchanges relating to the ggml tensor library for machine learning

pythonpython script changespython script changes

Tensor Encoding Schemehttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemeshttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemes

Status: Merged (completed).

ggml-org/llama.cppnumber 7931

#7931 In ggml-org/llama.cpp;

· by Eddie-Wang1120 was closed on Jun 23, 2024

Introduce Q8_0 and Q4_0 with Bf16 delta values

examples

ggmlchanges relating to the ggml tensor library for machine learningchanges relating to the ggml tensor library for machine learning

pythonpython script changespython script changes

Review Complexity : HighGenerally require indepth knowledge of LLMs or GPUsGenerally require indepth knowledge of LLMs or GPUs

Tensor Encoding Schemehttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemeshttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemes

Status: Open (in progress).

ggml-org/llama.cppnumber 7497

#7497 In ggml-org/llama.cpp;

· Srihari-mcw opened on May 23, 2024

Introduce bfloat16 support

Review Complexity : HighGenerally require indepth knowledge of LLMs or GPUsGenerally require indepth knowledge of LLMs or GPUs

Tensor Encoding Schemehttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemeshttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemes

Status: Merged (completed).

ggml-org/llama.cppnumber 6412

#6412 In ggml-org/llama.cpp;

· by jart was closed on May 8, 2024

IQ1_M: 1.75 bpw quantization

Review Complexity : HighGenerally require indepth knowledge of LLMs or GPUsGenerally require indepth knowledge of LLMs or GPUs

Tensor Encoding Schemehttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemeshttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemes

Status: Merged (completed).

ggml-org/llama.cppnumber 6302

#6302 In ggml-org/llama.cpp;

· by ikawrakow was closed on Mar 26, 2024

gguf : add support for I64 and F64 arrays

Review Complexity : HighGenerally require indepth knowledge of LLMs or GPUsGenerally require indepth knowledge of LLMs or GPUs

Tensor Encoding Schemehttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemeshttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemes

Status: Merged (completed).

ggml-org/llama.cppnumber 6062

#6062 In ggml-org/llama.cpp;

· by certik was closed on Mar 15, 2024

ggml : designate enum vals for integer types

Review Complexity : LowTrivial changes to code that most beginner devs (or those who want a break) can tackle. e.g. UI fixTrivial changes to code that most beginner devs (or those who want a break) can tackle. e.g. UI fix

Tensor Encoding Schemehttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemeshttps://github.com/ggerganov/llama.cpp/wiki/Tensor-Encoding-Schemes

Status: Merged (completed).

ggml-org/llama.cppnumber 6050

#6050 In ggml-org/llama.cpp;

· by ggerganov was closed on Mar 14, 2024