deployment

GGUF

A file format for storing quantized machine learning models, designed for efficient local inference using the llama.cpp library. GGUF replaced the older GGML format and supports metadata, multiple tensor types, and various quantization levels.