AI GGUF - LLM model file parser in C#

This GGUF file parser was written by Gyula Rábai for parsing and inspecting GGUF (GPT-Generated Unified Format) files commonly used for storing large language models. The parser can properly understand different data formats including qunatizatoin used in local AI model execution.

Download (Exe): GGUFParser.zip
Download (Source): GGUFParser.zip

Screenshots

Overview

This project provides a comprehensive GGUF file parser with a user-friendly console interface. It allows you to:

• Scan directories for GGUF files
• Load and inspect GGUF model files
• View file metadata and tensor information
• Support for various quantization formats

Features

• Directory Scanning: Recursively search for .gguf files in any directory
• File Inspection: Load and display detailed information about GGUF files
• Metadata Display: View key model metadata including:
  • Architecture (e.g., llama, mistral, etc.)
  • Model name
  • Context length
  • Embedding length
  • Attention heads
  • Tokenizer information
• Tensor Information: View tensor names, types, and dimensions
• Multi-format Support: Handles various GGUF file versions and quantization formats

Requirements

• .NET 8.0 SDK or later
• Windows, Linux, or macOS

Building

cd GGUFParser/GGUFParser
dotnet build

Running

dotnet run

The default directory for scanning is C:\AIModels. You can change this through the menu.

Usage

The application provides an interactive console menu:

1. Select Directory - Choose a directory to scan for GGUF files
2. View GGUF Files - List all found GGUF files in the selected directory
3. Load and Inspect GGUF File - Load a specific GGUF file and view its details
4. Exit - Close the application

Project Structure

GGUFParser/
├── GGUFParser.sln
├── GGUFParser/
│   ├── Program.cs              # Main application entry point
│   ├── GGUFParser.csproj       # Project file
│   ├── AIMath/                 # AI math operations and types
│   ├── AINum/                  # Numeric type implementations
│   ├── GGUFFile/               # GGUF file parsing
│   └── Matrix/                 # Matrix operations
        

Key Components

GGUFFile

The core module for parsing GGUF files:

• OzGGUFFile - Main file handler
• OzGGUF_Tensor - Tensor representation
• OzGGUF_MD - Metadata handling

Numeric Types

Comprehensive support for different numeric formats:

• Floating point: Float16, Float32, Float64, BFloat16
• Integer: Int8, Int16, Int32, Int64
• Unsigned: UInt8, UInt16, UInt32, UInt64
• Quantized: Various IQ, KQ, RQ formats