All tags
#Per‑Tensor Symmetric Quantization
Covers symmetric quantization where a single scale factor is applied per tensor, mapping signed integer ranges evenly around zero for efficient inference.
0
Models
0
Providers
1
Articles
Top articles
View all 1
AI ModelsRunning a 744‑Billion‑Parameter GLM‑5.2 Model on 25 GB RAM: Int4 Quantization, Expert Streaming, and a Single‑Machine C Server
This article walks through the end‑to‑end pipeline for compressing the GLM‑5.2 744B mixture‑of‑experts model to 4‑bit integers, streaming only the active experts from disk, and serving inference from a single commodity server with 25 GB of RAM using a lightweight C runtime.