All tags
#744B
Content tagged 744B covers advanced AI model optimization techniques, focusing on quantization, pruning, and efficient inference for large language models.
0
Models
0
Providers
1
Articles
Top articles
View all 1
AI ModelsRunning a 744‑Billion‑Parameter GLM‑5.2 Model on 25 GB RAM: Int4 Quantization, Expert Streaming, and a Single‑Machine C Server
This article walks through the end‑to‑end pipeline for compressing the GLM‑5.2 744B mixture‑of‑experts model to 4‑bit integers, streaming only the active experts from disk, and serving inference from a single commodity server with 25 GB of RAM using a lightweight C runtime.