# Quantization

Shrinking a model (for example to 4-bit numbers) so it runs on smaller hardware, with a small quality loss.

AI developers · Part 07: Hugging Face · Card 11 of 12 · Last updated 2026-10-13

## Analogy

Compressing images to WebP: much smaller, and most people can't tell.

## In an agent

Lets you run open models locally or on cheap servers for narrow agent tasks.

## Practice

### Flashcards

- **What is quantization?** Shrinking a model so it runs on smaller hardware, with a small quality loss.

## Questions people ask

### What is model quantization?

Quantisation shrinks a model by storing its numbers with fewer bits, for example 4-bit instead of 16-bit. The model needs much less memory and runs on smaller hardware, with a small loss in quality. It works like compressing images to WebP: much smaller, and most people cannot tell.

### When should I use a quantized model?

When you want to run an open model on a laptop, a cheap server or limited GPU memory, especially for narrow tasks such as classification or simple extraction. Test it on your own examples first: for hard reasoning, the quality loss can matter more than the memory saved.

## Related concepts

- [The Hugging Face Hub](https://studyvisually.com/learn/tech/ai-developers/hugging-face/hugging-face-hub)
- [Transformers.js](https://studyvisually.com/learn/tech/ai-developers/hugging-face/transformers-js)
- [pipeline()](https://studyvisually.com/learn/tech/ai-developers/hugging-face/pipeline)
- [Local embeddings](https://studyvisually.com/learn/tech/ai-developers/hugging-face/local-embeddings)

Web page: https://studyvisually.com/learn/tech/ai-developers/hugging-face/quantization
Study this card in the app: https://studyvisually.com/app/learn?category=ai-developers&card=quantization