Question 1

How much VRAM does bloom need?

Accepted Answer

bloom (176.2B) requires approximately 393.94335021972654 GB of VRAM at full precision (FP16), 197 GB at 8-bit quantization (Q8), and 98.5 GB at 4-bit quantization (Q4). The actual VRAM usage may be slightly higher due to KV cache and runtime overhead.

Question 2

What GPUs can run bloom?

Accepted Answer

We currently have no compatible GPUs indexed for bloom. This may be because the model's VRAM requirements exceed available GPUs in our database, or VRAM data hasn't been computed yet.

Question 3

What is the cheapest GPU to run bloom?

Accepted Answer

We don't currently have cloud pricing data for GPUs compatible with bloom. Cloud pricing data is updated regularly — check back soon.

Question 4

Can bloom run on consumer hardware?

Accepted Answer

bloom requires approximately 393.94335021972654 GB of VRAM at FP16, and even with 4-bit quantization (Q4) it needs about 98.5 GB. This exceeds the 24 GB available on the highest-end consumer GPU (RTX 4090), so bloom requires datacenter or cloud GPUs.

Question 5

What quantization options are available for bloom?

Accepted Answer

bloom supports the following quantization levels: FP16 (full precision): ~393.94335021972654 GB, Q8 (8-bit): ~197 GB, Q6 (6-bit): ~147.7 GB, Q4 (4-bit): ~98.5 GB. Lower quantization reduces VRAM usage at the cost of some quality loss. Q8 offers a good balance between quality and memory savings, while Q4 provides maximum compression.

bloom

VRAM Requirements

Compatible GPUs

bloom FAQ

VRAM requirements for bloom at different quantization levels
Quantization	VRAM Required
FP16	393.9 GB
Q8	197 GB
Q6	147.7 GB
Q4	98.5 GB