Question 1

How much VRAM does EMO need?

Accepted Answer

EMO (13.6B) requires approximately 30.328276062011717 GB of VRAM at full precision (FP16), 15.2 GB at 8-bit quantization (Q8), and 7.6 GB at 4-bit quantization (Q4). The actual VRAM usage may be slightly higher due to KV cache and runtime overhead.

Question 2

What GPUs can run EMO?

Accepted Answer

We have 10 compatible GPUs indexed for EMO, including A40, L40S, L40, A100 40GB, RTX A6000 and 5 more. Compatibility is determined by whether the GPU's VRAM can fit the model's memory footprint, potentially using quantization to reduce memory usage.

Question 3

What is the cheapest GPU to run EMO?

Accepted Answer

The cheapest GPU to run EMO in the cloud is the A40 at $0.44/hr. Other affordable options include L40S ($0.60/hr) and L40 ($0.82/hr). Prices reflect the lowest on-demand rate across cloud providers.

Question 4

Can EMO run on consumer hardware?

Accepted Answer

Yes, EMO can run on consumer GPUs like the RTX A6000 (48 GB VRAM) at full precision (FP16).

Question 5

What quantization options are available for EMO?

Accepted Answer

EMO supports the following quantization levels: FP16 (full precision): ~30.328276062011717 GB, Q8 (8-bit): ~15.2 GB, Q6 (6-bit): ~11.4 GB, Q4 (4-bit): ~7.6 GB. Lower quantization reduces VRAM usage at the cost of some quality loss. Q8 offers a good balance between quality and memory savings, while Q4 provides maximum compression. As a Mixture of Experts (MoE) model with 128 experts (8 active), EMO has a larger total weight size but only activates a subset of parameters per token, which can affect effective memory usage during inference.

GPU	VRAM	Best Quant	From	Rent
A40	48 GB	FP16	$0.44/hr	Rent
L40S	48 GB	FP16	$0.60/hr	Rent
L40	48 GB	FP16	$0.82/hr	Rent
A100 40GB	40 GB	FP16	$0.93/hr	Rent
RTX A6000	48 GB	FP16	$1.09/hr
A100 80GB PCIe	80 GB	FP16	$1.39/hr	Rent
A100 80GB	80 GB	FP16	$1.39/hr	Rent
H100 MEGA	80 GB	FP16	$1.80/hr	Rent
H100 SXM	80 GB	FP16	$1.80/hr	Rent
H100 NVL	94 GB	FP16	$1.80/hr	Rent

EMO

VRAM Requirements

Compatible GPUs

EMO FAQ

VRAM requirements for EMO at different quantization levels
Quantization	VRAM Required
FP16	30.3 GB
Q8	15.2 GB
Q6	11.4 GB
Q4	7.6 GB