Cost-optimized inference server with 4x NVIDIA L4 24GB GPUs — perfect for serving Llama 3 8B, Mistral, and embedding models at scale. Xeon Gold 5416S with 256GB ECC DDR5 and 4x1.92TB NVMe. Each L4 consumes only 72W for high efficiency. Supports vLLM, TensorRT-LLM, and Triton Inference Server.
| Processor | Intel Xeon Gold 5416S (2.0 GHz, 30MB) |
| Generation | 4th Gen Sapphire Rapids |
| Cores / Threads | 16 cores / 32 threads |
| Graphics | 4× NVIDIA L4 24GB GDDR6 (96GB VRAM) |
| RAM Installed | 256GB DDR5-4800 ECC |
| RAM Type | DDR5 ECC Registered |
| RAM Slots | 32 DIMM slots |
| Max RAM | 8TB |
| Storage | 4× 1.92TB NVMe SSD |
| Storage Type | PCIe 5.0 NVMe |
| Max Storage | 8× 2.5" NVMe bays |
| Operating System | Ubuntu 22.04 + vLLM / TensorRT-LLM |
| Weight | 32 kg |
| Ports | 4× 25GbE · iDRAC · 6× USB |
| Form Factor | 2U Custom Server |
| Coverage | 3 Years Onsite |
No reviews yet. Be the first to review this product!
$89,900 $99,900
$15,900 $18,400
$38,900 $44,900
$21,900 $24,900
