Inference 70B L40 Ai server RTX 4090 Server

Inference 70B L40 Ai server RTX 4090 ServerThis product listing is kept for reference only. This server has been replaced by the new K AI product line. For the current equivalent or an upgraded configuration, please see our AI Servers collection. Recommended replacement: K AI 192 Rome L40 1448TOPS (4x NVIDIA L40, same platform, updated build) 70B L40 Computer Specifications GPU: 6x NVIDIA L40 (288 GB VRAM total) Motherboard: ASRock Rack ROMED8 2T CPU: AMD EPYC 7542 RAM: 512GB SK Hynix 2666MHz

Hydrocooling asic

Mid-tier model R&D bench — rapid iteration on 30-70B fine-tunes

it provides a solid balance between power and profitability

No NVLink on GeForce 5090 — tensor-parallel 2-way P2P uses PCIe

VRAM GDDR7

Parameters Default hashrate mode: 5TH/S±5% Default hashrate Power: 400W±5% Low Power Mode: 3

Measured 19-22 GB/s inter-GPU peer-to-peer on 4-GPU bench

1 T2V/I2V 14B at Q4-Q6 (~16 GB)

Note: Specific miner capacities and power consumption details are available upon request

GPU draw: 8 x 450 W = 3 600 W

4GB CPU cooler: AMD Output: 310 Mh/s

Consumer 4090 has no NVLink