How to Maintain AI SLAs While Making Data Center Load Flexible
Short answer
Keep SLAs intact by applying a different flexibility method to each workload type. Training has no SLA to protect, so it can be throttled or curtailed. Inference is response-time sensitive, so it is routed to another data center instead of being curtailed. This is how LōD Flexible Compute approaches grid flexibility.
The core principle: match the method to the workload
Treating all AI workloads the same leaves flexibility on the table or puts SLAs at risk. The right approach depends on whether a workload can tolerate delay or reduced throughput.
Method by workload type
- Training: throttle or curtail freely, managed at the GPU level through NVIDIA DSX interconnection
- Inference: distribute across data centers in real time with CEDN, with no impact on response time
- Site assets: automate batteries and generators through LōD IoT for additional flexibility
Questions to answer before enabling flexibility
- Which workloads carry a response-time SLA, and which do not?
- Is there inference capacity at a second site to route to?
- Which on-site assets, such as batteries or generators, can contribute?
Limitations and scope
- Contractual SLAs differ by customer; workload classification should be agreed with the operator before flexibility is enabled.
- Routing depends on available capacity at the destination site.
- This page describes the approach, not a guarantee for any specific deployment.