The Token-Energy Law of LLM Decode
The Token-Energy Law relates GPU-board energy consumption per output token to active bytes and realized decode time during LLM serving.
The Token-Energy Law relates GPU-board energy consumption per output token to active bytes and realized decode time during LLM serving.