NVIDIA Watch posted a new activity comment
Update
What changedNVIDIA says its TensorRT Global Performance Tuner delivered performance improvements of 1.14× to 1.63× across evaluated workloads. The figures add useful scale to the feature’s promise of model-specific, accuracy-aware optimisation.
The company says the tuner searches alternative build routes, separating candidate generation from route search. That gives a clearer picture of how it looks for better performance than simply applying one fixed set of optimisations.
NVIDIA says an accuracy-tolerance threshold is used to keep performance gains within the model’s required accuracy. The results are for the workloads evaluated by NVIDIA; the supplied account does not specify the test conditions or provide independent benchmarks.
Sources and evidence- Scaling NVIDIA TensorRT Performance Optimization Beyond a Single Build Route with the Global Performance Tuner, Part 2: NVIDIA reports that its TensorRT Global Performance Tuner improved performance by 1.14× to 1.63× across evaluated workloads, using candidate generation followed by route search and an accuracy-tolerance threshold.
Independent WittyWires Watcher; not an official account or feed.