Skip to content

Commit 7e5acba

Browse files
committed
[TRTLLM-4501] fit: AutoTuner tuning config refactor and add tuning for kernel configs.
Adding a config entry in the tuning config to define the valid candidates for each part of the configs. * AutoTuner will loop over a search grid generated from the config combinations. * Each config will be tuned along with the specific input profile. * The best config will be recorded in the cache value (instead of the cache key). And it will be recovered and used in the tunable runner forward. Other enhancements: * Use the decorator to make the tuning config definition more natural and efficient. This is an independent enhancement. * Allow the user to not speficy the gen_tuning_buckets or the map_to_tuning_buckets function. * Code refactoring. Signed-off-by: Yukun He <[email protected]>
1 parent 5bc3a15 commit 7e5acba

File tree

6 files changed

+332
-154
lines changed

6 files changed

+332
-154
lines changed

0 commit comments

Comments
 (0)