[Help Wanted] Refactor matching logic

[kylesayrs]

• There currently exist 17 different functions for matching modules/parameters. Let's consolidate this logic by using only the functions defined in utils/match.py

Function	Used for/by	Maps to
get_layer_by_name	awq	attrgetter(layer_name)(module)
get_layers_params	pruning modifiers	match_named_parameters
get_param	None	Delete
get_params	get_layers_params , get_param	Delete
set_layer	Distillation modifier	Module.set_submodule
get_layer	None	Delete
get_layers	Lots	match_named_modules
match_layers_params	get_layers	Delete
get_default_params	match_layers_params	Delete
match_targets	Many	match_named_modules
get_quantizable_layers	match_layers_params	Delete, use explicit targets
get_prunable_layers	match_layers_params	Delete, use explicit targets
get_terminal_layers	match_layers_params	Delete, use explicit targets
get_matching_layer	Smoothquant	match_modules_set
AWQ.set_resolved_mappings	AWQ	match_modules_set
match_modules	sequential pipeline	match_named_modules
get_linear_layers	infer_sparsity_structure_from_model	match_named_modules

[Etelis]

@dsikka Is this one solved?
if not, could you assign me?
and also tell me what's left?

[kylesayrs]

Hi @Etelis!

It seems like this is still very open (as there are still a few issues that need resolving with #1687. If you're looking for places to contribute, this is a great ticket to look at!

Another great place for contribution is enabling #1382, multiGPU compression. I'll write up a ticket soon (and if this sounds interesting, I'll assign it to you). The TLDR is to take existing algorithms and multithread them to allow compression across GPUs, leading to speedups

[Etelis]

Hi @Etelis!

It seems like this is still very open (as there are still a few issues that need resolving with #1687. If you're looking for places to contribute, this is a great ticket to look at!

Another great place for contribution is enabling #1382, multiGPU compression. I'll write up a ticket soon (and if this sounds interesting, I'll assign it to you). The TLDR is to take existing algorithms and multithread them to allow compression across GPUs, leading to speedups

Hey,
Thanks for the time :)
Fixing: #1687
And also I'd love you to assign me the other one as well!

If you guys have slack / discord or smt I would love to join as well

[kylesayrs]

@Etelis Awesome!

In terms of this issue, feel free to branch or create your own, creating your own might be best. Make sure to take a quick look at the interfaces in match.py so you know what's available

[kylesayrs]

@Etelis In terms of multigpu compression, see the issue I've created for an in-depth explanation of my thoughts on the issue. I'm be curious to hear what you think.

#1809

[kylesayrs]

@Etelis In terms of communications, I've added you to our vLLM post-training channels! Feel free to ping myself or others in the channel if you want to collaborate more in real time :)