Skip to content

Conversation

HolyWu
Copy link
Contributor

@HolyWu HolyWu commented Nov 4, 2024

Description

Using INormalizationLayer has higher performance than the break-down implementation. Also resolved the issue mentioned by @cehongwang at #3014 (comment).

Break-down implementation:
Min=8.936863899230957 ms, Mean=9.042274608612061 ms, Max=9.334688186645508 ms

INormalizationLayer:
Min=7.136256217956543 ms, Mean=7.218872976303101 ms, Max=7.480256080627441 ms

Checklist:

  • My code follows the style guidelines of this project (You can use the linters)
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas and hacks
  • I have made corresponding changes to the documentation
  • I have added tests to verify my fix or my feature
  • New and existing unit tests pass locally with my changes
  • I have added the relevant labels to my PR in so that relevant reviewers are notified

@github-actions github-actions bot added component: tests Issues re: Tests component: conversion Issues re: Conversion stage component: converters Issues re: Specific op converters component: api [Python] Issues re: Python API component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths labels Nov 4, 2024
@github-actions github-actions bot requested a review from zewenli98 November 4, 2024 11:56
@HolyWu HolyWu changed the title Use INormalizationLayer layer for GroupNorm Use INormalizationLayer for GroupNorm Nov 4, 2024
@HolyWu HolyWu mentioned this pull request Nov 11, 2024
7 tasks
Copy link
Collaborator

@zewenli98 zewenli98 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI error seems not related to the PR. This PR LGTM

@zewenli98
Copy link
Collaborator

@HolyWu I think this PR is almost good to go. Can you rebase/merge again? thanks

@zewenli98 zewenli98 merged commit 38b1804 into pytorch:main Dec 12, 2024
63 of 68 checks passed
@HolyWu HolyWu deleted the group_norm_layer branch December 12, 2024 23:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla signed component: api [Python] Issues re: Python API component: conversion Issues re: Conversion stage component: converters Issues re: Specific op converters component: dynamo Issues relating to the `torch.compile` or `torch._dynamo.export` paths component: tests Issues re: Tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants