Error when pass norm_name = 'layer' for swin_unetr

[DiDiDD]

Hello,

I wanted to apply layer norm for model swin_unetr by setting norm_name='layer', but I got errors TypeError: LayerNorm.init() missing 1 required positional argument: 'normalized_shape'.

But i got no problems when applying batch norm and i just set norm_name='batch'.

model_su = SwinUNETR(img_size = (512, 512), in_channels=3, out_channels=3,
depths=depths,
num_heads=(3, 6, 12, 24),
feature_size=12,
norm_name= norm_name,
drop_rate=0.0,
attn_drop_rate=0.0,
dropout_path_rate=0.0,
normalize=True,
use_checkpoint=False,
spatial_dims=2,
downsample='merging')

Can someone help me, please? Many thanks!

[tangy5]

Emmm, Did you use the latest MONAI repo? You can try update the dependencies. The layer norm should work. Thanks for the interests of the work.

[DiDiDD]

Thank you for the reply! I've tried to run with updated dependencies now (which I followed the steps on monai's web), but it still pops the same error. May I ask if swin_unetr with layer norm can run on your side, please? Many thanks!

[tangy5]

Yes, the Layer norm work all good, we also have the unit test and integration tests covers the SwinUNETR network in MONAI repo.

I guess the Pytorch version possibly matters, you could do a new env and install the monai with pytorch.

[MiguelMonteiro]

I got the same error using monai 1.3.0 and pytorch 2.0.1 when attempting to use DenseNet. It seems layer norm needs to know the shape of the input and nowhere in the norm layer factory code is this passed/inferred.
Simple code to reproduce

from monai.networks.nets.densenet import densenet121
model = densenet121(spatial_dims=3, in_channels=1, out_channels=2048, norm="layer")