gddlZddlmZGddejZGddej ZGddeZy)NcPeZdZdejdejffd ZxZS) LayerNorm32xreturncpt||jj|jSNsuperforwardfloattypedtypeselfr __class__s 3G:\Projects\Beeuty-MapMaker\trellis\modules\norm.pyr zLayerNorm32.forward'wqwwy)..qww77__name__ __module__ __qualname__torchTensorr __classcell__rs@rrrs#88%,,88rrcTeZdZdZdej dej ffd ZxZS) GroupNorm32zM A GroupNorm layer that converts to float32 before the forward pass. rrcpt||jj|jSrr rs rr zGroupNorm32.forwardrr)rrr__doc__rrr rrs@rrr s(88%,,88rrcPeZdZdejdejffd ZxZS)ChannelLayerNorm32rrc|j}|jdgtd|dj}t||}|jd|dz gtd|dz j}|S)Nr)dimpermuterange contiguousr r )rrDIMrs rr zChannelLayerNorm32.forwardsxeeg AIIa +%3- + + 6 6 8 GOA  AIIaQ 1q#a% 1 < < >rrrs@rr"r"s#%,,rr")rtorch.nnnn LayerNormr GroupNormrr"rrr0s7 8",,8 8",,8r