RЦi!SrSSKJrJrJr SSKrSSKJr SSKJs J r SSK J r SSK JrJrJr "SS\R"5r\ "S S \R"55r"S S \R"5r\ "S S\R"55rg)aConvolution with Weight Standardization (StdConv and ScaledStdConv) StdConv: @article{weightstandardization, author = {Siyuan Qiao and Huiyu Wang and Chenxi Liu and Wei Shen and Alan Yuille}, title = {Weight Standardization}, journal = {arXiv preprint arXiv:1903.10520}, year = {2019}, } Code: https://github.com/joe-siyuan-qiao/WeightStandardization ScaledStdConv: Paper: `Characterizing signal propagation to close the performance gap in unnormalized ResNets` - https://arxiv.org/abs/2101.08692 Official Deepmind JAX code: https://github.com/deepmind/deepmind-research/tree/master/nfnets Hacked together by / copyright Ross Wightman, 2021. )OptionalTupleUnionN)register_notrace_module) get_paddingget_padding_valuepad_samec^\rSrSrSrSS\S\S\\\\\44S\\\\\44S\\\\\\44S\\\\\44S \S \ S \ 4U4S jjjr S r Sr U=r$) StdConv2dzConv2d with Weight Standardization. Used for BiT ResNet-V2 models. Paper: `Micro-Batch Training with Batch-Channel Normalization and Weight Standardization` - https://arxiv.org/abs/1903.10520v2 in_channel out_channels kernel_sizestridepaddingdilationgroupsbiasepsc R>Uc [X4U5n[T U] XX4XVXxXS9 Xlg)Nrrrrrdevicedtype)rsuper__init__r) selfrrrrrrrrrrr __class__s S/var/www/html/ai-image-ml/venv/lib/python3.13/site-packages/timm/layers/std_conv.pyrStdConv2d.__init__#s= ?!+x@G  kvQW  fc b[R"URRSURS5SSSSUR S9R UR5n[R"XURURURURUR5nU$NrT)trainingmomentumr) F batch_normweightreshaperr reshape_asconv2drrrrrrxr*s rforwardStdConv2d.forward8s KK  4#4#4b 9     *T[[ !  HHQ 4;; dmmUYU`U` ar!)r)rNrrFư>NN)__name__ __module__ __qualname____firstlineno____doc__intrrrboolfloatrr0__static_attributes__ __classcell__rs@rr r s34=A45sE#s(O34  #uS#X./  eCsCx$89: CsCx01*  r!r c^\rSrSrSrSS\S\S\\\\\44S\\\\\44S\S\\\\\44S \S \ S \ 4U4S jjjr S r Sr U=r$) StdConv2dSameEzConv2d with Weight Standardization. TF compatible SAME padding. Used for ViT Hybrid model. Paper: `Micro-Batch Training with Batch-Channel Normalization and Weight Standardization` - https://arxiv.org/abs/1903.10520v2 rrrrrrrrrc X>[XSXFS9up\[T U] XX4XVXxXS9 XlXlg)Nrrr)r rrsame_padr)rrrrrrrrrrrr is_dynamicrs rrStdConv2dSame.__init__Ls?0Vg  k'V  B# r!c UR(a+[XRURUR5n[ R "URRSURS5SSSSURS9RUR5n[ R"XURURURURUR5nU$r#)rCr rrrr(r)r*r+rrr,r-rrrr.s rr0StdConv2dSame.forwardas ==,,dkk4==IA KK  4#4#4b 9     *T[[ !  HHQ 4;; dmmUYU`U` ar!)rrC)rSAMErrFr2NN)r3r4r5r6r7r8rrstrr9r:rr0r;r<r=s@rr?r?Es34!45sE#s(O34  #uS#X./   CsCx01*  r!r?c^\rSrSrSrSS\S\S\\\\\44S\\\\\44S\\\\\\4\ 4S\\\\\44S \S \ S \ S \ S \ 4U4Sjjjr SU4Sjjr SrSrU=r$)ScaledStdConv2dpa$Conv2d layer with Scaled Weight Standardization. Paper: `Characterizing signal propagation to close the performance gap in unnormalized ResNets` - https://arxiv.org/abs/2101.08692 NOTE: the operations used in this impl differ slightly from the DeepMind Haiku impl. The impact is minor. in_channelsrrrrrrrgammar gain_initc Z>XS.nUc [X4U5n[TU]"XU4XEUXxS.UD6 XRSR 5S--UlXlXl[R"[R"URSSS440UD65Ul UR5 g)Nrrrrrrrrr)rrrr*numelscalerrOnn Parametertorchemptyrgainreset_parameters)rrMrrrrrrrrNrrOrrddrs rrScaledStdConv2d.__init__ys  / ?!+x@G  { ,;A]e ,(* ,[[^113t;; "LLd.?.?Aq-I!PR!PQ  r!c>[US5(aM[RRR UR UR 5 [TU]!5 ggNrZ hasattrrXrVinit constant_rZrOrr[rrs rr[ ScaledStdConv2d.reset_parameters@ 4 HHMM # #DIIt~~ > G $ & !r!c [R"URRSURS5SSUR UR -RS5SSURS9RUR5n[R"XURURURURUR5$Nrr$Tr%)r*r&r'r)r(r)r*r+rrZrUviewrr,r-rrrrrr.s rr0ScaledStdConv2d.forwards KK  4#4#4b 9  II *004  *T[[ ! xx499dkk4<<X\XcXcddr!)rrZrOrU) rNrrT?r2rkNNreturnN)r3r4r5r6r7r8rrrrIr9r:rr[r0r;r<r=s@rrKrKps34BF45"   sE#s(O34  #uS#X./  eCsCx#$=>?  CsCx01       <' e er!rKc^\rSrSrSrSS\S\S\\\\\44S\\\\\44S\S\\\\\44S \S \ S \ S \ S \ 4U4Sjjjr SU4Sjjr Sr SrU=r$)ScaledStdConv2dSameaLConv2d layer with Scaled Weight Standardization and Tensorflow-like SAME padding support Paper: `Characterizing signal propagation to close the performance gap in unnormalized ResNets` - https://arxiv.org/abs/2101.08692 NOTE: the operations used in this impl differ slightly from the DeepMind Haiku impl. The impact is minor. rMrrrrrrrrNrrOc `>XS.n[XSXFS9up_[TU]"XU4XEUXxS.UD6 XRSR 5S--UlXlXlXl[R"[R"URSSS440UD65UlUR5 g)NrQrBrRrrSr)r rrr*rTrUrCrrOrVrWrXrYrrZr[)rrMrrrrrrrrNrrOrrr\rDrs rrScaledStdConv2dSame.__init__s  //Vg  { ,;A]e ,(* ,[[^113t;; " "LLd.?.?Aq-I!PR!PQ  r!c>[US5(aM[RRR UR UR 5 [TU]!5 ggr_r`rds rr[$ScaledStdConv2dSame.reset_parametersrfr!c $UR(a+[XRURUR5n[ R "URRSURS5SSURUR-RS5SSURS9RUR5n[ R"XUR URUR"URUR$5$rh)rCr rrrr(r)r*r+rrZrUrirr,r-rrrr.s rr0ScaledStdConv2dSame.forwards ==,,dkk4==IA KK  4#4#4b 9  II *004  *T[[ ! xx499dkk4<<X\XcXcddr!)rrZrOrCrU) rrHrrTrkr2rkNNrl)r3r4r5r6r7r8rrrIr9r:rr[r0r;r<r=s@rroros34!45"   sE#s(O34  #uS#X./    CsCx01       <' e er!ro)r7typingrrrrXtorch.nnrVtorch.nn.functional functionalr(_fxrrrr r Conv2dr r?rKror!rr~s$*) (==% %P'BII''T9ebii9ex;e"));e;er!