RЦi \SrSSKrSSKJs Jr SSKJrJr SSK r SSK J r "SS\5r g)a1 SGDP Optimizer Implementation copied from https://github.com/clovaai/AdamP/blob/master/adamp/sgdp.py Paper: `Slowing Down the Weight Norm Increase in Momentum-based Optimizers` - https://arxiv.org/abs/2006.08217 Code: https://github.com/clovaai/AdamP Copyright (c) 2020-present NAVER Corp. MIT license N) Optimizerrequired) projectioncn^\rSrSr\SSSSSSS4U4Sjjr\R"5S Sj5rSr U=r $) SGDPrFg:0yE>g?c L>[UUUUUUUU S9n [[U]X5 g)N)lrmomentum dampening weight_decaynesterovepsdeltawd_ratio)dictsuperr__init__) selfparamsr r r rrrrrdefaults __class__s N/var/www/html/ai-image-ml/venv/lib/python3.13/site-packages/timm/optim/sgdp.pyr SGDP.__init__s8%   dD"64c SnUb%[R"5 U"5nSSS5 URGH#nUSnUSnUSnUSnUSGHnURcMURn URUn [ U 5S:Xa[R "U5U S'U Sn U RU5RU SU- S9 U(aXU --n OU n Sn [ UR5S :a[XXS US US 5upUS:wa&URSUS US-U -S U- - - 5 URXS *S9 GM GM& U$!,(df  GNE=f)Nrr r rrrg?)alpharrrrr ) torch enable_grad param_groupsgradstatelen zeros_likemul_add_shaper)rclosurelossgrouprr r rpr"r#bufd_prs rstep SGDP.step,s  ""$y%&&E 0LZ(Hk*IZ(H8_66>vv 1 u:?(-(8(8(;E*%J'"''BN'CC/CCqwwrAs, 5 D9Dr