RЦiu SrSSKJr SSKJr SSKJrJrJrJ r J r SSK r SSK J r SSKJ s Jr SSK Jr SSKJr SSKJrJr SS KJrJrJrJrJrJrJrJrJ r S S K!J"r" S S K#J$r$ S S K%J&r&J'r' S SK(J)r)J*r* S/r+"SS\ RX5r-"SS\ RX5r."SS\ RX5r/"SS\ RX5r0"SS\ RX5r1"SS\ RX5r2"SS\5r3"SS\5r4"S S!\ R:5r5"S"S#\ RX5r6"S$S%\ RX5r7"S&S'\ RX5r8"S(S)\ RX5r9"S*S+\ RX5r:"S,S\ RX5r;S-r\)"0S0\>"S1S2S39_S4\>"S1S2S39_S5\>"S1S2S39_S6\>"S1S7S39_S8\>"S1S7S39_S9\>"S1S:9_S;\>"S1S:9_S<\>"S1S:9_S=\>"S1S:9_S>\>"S1S:9_S?\>"S1S@SA9_SB\>"S1S@SCSDSE9_SF\>"S1S@SA9_SG\>"S1S@SCSDSE9_SH\>"S1S@SISJ9_SK\>"S1S@SA9_SL\>"S1S@SCSDSE9_0SM\>"S1S@SA9_SN\>"S1S@SCSDSE9_SO\>"S1S@SISJ9_SP\>"S1S@SA9_SQ\>"S1S@SCSDSE9_SR\>"S1S@SA9_SS\>"S1S@SCSDSE9_ST\>"S1S@SISJ9_SU\>"S1S@SA9_SV\>"S1S@SCSDSE9_SW\>"S1S@SA9_SX\>"S1S@SCSDSE9_SY\>"S1S@SISJ9_SZ\>"S1S@SA9_S[\>"S1S@SCSDSE9_S\\>"S1S@SA9_S]\>"S1S@SCSDSE9_E0S^\>"S1S@SISJ9_S_\>"S1S@SA9_S`\>"S1S@SCSDSE9_Sa\>"S1S@SA9_Sb\>"S1S@SCSDSE9_Sc\>"S1S@SISJ9_Sd\>"S1S@SA9_Se\>"S1S@SCSDSE9_Sf\>"S1S@SA9_Sg\>"S1S@SCSDSE9_Sh\>"S1S@SISJ9_Si\>"S1S@SA9_Sj\>"S1S@SCSDSE9_Sk\>"S1S@SA9_Sl\>"S1S@SCSDSE9_Sm\>"S1S@SISJ9_E5r?\*SSn\;4Sojj5r@\*SSn\;4Spjj5rA\*SSn\;4Sqjj5rB\*SSn\;4Srjj5rC\*SSn\;4Ssjj5rD\*SSn\;4Stjj5rE\*SSn\;4Sujj5rF\*SSn\;4Svjj5rG\*SSn\;4Swjj5rH\*SSn\;4Sxjj5rI\*SSn\;4Syjj5rJ\*SSn\;4Szjj5rK\*SSn\;4S{jj5rL\*SSn\;4S|jj5rM\*SSn\;4S}jj5rN\*SSn\;4S~jj5rO\*SSn\;4Sjj5rP\*SSn\;4Sjj5rQg)a Poolformer from MetaFormer is Actually What You Need for Vision https://arxiv.org/abs/2111.11418 IdentityFormer, RandFormer, PoolFormerV2, ConvFormer, and CAFormer from MetaFormer Baselines for Vision https://arxiv.org/abs/2210.13452 All implemented models support feature extraction and variable input resolution. Original implementation by Weihao Yu et al., adapted for timm by Fredo Guan and Ross Wightman. Adapted from https://github.com/sail-sg/metaformer, original copyright below ) OrderedDict)partial)ListOptionalTupleUnionTypeN)Tensor)FinalIMAGENET_DEFAULT_MEANIMAGENET_DEFAULT_STD) trunc_normal_DropPathcalculate_drop_path_ratesSelectAdaptivePool2d GroupNorm1 LayerNorm LayerNorm2dMlpuse_fused_attn)build_model_with_cfg)feature_take_indices) checkpointcheckpoint_seq)generate_default_cfgsregister_model MetaFormerc l^\rSrSrSrS S\S\S\\\R4U4Sjjjr Sr Sr U=r $) Stem;zW Stem implemented by a layer of convolution. Conv2d params constant across all models. in_channels out_channels norm_layerc>XES.n[TU]5 [R"UU4SSSS.UD6UlU(aU"U40UD6Ulg[R "5Ulg)Ndevicedtype kernel_sizestridepadding)super__init__nnConv2dconvIdentitynorm)selfr#r$r%r(r)dd __class__s U/var/www/html/ai-image-ml/venv/lib/python3.13/site-packages/timm/models/metaformer.pyr2 Stem.__init__Asj / II        7AJ|2r2 bkkm cJURU5nURU5nU$Nr5r7r8xs r;forward Stem.forwardU! IIaL IIaLr=r@NNN__name__ __module__ __qualname____firstlineno____doc__intrr r3Moduler2rC__static_attributes__ __classcell__r:s@r;r!r!;sU59 TTT!bii1 TT(r=r!c|^\rSrSrSrS S\S\S\S\S\S\\\R4 U4S jjjr S r S r U=r $) Downsampling[z5 Downsampling implemented by a layer of convolution. r#r$r.r/r0r%c >XxS.n [T U]5 U(a U"U40U D6O[R"5Ul[R "UU4UUUS.U D6Ulg)Nr'r-)r1r2r3r6r7r4r5) r8r#r$r.r/r0r%r(r)r9r:s r;r2Downsampling.__init__`sc / 5?J{1b1R[[] II   $     r=cJURU5nURU5nU$r?)r7r5rAs r;rCDownsampling.forwardwrEr=r@)rrNNNrGrQs@r;rSrS[st48         !bii1  .r=rSc T^\rSrSrSrS S\S\S\S\4U4SjjjrSr S r U=r $) Scale}z* Scale vector by element multiplications. dim init_value trainableuse_nchwc>XVS.n[TU]5 U(aUSS4OU4Ul[R"U[ R "U40UD6-US9Ulg)Nr'r requires_grad)r1r2shaper3 Parametertorchonesscale) r8r\r]r^r_r(r)r9r:s r;r2Scale.__init__sQ / $,c1a[3& \\*uzz#/D/D"DT]^ r=cPXRRUR5-$r?)rgviewrcrAs r;rC Scale.forwards::??4::...r=)rgrc)?TTNN) rHrIrJrKrLrMfloatboolr2rCrOrPrQs@r;rZrZ}sY!$"! _ _ _ _  _ _//r=rZc>^\rSrSrSrSS\4U4SjjjrSrSrU=r $) SquaredReLUz0 Squared ReLU: https://arxiv.org/abs/2109.08668 inplacecT>[TU]5 [R"US9Ulg)Nrr)r1r2r3ReLUrelu)r8rrr:s r;r2SquaredReLU.__init__s GGG, r=cL[R"URU55$r?)resquarervrAs r;rCSquaredReLU.forwards||DIIaL))r=)rvF) rHrIrJrKrLrnr2rCrOrPrQs@r;rprps#---**r=rpc h^\rSrSrSrS S\S\S\S\S\\S\4 U4S jjjr S r S r U=r $) StarReLUz StarReLU: s * relu(x) ** 2 + b scale_value bias_valuescale_learnablebias_learnablemoderrc ,>XxS.n [T U]5 X`l[R"US9Ul[R "U[R"S0U D6-US9Ul [R "U[R"S0U D6-US9Ul g)Nr'rtra)r) r1r2rrr3rurvrdrerfrgbias) r8rrrrrrrr(r)r9r:s r;r2StarReLU.__init__su /  GGG, \\+ 0C0C"CSbc LLejj.Ab.A!AQ_` r=c^URURU5S--UR-$)Nr,)rgrvrrAs r;rCStarReLU.forwards'zzDIIaLA-- 99r=)rrrrvrg)rlTTNFNN) rHrIrJrKrLrmrnrstrr2rCrOrPrQs@r;r}r}s| "% #$(#'"&!aaa" a ! a 3- aaa$::r=r}c~^\rSrSr%Sr\\\S'SS\S\S\ \S\S\ S \ S \4U4S jjjr S r S r U=r$) Attentionz` Vanilla self-attention from Transformer: https://arxiv.org/abs/1706.03762. Modified from timm. fused_attnr\head_dim num_headsqkv_bias attn_drop proj_drop proj_biasc  >XS.n [T U]5 X lUS-Ul[ 5UlU(aUOX-UlUR S:XaSUlUR UR-Ul[R"XRS-4SU0U D6Ul [R"U5Ul [R"URU4SU0U D6Ul [R"U5Ulg)Nr'grrr)r1r2rrgrrr attention_dimr3LinearqkvDropoutrprojr) r8r\rrrrrrr(r)kwargsr9r:s r;r2Attention.__init__s /  % (*&/S_ >>Q DN!^^dmm;99S"4"4q"8NxN2NI.IId00#LILL I.r=cURup#nURU5RX#SURUR5R SSSSS5nUR S5upgnUR(a>[R"XgUUR(aURROSS9nOEXgRSS 5-UR-n U RS S 9n URU 5n X-nURSS5RX#U5nUR!U5nUR#U5nU$) Nrr,rrr+r) dropout_p)r\)rcrreshaperrpermuteunbindrFscaled_dot_product_attentiontrainingrp transposergsoftmaxrr) r8rBBNCrqkvattns r;rCAttention.forwards''ahhqk!!!4>>4==IQQRSUVXY[\^_`**Q-a ??..a.2mm$..**A  B++tzz9D<>$'DA KK1  % %aA . IIaL NN1 r=) rrrrrrrrrg) NFrrFNN)rHrIrJrKrLr rn__annotations__rMrrmr2rCrOrPrQs@r;rrsd  '+"!!#/// } /  /  ////>r=rc0^\rSrSrS\4U4SjjrSrU=r$)GroupNorm1NoBias num_channelsc d>[TU]"U40UD6 URSS5UlSUlgNepsư>r1r2getrrr8rrr:s r;r2GroupNorm1NoBias.__init__. 00::eT* r=rrrHrIrJrKrMr2rOrPrQs@r;rrSr=rc0^\rSrSrS\4U4SjjrSrU=r$)LayerNorm2dNoBiasirc d>[TU]"U40UD6 URSS5UlSUlgrrrs r;r2LayerNorm2dNoBias.__init__rr=rrrQs@r;rrrr=rc0^\rSrSrS\4U4SjjrSrU=r$)LayerNormNoBiasi rc d>[TU]"U40UD6 URSS5UlSUlgrrrs r;r2LayerNormNoBias.__init__ rr=rrrQs@r;rr rr=rc^\rSrSrSrS\\RSSSSS4S\S \ S \ \RS \ \RS \ S \S\4U4Sjjjr SrSrU=r$)SepConvizT Inverted separable convolution from MobileNetV2: https://arxiv.org/abs/1801.04381. r,Fr*rNr\expansion_ratio act1_layer act2_layerrr.r0c  >XS.n [T U]5 [X!-5n [R"X4SUS.U D6Ul[ U[5(aU"S0U D6OU"5Ul[R"U U 4UUU US.U D6Ul [ U[5(aU"S0U D6OU"5Ul [R"X4SUS.U D6Ul g)Nr'r)r.r)r.r0groupsr) r1r2rMr3r4pwconv1 issubclassr}act1dwconvact2pwconv2)r8r\rrrrr.r0r(r)rr9 mid_channelsr:s r;r2SepConv.__init__s / ?01 yySSPRS (2:x(H(HJ$$jl ii   $    )3:x(H(HJ$$jl yySSPRS r=cURU5nURU5nURU5nURU5nUR U5nU$r?)rrrrrrAs r;rCSepConv.forward3sH LLO IIaL KKN IIaL LLOr=)rrrrr)rHrIrJrKrLr}r3r6rMrmr rNrnr2rCrOrPrQs@r;rrs&'*2*,++ TT#TRYY T RYY T  TTTT<r=rc>^\rSrSrSrSS\4U4SjjjrSrSrU=r $)Poolingi<zL Implementation of pooling for PoolFormer: https://arxiv.org/abs/2111.11418 pool_sizec `>[TU]5 [R"USUS-SS9Ulg)Nrr,F)r/r0count_include_pad)r1r2r3 AvgPool2dpool)r8rrr:s r;r2Pooling.__init__As) LL1i1n`ef r=c,URU5nX!- $r?r)r8rBys r;rCPooling.forwardEs IIaLu r=r)r) rHrIrJrKrLrMr2rCrOrPrQs@r;rr<s&g#ggr=rc^\rSrSrSrSS\\SSSS4S\S \S \S \ \ RS \ \ RS \S\ 4U4Sjjjr SrSrU=r$)MlpHeadiJzMLP classification head r+rTNr\ num_classes mlp_ratio act_layerr% drop_raterc &>XS.n [T U]5 [X1-5n [R"X4SU0U D6UlU"5UlU"U 40U D6Ul[R"X4SU0U D6Ul[R"U5Ul g)Nr'r) r1r2rMr3rfc1actr7fc2r head_drop) r8r\rrrr%rrr(r)r9hidden_featuresr:s r;r2MlpHead.__init__Ns / io.99SCCC;5"5 99_KKKI.r=cURU5nURU5nURU5nURU5nUR U5nU$r?)rrr7rrrAs r;rCMlpHead.forwardcsI HHQK HHQK IIaL NN1  HHQKr=)rrrrr7)rHrIrJrKrLrprrMrmr r3rNrnr2rCrOrPrQs@r;rrJs $ )4*3!/// / BII / RYY ////*r=rc^\rSrSrSr\\S\SSSSSSS4 S\S\ \ RS \ \ RS \ S \ \ RS \ S \ S\ S\\ S\\ 4U4SjjjrSrSrU=r$)MetaFormerBlockilz) Implementation of one MetaFormer block. FrTNr\ token_mixermlp_actmlp_biasr%r drop_pathr_layer_scale_init_valueres_scale_init_valuec  >XS.n[TU]5 [[4XUS.UD6n[[4XUS.UD6nU"U40UD6UlU"SXS.UDU D6UlUS:a [ U5O[R"5Ul U bU"5O[R"5Ul U bU"5O[R"5Ul U"U40UD6Ul [U[SU-54UUUUS.UD6UlUS:a [ U5O[R"5UlU bU"5O[R"5UlU b U"5Ulg[R"5Ulg)Nr')r\r]r_)r\rrr+)rrdropuse_convr)r1r2rrZnorm1rrr3r6 drop_path1 layer_scale1 res_scale1norm2rrMmlp drop_path2 layer_scale2 res_scale2)r8r\rrrr%rrr_rrr(r)rr9ls_layerrs_layerr:s r;r2MetaFormerBlock.__init__qsW  / 5fcW_fcef5dcU]dacd*r* &T3TrTVT1:R(9-R[[]*@*LHJRTR]R]R_(<(H(*bkkm*r*   CL    2;R(9-R[[]*@*LHJRTR]R]R_(<(H(*bkkmr=c FURU5URURURUR U5555-nUR U5UR URURURU5555-nU$r?) r rrrrrr r r r rAs r;rCMetaFormerBlock.forwards OOA    $$TZZ]3   OOA    HHTZZ]+   r=) rr rr r rr r rr)rHrIrJrKrLrr}rrMr r3rNrnrmrr2rCrOrPrQs@r;rrls,3'/"*5!!!6:48'\'\bii'\"))_ '\  '\ RYY '\'\'\'\%-UO'\#+5/'\'\R  r=rc^^\rSrSrS\R \S\\SS/S-SSSS4 S\S\S\S \ \RS \ \RS \ S \ \ \RS \ \RS\ S\\ S\ \ S\ \ 4U4Sjjjr\R"R$SSj5rS\4SjrSrU=r$)MetaFormerStageir,FrNin_chsout_chsdepthrrrdownsample_normr%rdp_ratesrrc >XS.n[TU]5 SUl[U[5(+UlX:Xa[ R"5O[UU4SSSUS.UD6Ul [ R"[U5Vs/sH(n[SUUUUUU U UU U UR S. UDUD6PM* sn6Ul gs snf) Nr'Frr,r)r.r/r0r%) r\rrrr%rrrrr_r)r1r2grad_checkpointingrrr_r3r6rS downsample Sequentialrangerblocks)r8rrrrrrrr%rrrrr(r)rr9ir:s r;r2MetaFormerStage.__init__s$ / "'&{I>> ,2+<"++-,  C & C C mm< &! a'6 ' #!qk#9!5]] '  '  '   &! " &!s/B<cXlgr?)r)r8enables r;set_grad_checkpointing&MetaFormerStage.set_grad_checkpointings"(r=rBcURU5nURup#pEUR(d"URX#S5R SS5nUR (a:[ RR5(d[URU5nOURU5nUR(d"UR SS5RX#XE5nU$)Nrrr,) rrcr_rrrrejit is_scriptingrr )r8rBrrHWs r;rCMetaFormerStage.forwards OOA WW a}} !#--a3A  " "599+A+A+C+Ct{{A.A AA}} Aq!))!5Ar=)r rrr_T)rHrIrJrKr3r6r}rrMr rNrnrrmrr2rer(ignorer%r rCrOrPrQs@r;rrs +-;;'/"9D*5!%'D1H6:48/"/"/" /" bii /" "))_ /"/"&d299o6/"RYY/"/"5k/"%-UO/"#+5//"/"b YY))r=rc#^\rSrSrSrSSSSS\\SS S S S S \\\S S S 4S \ S\ S\ S\ \ S4S\ \ S4S\ \ \R\\ \R4S\ \RS\S\S\S\S\\ \\\4S\ \ \\S4\\\4S\\ \RS\ \ \R\\ \R4S\ \RS\4"U4SjjjrS r\R.R0S4S!j5r\R.R0S"\R4S#j5rS5S\ S\\ 4S$jjrS6S%\R8S&\\ \ \\ 4S'\S(\S)\ S*\S"\ \\R8\ \R8\\R8444S+jjrS7S&\ \ \\ 4S,\S-\4S.jjrS8S%\S/\4S0jjrS%\4S1jr S%\4S2jr!S3r"U=r#$)9riaMetaFormer A PyTorch impl of : `MetaFormer Baselines for Vision` - https://arxiv.org/abs/2210.13452 Args: in_chans (int): Number of input image channels. num_classes (int): Number of classes for classification head. global_pool: Pooling for classifier head. depths (list or tuple): Number of blocks at each stage. dims (list or tuple): Feature dimension at each stage. token_mixers (list, tuple or token_fcn): Token mixer for each stage. mlp_act: Activation layer for MLP. mlp_bias (boolean): Enable or disable mlp bias term. drop_path_rate (float): Stochastic depth rate. drop_rate (float): Dropout rate. layer_scale_init_values (list, tuple, float or None): Init value for Layer Scale. None means not use the layer scale. Form: https://arxiv.org/abs/2103.17239. res_scale_init_values (list, tuple, float or None): Init value for res Scale on residual connections. None means not use the res scale. From: https://arxiv.org/abs/2110.09456. downsample_norm (nn.Module): Norm layer used in stem and downsampling layers. norm_layers (list, tuple or norm_fcn): Norm layers for each stage. output_norm: Norm layer before classifier head. use_mlp_head: Use MLP classification head. rravgr,r,r,@i@FrN)NNrlrlTin_chansr global_pooldepths.dims token_mixersrrdrop_path_rateproj_drop_raterlayer_scale_init_valuesres_scale_init_valuesr norm_layers output_norm use_mlp_headc >[TU]5 UUS.nU[4;a [U40UD6nX lXlUSUlXlUUl[U5Ul [U[[45(dU/n[U[[45(dU/n[U[[45(dU/UR-n[U[[45(dU/UR-n[U [[45(dU /UR-n [U [[45(dU /UR-n SUl/Ul[!UUS4SU0UD6Ul/nUSn[%XSS9n['UR5HfnU[)UUU4UUUUUUU UUU UU UUUUS. UDUD6/- nUUnU=R[+UUS US --S U3S 9/- slMh [,R."U6UlUS:azUR(a5[3UR U4S UR0UD6nUR UlOI[,R6"UR U40UD6nUR UlO[,R8"5n[,R."[;S [=US94SU"UR 40UD64SU(a[,R>"S5O[,R8"54SUR(a[,R@"U 5O[,R8"54SU4/55Ul!UREURF5 g)Nr'rFrr%T) stagewise) rrrrrrrrrr%r,zstages.)num_chs reductionmodulerr8 pool_typer7flattenrrfc)$r1r2r}rrr7 num_featuresrrBlen num_stages isinstancelisttupler feature_infor!stemrrrdictr3rstagesrhead_hidden_sizerr6rrFlattenrheadapply _init_weights)r8r7rr8r9r:r;rrr<r=rr>r?rr@rArBr(r)rr9rUprev_dimrr!finalr:s r;r2MetaFormer.__init__ sW.  / xk !g,,G&  H"(f+&4-00XF$u ..6D,u 66(>DOO;L+e}55&-$//9K1D%=AA'>&?$//&Q #/$??%:$;doo$M !"'  G '    7,^tTt'A QQi(O!(!!'>q'A%:1%= /&q> F AwH   $tAw!ac(U\]^\_S`"a!b b %((mmV,  ?   1 1;_$.._\^_(,(9(9% $"3"3[GBG(,(9(9%KKMEMM+ 0;G H [!2!29b9 :  1 "++- H d.?.?RZZ *R[[] S 5M / #    4%%&r=c[U[R[R45(aM[ UR SS9 UR b+[RRUR S5 ggg)Ng{Gz?)stdr) rOr3r4rrweightrinit constant_)r8ms r;rZMetaFormer._init_weightsxsV a"))RYY/ 0 0 !(( ,vv!!!!&&!," 1r=cTXlURHnURUS9 M g)N)r$)rrUr%)r8r$stages r;r%!MetaFormer.set_grad_checkpointing~s'"([[E  ( ( ( 7!r=returnc.URR$r?)rXrK)r8s r;get_classifierMetaFormer.get_classifiersyy||r=cX4S.nXlUbY[US9URlU(a[R "S5O[R "5URlUS:aXUR(a$[URU4SUR0UD6nO8[R"URU40UD6nO[R "5nX`Rl g)Nr'rHrrr)rrrXr8r3rWr6rJrBrrLrrrK)r8rr8r(r)r9r\s r;reset_classifierMetaFormer.reset_classifiers /&  "$8;$ODII !1< 1 "++-DII  ?   1 1;_$.._\^_ $"3"3[GBGKKME r=rBindicesr7 stop_early output_fmtintermediates_onlycUS;dS5e/n[[UR5U5upURU5n[R R 5(dU(d URn OURSU S-n [U 5HeupUR(a/[R R 5(d [X5nOU "U5nX;dMTURU5 Mg U(aU$X4$)aForward features that returns intermediates. Args: x: Input image tensor indices: Take last n blocks if int, all if None, select matching indices if sequence norm: Apply norm layer to compatible intermediates stop_early: Stop iterating over blocks when last desired intermediate hit output_fmt: Shape of intermediate feature outputs intermediates_only: Only return intermediate features Returns: )NCHWzOutput shape must be NCHW.Nr) rrMrUrSrer(r) enumeraterrappend) r8rBror7rprqrr intermediates take_indices max_indexrUfeat_idxrfs r;forward_intermediates MetaFormer.forward_intermediatess*Y&D(DD& "6s4;;7G"Q  IIaL 99 ! ! # #:[[F[[)a-0F(0OH&&uyy/E/E/G/Gu(!H'$$Q' 1  r= prune_norm prune_headc[[UR5U5upEURSUS-UlU(aURSS5 U$)z?Prune layers not required for specified intermediates. Nrr)rrMrUrm)r8ror}r~rxrys r;prune_intermediate_layers$MetaFormer.prune_intermediate_layerssK#7s4;;7G"Q kk.9q=1   ! !!R (r= pre_logitsc"URRU5nURRU5nURRU5nURR U5nU(aU$URR U5$r?)rXr8r7rJrrK)r8rBrs r; forward_headMetaFormer.forward_headsg II ! !! $ IINN1  II  a  IINN1 q3DIILLO3r=cURU5nUR(a;[RR 5(d[ UR U5nU$UR U5nU$r?)rSrrer(r)rrUrAs r;forward_featuresMetaFormer.forward_featuressU IIaL  " "599+A+A+C+Ct{{A.A AAr=cJURU5nURU5nU$r?)rrrAs r;rCMetaFormer.forwards'  ! !! $   a r=) rrRrrXrVr7rrLrNrUrSrBr-rF)NFFrtF)rFTr{)$rHrIrJrKrLrr}rrrMrrrr r3rNrrnrmrr2rZrer(r.r%rjrmr r{rrrrCrOrPrQs@r;rrs6#$&2$7JQ'/"$&$&"KO_u9JIZ+6!%)i'i'i' i' #s(O i' S/ i' RYYd299o1F FGi'"))_i'i'"i'"i'i'&.eE4;4F.G%Hi'$)x/C)Dd8TY?F[)[#\i'&d299o6i' tBIIT"))_0EEF!i'"bii#i'$%i'i'V-  YY88  YY  C hsm $8<$$',+ ||+ eCcN34+  +  +  + !%+  tELL!5tELL7I)I#JJ K+ ^./$# 3S >*   4f4$4&r=cSU;aU$SSKn0nSU;nUR5nUR5GHupgU(aURSSU5nUR SS5nUR SS 5nUR S S 5nUR S S5nUR S S 5nUR SS5nUR SS5nURSSU5nUR SS5nUR SS5nURSSU5nUR SS5nUR SS5nUR SS5nUR SS5nURSS U5nURS!S"U5nUR XV:waAUR 5XVR 5:XaURXVR 5nXsU'GM U$)#Nzstem.conv.weightrznetwork.0.0.mlp.fc1.weightzlayer_scale_([0-9]+)zlayer_scale\1.scalez network.1zdownsample_layers.1z network.3zdownsample_layers.2z network.5zdownsample_layers.3z network.2z network.4z network.6networkrUzdownsample_layers.([0-9]+)zstages.\1.downsamplezdownsample.projzdownsample.convzpatch_embed.projzpatch_embed.convz([0-9]+).([0-9]+)z \1.blocks.\2zstages.0.downsample patch_embedrS post_normr7pre_normz^headhead.fcz^normz head.norm)re state_dictitemssubreplacercnumelr)rmodelrout_dictis_poolformerv1model_state_dictrrs r;checkpoint_filter_fnrsZ' H2j@O'')  " .0FJA +'<=A +'<=A +'<=A +{3A +{3A +{3A )X.A FF02I1 M II'): ; II(*< = FF'! < II+] ; IImV , IIk6 * IIj& ) FF8Y * FF8[! , 77&) )aggi;K;N;T;T;V.V *-334A 3#4 Or=c [S[URSS5555nURSU5n[ [ UU4[ [SUS9S.UD6nU$)Nc3*# UH upUv M g7fr?r).0r!_s r; %_create_metaformer..s\.[da.[sr9r1 out_indicesT)flatten_sequentialr)pretrained_filter_fn feature_cfg)rQrurpoprrrrT)variant pretrainedrdefault_out_indicesrrs r;_create_metaformerrsl\i 8\8Z.[\\**],?@K  2DkJ     E Lr=c 2USSSSS[[SSSS . UE$) Nr)rr)r*r*rlbicubicrz stem.convz apache-2.0) urlr input_sizercrop_pct interpolationmeanr_ classifier first_convlicenser )rrs r;_cfgrs3=v)%.B{    r=zpoolformer_s12.sail_in1kztimm/g?) hf_hub_idrzpoolformer_s24.sail_in1kzpoolformer_s36.sail_in1kzpoolformer_m36.sail_in1kgffffff?zpoolformer_m48.sail_in1kzpoolformerv2_s12.sail_in1k)rzpoolformerv2_s24.sail_in1kzpoolformerv2_s36.sail_in1kzpoolformerv2_m36.sail_in1kzpoolformerv2_m48.sail_in1kzconvformer_s18.sail_in1kz head.fc.fc2)rrzconvformer_s18.sail_in1k_384)rr) r)rrrrz!convformer_s18.sail_in22k_ft_in1kz%convformer_s18.sail_in22k_ft_in1k_384zconvformer_s18.sail_in22kiQU)rrrzconvformer_s36.sail_in1kzconvformer_s36.sail_in1k_384z!convformer_s36.sail_in22k_ft_in1kz%convformer_s36.sail_in22k_ft_in1k_384zconvformer_s36.sail_in22kzconvformer_m36.sail_in1kzconvformer_m36.sail_in1k_384z!convformer_m36.sail_in22k_ft_in1kz%convformer_m36.sail_in22k_ft_in1k_384zconvformer_m36.sail_in22kzconvformer_b36.sail_in1kzconvformer_b36.sail_in1k_384z!convformer_b36.sail_in22k_ft_in1kz%convformer_b36.sail_in22k_ft_in1k_384zconvformer_b36.sail_in22kzcaformer_s18.sail_in1kzcaformer_s18.sail_in1k_384zcaformer_s18.sail_in22k_ft_in1kz#caformer_s18.sail_in22k_ft_in1k_384zcaformer_s18.sail_in22kzcaformer_s36.sail_in1kzcaformer_s36.sail_in1k_384zcaformer_s36.sail_in22k_ft_in1kz#caformer_s36.sail_in22k_ft_in1k_384zcaformer_s36.sail_in22kzcaformer_m36.sail_in1kzcaformer_m36.sail_in1k_384zcaformer_m36.sail_in22k_ft_in1kz#caformer_m36.sail_in22k_ft_in1k_384zcaformer_m36.sail_in22kzcaformer_b36.sail_in1kzcaformer_b36.sail_in1k_384zcaformer_b36.sail_in22k_ft_in1kz#caformer_b36.sail_in22k_ft_in1k_384zcaformer_b36.sail_in22krhc r[S/SQ/SQS[RS[SSSS. UD6n[ S SU0UD6$) Nr1r3Th㈵>F r9r:rrrr@r>r?rBrr)poolformer_s12rTr3GELUrrrr model_kwargss r;rrsO   $"   L  V: V VVr=c r[S/SQ/SQS[RS[SSSS. UD6n[ S SU0UD6$) Nr+r+rr+r3TrFrrr)poolformer_s24rrs r;rrO   $"   L  V: V VVr=c r[S/SQ/SQS[RS[SSSS. UD6n[ S SU0UD6$) Nr2r2r2r3TrFrrr)poolformer_s36rrs r;rrrr=c r[S/SQ/SQS[RS[SSSS. UD6n[ S SU0UD6$) Nr`rTrFrrr)poolformer_m36rrs r;rrrr=c r[S/SQ/SQS[RS[SSSS. UD6n[ S SU0UD6$) NrrrTrFrrr)poolformer_m48rrs r;rrrr=c L[S/SQ/SQ[SS.UD6n[SSU0UD6$)Nr1r3Fr9r:r@rBrr)poolformerv2_s12rTrrrs r;rrs< $   L  XZ X< XXr=c L[S/SQ/SQ[SS.UD6n[SSU0UD6$)Nrr3Frrr)poolformerv2_s24rrs r;rr< $   L  XZ X< XXr=c L[S/SQ/SQ[SS.UD6n[SSU0UD6$)Nrr3Frrr)poolformerv2_s36rrs r;rr)rr=c L[S/SQ/SQ[SS.UD6n[SSU0UD6$)NrrFrrr)poolformerv2_m36rrs r;rr4rr=c L[S/SQ/SQ[SS.UD6n[SSU0UD6$)NrrFrrr)poolformerv2_m48rrs r;rr?rr=c T[S/SQ/SQ[[S.UD6n[SSU0UD6$)Nrr rr3r9r:r;r@rr)convformer_s18rTrrrrs r;rrJs< %   L  V: V VVr=c T[S/SQ/SQ[[S.UD6n[SSU0UD6$)Nrrrrr3rrr)convformer_s36rrs r;rrU< %   L  V: V VVr=c T[S/SQ/SQ[[S.UD6n[SSU0UD6$)Nrrrri@rrr)convformer_m36rrs r;rr`rr=c T[S/SQ/SQ[[S.UD6n[SSU0UD6$)Nrr5r6rrrr)convformer_b36rrs r;rrks< !%   L  V: V VVr=c [S/SQ/SQ[[[[/[/S-[/S--S.UD6n[ SSU0UD6$)Nrr3r,rrr) caformer_s18rTrrrrrrs r;rrvsZ w 9=&'!+.?!.CC   L  T T| TTr=c [S/SQ/SQ[[[[/[/S-[/S--S.UD6n[ SSU0UD6$)Nrr3r,rrr) caformer_s36rrs r;rrZ w 9=&'!+.?!.CC   L  T T| TTr=c [S/SQ/SQ[[[[/[/S-[/S--S.UD6n[ SSU0UD6$)Nrrr,rrr) caformer_m36rrs r;rrrr=c [S/SQ/SQ[[[[/[/S-[/S--S.UD6n[ SSU0UD6$)Nrrr,rrr) caformer_b36rrs r;rrsZ !w 9=&'!+.?!.CC   L  T T| TTr=r{)r)RrL collectionsr functoolsrtypingrrrrr retorch.nnr3torch.nn.functional functionalrr torch.jitr timm.datar r timm.layersrrrrrrrrr_builderr _featuresr _manipulaterr _registryrr__all__rNr!rSrZrpr}rrrrrrrrrrrrr default_cfgsrrrrrrrrrrrrrrrrrrrr=r;r s 8$55 A   ++3< .299@299D/BII/. *")) *:ryy:69 9~z bll)bii)X bii biiD;bii;|FbiiFRrrl"J  %V&!V&! V&!V&!V&!V&"!$"9#V&$!$"9%V&&!$"9'V&(!$"9)V&*!$"9+V&. !"/V&4#D ]h%P5V&:( *";V&@,T ]h.PAV&F  e"5GV&N !"OV&T#D ]h%PUV&Z( *"[V&`,T ]h.PaV&f  e"5gV&n !"oV&t#D ]h%PuV&z( *"{V&@,T ]h.PAV&F  e"5GV&N !"OV&T#D ]h%PUV&Z( *"[V&`,T ]h.PaV&f  e"5gV&nd "oV&t!$ ]h#PuV&z&t ("{V&@*4 ]h,PAV&Ft e 5GV&Nd "OV&T!$ ]h#PUV&Z&t ("[V&`*4 ]h,PaV&ft e 5gV&nd "oV&t!$ ]h#PuV&z&t ("{V&@*4 ]h,PAV&Ft e 5GV&Nd "OV&T!$ ]h#PUV&Z&t ("[V&`*4 ]h,PaV&ft e 5gV&V r W* W W W* W W W* W W W* W W W* W WYJYYYJYYYJYYYJYYYJYYW*WWW*WWW*WWW*WWU UUU UUU UUU UUr=