RЦitSrS/rSSKrSSKJr SSKJrJrJrJ r J r J r J r SSK r SSKJr SSKJs Jr SSKJrJr SSKJrJrJrJrJrJrJr SS KJr SS K J!r! SS K"J#r# SS K$J%r%J&r& SS K'J(r(J)r) "SS\ RRT5r+"SS\RX5r-"SS\RX5r."SS\RX5r/"SS\RX5r0"SS\RX5r1"SS\ RRX5r2"SS\RX5r3\#"\35 "SS\RX5r4"S S\RX5r5S!r6S6S"jr7\)"\7"S#S$S%9\7"S#S&9\7"S#S&9\7"S#S$S%9\7"S#S&9\7"S#S&9\7"S#S$S%9\7"S#S&9\7"S#S&9\7"S#S'S(S)S*9\7"S#S+S,S)S-S.9S/. 5r8S7S0jr9\(S7S1j5r:\(S7S2j5r;\(S7S3j5r<\(S7S4j5r=\(S7S5j5r>g)8zTinyViT Paper: `TinyViT: Fast Pretraining Distillation for Small Vision Transformers` - https://arxiv.org/abs/2207.10666 Adapted from official impl at https://github.com/microsoft/Cream/tree/main/TinyViT TinyVitN)partial)DictListOptionalTupleUnionTypeAnyIMAGENET_DEFAULT_MEANIMAGENET_DEFAULT_STD) LayerNorm2dNormMlpClassifierHeadDropPath trunc_normal_resize_rel_pos_bias_table_levituse_fused_attncalculate_drop_path_rates)build_model_with_cfg)feature_take_indices)register_notrace_module) checkpointcheckpoint_seq)register_modelgenerate_default_cfgsc^\rSrSrS S\S\S\S\S\S\S\S \4U4S jjjr\R"5S 5r S r U=r $)ConvNormin_chsout_chsksstridepaddilationgroupsbn_weight_initc >XS.n [T U]5 [R"XX4XVU4SS0U D6Ul[R "U40U D6Ul[RRRUR RU5 [RRRUR RS5 g)NdevicedtypebiasFr) super__init__nnConv2dconv BatchNorm2dbntorchinit constant_weightr-) selfr!r"r#r$r%r&r'r(r+r,dd __class__s S/var/www/html/ai-image-ml/venv/lib/python3.13/site-packages/timm/models/tiny_vit.pyr/ConvNorm.__init__s / IIfr3&cW\c`bc ../B/  ?   a0c :URURp!URURUR-S-- nURUSS2SSS4-nUR UR UR-URUR-S-- - n[RRURS5URR-URS5URSSURRURRURRURRS9nURR R#U5 UR R R#U5 U$)Ng?rr)r$paddingr&r')r2r4r8 running_varepsr- running_meanr5r0r1sizer'shaper$rAr&datacopy_)r9cr4wbms r<fuse ConvNorm.fuse2s3 4772 II"&&0S8 8 HHqD$,- - GGboo 1 ^^bff $ ,- - HHOO FF1I (( (!&&)QWWQR[99##TYY->->I[I[dhdmdmdtdt  v  A  !r>)r4r2)rrrrrrNN) __name__ __module__ __qualname____firstlineno__intfloatr/r5no_gradrM__static_attributes__ __classcell__r;s@r<rrs $%111 1  1  111"11( ]]_  r>rc`^\rSrSrSS\S\S\\R4U4SjjjrSr Sr U=r $) PatchEmbedAr!r" act_layerc>XES.n[TU]5 SUl[XS-SSS40UD6UlU"5Ul[US-USSS40UD6Ulg)Nr*r@r)r.r/r$rconv1actconv2)r9r!r"r\r+r,r:r;s r<r/PatchEmbed.__init__Bs` /  flAq!BrB ;glGQ1CC r>clURU5nURU5nURU5nU$N)r`rarbr9xs r<forwardPatchEmbed.forwardQs. JJqM HHQK JJqMr>)rar`rbr$NN rOrPrQrRrSr r0Moduler/rhrVrWrXs@r<rZrZAsH  D D DBII D Dr>rZc h^\rSrSrS S\S\S\S\\RS\4 U4Sjjjr Sr S r U=r $) MBConvXr!r" expand_ratior\ drop_pathch>XgS.n[T U]5 [X-5n [X4SS0UD6UlU"5Ul[X4SSSU S.UD6UlU"5Ul[X4SSS.UD6UlU"5Ul US:a[U5Ul g[R"5Ul g)Nr*r#rr_r#r$r%r')r#r()r.r/rSrr`act1rbact2conv3act3rr0Identityrq) r9r!r"rpr\rqr+r,r:mid_chsr;s r<r/MBConv.__init__Ys / f+,f:!:r: K g\1QAg\Y[\ K gO1SOBO K 09B),BKKMr>cUnURU5nURU5nURU5nURU5nUR U5nUR U5nX- nUR U5nU$re)r`rurbrvrwrqrx)r9rgshortcuts r<rhMBConv.forwardnso JJqM IIaL JJqM IIaL JJqM NN1   IIaLr>)rurvrxr`rbrwrqrj) rOrPrQrRrSrTr r0rlr/rhrVrWrXs@r<rnrnXs^RRR R BII R  RR*  r>rnc`^\rSrSrSS\S\S\\R4U4SjjjrSr Sr U=r $) PatchMerging{dimout_dimr\c>XES.n[TU]5 [XSSS40UD6UlU"5Ul[X"SSS4SU0UD6UlU"5Ul[X"SSS40UD6Ulg)Nr*rrr_r@r')r.r/rr`rurbrvrw)r9rrr\r+r,r:r;s r<r/PatchMerging.__init__|st / cAq!:r: K g1aNN2N K g1a>2> r>cURU5nURU5nURU5nURU5nUR U5nU$re)r`rurbrvrwrfs r<rhPatchMerging.forwardsH JJqM IIaL JJqM IIaL JJqMr>)rurvr`rbrwrjrkrXs@r<rr{sC  ???BII ?? r>rc |^\rSrSrS S\S\S\\RS\\ \ \ 4S\ 4 U4Sjjjr Sr S r U=r$) ConvLayerrdepthr\rqconv_expand_ratioc>XgS.n[T U]5 XlX l[R "[ U5V s/sH,n [UUUU[U[5(aXIOU40UD6PM. sn 6Ul gs sn fNr*) r.r/rrr0 Sequentialrangern isinstancelistblocks) r9rrr\rqrr+r,r:ir;s r<r/ConvLayer.__init__s /  mm5\ & " ! *9d ; ;      " &   & s3A:c(URU5nU$re)rrfs r<rhConvLayer.forwards KKNr>)rrr)rt@NN)rOrPrQrRrSr r0rlr rTrr/rhrVrWrXs@r<rrsk 46')   BII  UDK/0  %   4r>rc ^\rSrSrSS\R \R SSS4S\S\\S\\S\ \RS\ \RS \ 4 U4S jjjr S r S rU=r$) NormMlpNrt in_featureshidden_features out_features norm_layerr\dropc f>XxS.n [T U]5 U=(d UnU=(d UnU"U40U D6Ul[R"X40U D6UlU"5Ul[R"U5Ul[R"X#40U D6Ul [R"U5Ul gr) r.r/normr0Linearfc1raDropoutdrop1fc2drop2) r9rrrrr\rr+r,r:r;s r<r/NormMlp.__init__s / #2{ )8[{1b1 99[@R@;ZZ% 99_AbAZZ% r>cURU5nURU5nURU5nURU5nUR U5nUR U5nU$re)rrrarrrrfs r<rhNormMlp.forwardsU IIaL HHQK HHQK JJqM HHQK JJqMr>)rarrrrr)rOrPrQrRr0 LayerNormGELUrSrr rlrTr/rhrVrWrXs@r<rrs.2*.*,,,)+&&&c]&#3- & RYY & BII &&&,r>rc P^\rSrSr%\R R \\S'\ \ \R4\S'SS\ S\ S\ S\ S\ \ \ 44 U4S jjjr\R"5SU4S jj5rS \R"S \R4S jrSrSSjrSSjrSSjrSrU=r$) Attention fused_attnattention_bias_cacherkey_dim num_heads attn_ratio resolutionc <>XgS.n[T U]5 [U[5(a[ U5S:XdeX0lUS-UlX l[XB-5Ul URU-Ul X@l XPl [5Ul[R "U40UD6Ul[R$"XURSU---40UD6Ul[R$"URU40UD6UlUSUS-n USUS-n [*RR-[*R."X:40UD65UlUR3S[*R."X4U[*R4S9SS9 0UlUR95 g) Nr*r@grrattention_bias_idxsF) persistent)r.r/rtuplelenrscalerrSval_dimrrrrrr0rrrqkvprojr5 Parameteremptyattention_biasesregister_bufferlongrreset_parameters) r9rrrrrr+r,r:N num_offsetsr;s r<r/Attention.__init__sf / *e,,ZA1EEE"_  :/0 ||i/ $$(*LL++ 99St||a'k/I"JQbQIIdllC626 qMJqM ) mjm3 % 2 25;;y3\Y[3\ ]  ! KKvUZZ @  %'! r>cf>[TU]U5 U(aUR(a0Ulgggre)r.trainr)r9moder;s r<rAttention.trains)  d D--(*D %.4r>r+returncJ[RR5(dUR(aURSS2UR 4$[ U5nX R;a*URSS2UR 4URU'URU$re)r5jit is_tracingtrainingrrstrr)r9r+ device_keys r<get_attention_biasesAttention.get_attention_biasess 99   ! !T]]((D,D,D)DE EVJ!:!::8<8M8MaQUQiQiNi8j))*5,,Z8 8r>cURUR5nURup4nURU5nUR U5nUR X4UR S5RURURUR/SS9upxn URSSSS5nURSSSS5nU RSSSS5n UR(a[R"XxXS9nO9XpR-nXxRSS5-n X-n U R!SS9n X-nURSS5R#X4UR$5nUR'U5nU$) Nr_)rrr@r) attn_mask)rr+rFrrviewrsplitrrpermuterFscaled_dot_product_attentionr transposesoftmaxreshaperr) r9rg attn_biasBr_rqkvattns r<rhAttention.forwardsC--ahh7 ''a IIaLhhqk((14::DLL$,,X\XdXd;ekl:ma IIaAq ! IIaAq ! IIaAq ! ??..qQLAJJA{{2r**D#D<cv[RRUR5 UR 5 g)z"Initialize parameters and buffers.N)r0r6zeros_r _init_buffersr9s r<rAttention.reset_parameters)s$ t,,- r>c HURRn[[R"[ UR S5[ UR S555n[U5n0n/nUHYnUHPn[USUS- 5[USUS- 54nX;a [U5XH'URXH5 MR M[ URR[R"XQ[RS9RX355 0Ulg)z.Compute and fill non-persistent buffer values.rrr*N)rr+r itertoolsproductrrrabsappendrHr5tensorrrr) r9r+pointsrattention_offsetsidxsp1p2offsets r<rAttention._init_buffers.s))00i''dooa.@(A5YZI[C\]^ KBbebem,c"Q%"Q%-.@A2034E0F%- -56    &&u||Duzz'Z'_'_`a'ef$&!r>c$UR5 g)z"Initialize non-persistent buffers.N)rrs r<init_non_persistent_buffers%Attention.init_non_persistent_buffers>s r>) rrrrrrrrrrrrr)r^)rNNT)rN)rOrPrQrRr5rFinalbool__annotations__rrTensorrSrr/rUrr+rrhrrrrVrWrXs@r<rrs %%sELL011 *2% % %  %  % c3h % % N ]]_++ 95<<9ELL92 ' r>rc^\rSrSrSrSSSSS\R SS4S\S \S \S \S \S \S\S\ \R4U4Sjjjr Sr S\ 4SjrSrU=r$) TinyVitBlockiCaTinyViT Block. Args: dim (int): Number of input channels. num_heads (int): Number of attention heads. window_size (int): Window size. mlp_ratio (float): Ratio of mlp hidden dim to embedding dim. drop (float, optional): Dropout rate. Default: 0.0 drop_path (float, optional): Stochastic depth rate. Default: 0.0 local_conv_size (int): the kernel size of the convolution between Attention and MLP. Default: 3 act_layer: the activation function. Default: nn.GELU rrtr_Nrr window_size mlp_ratiorrqlocal_conv_sizer\c >XS.n [TU]5 XlX lUS:dS5eX0lX@lX-S:XdS5eX-n X34n [ UU U4SU S.U D6UlUS:a [U5O[R"5Ul [S U[X-5UUS.U D6UlUS:a [U5O[R"5UlUS -n[!X4USXS .U D6Ulg) Nr*rz"window_size must be greater than 0z"dim must be divisible by num_headsr)rrrt)rrr\rr@rs)r.r/rrr r rrrr0ry drop_path1rrSmlp drop_path2r local_conv)r9rrr r rrqr r\r+r,r:head_dimwindow_resolutionr%r;s r<r/TinyVitBlock.__init__Rs / "QD DD&"!#I%II##(6    (     2;R(9-R[[] 0    2;R(9-R[[]""3esebder>c URup#pEX4-nUnX0R:XaFX@R:Xa7URX&U5nURU5nUR X#XE5nGOURX0R-- UR-nURX@R-- UR-n US:=(d U S:n U (a[ R "USSSU SU45nX8-XI-pXR-n XR-nUR X-URXRU5RSS5RX--U-URUR-U5nURU5nUR X-XRURU5RSS5RX+X5nU (aUSS2SU2SU24R5nXpRU5-nURSSSS5nURU5nURX%U5RSS5nXRURU55-nUR X#XE5$Nrr@r_r)rFr rrrrr%r contiguousrrrrr)r9rgrHWCLr}pad_bpad_rrApHpWnHnWs r<rhTinyVitBlock.forwardsFWW a E   Q*:*:%: !"A ! AqQ"A%%,<,<(<<@P@PPE%%,<,<(<<@P@PPEai,519GEE!aAua78Y '''B'''Bqd..4D4DaHRRSTVWX`` T--0@0@@!A ! Aqb"2"2D4D4DaHRRSTVWX``abhjnAa!RaRiL++- q) ) IIaAq ! OOA  IIaA  ( (A .  , ,vvaA!!r>rcnSURSURSURSUR3$)Ndim=z , num_heads=z, window_size=z , mlp_ratio=)rrr r rs r< extra_reprTinyVitBlock.extra_reprs@dhhZ|DNN+;<"../|DNN;KM Mr>) rrrrrrr rr )rOrPrQrR__doc__r0rrSrTr rlr/rhrr%rVrWrXs@r<rrCs $ !!!#$)+-f-f-f -f  -f  -f-f!-fBII-f-f^&"PMCMMr>rc^\rSrSrSrSSSSS\R SS4S\S\S \S \S \S \S \S\ \\ \4S\ \ \RS\S\ \R4U4SjjjrSrS\4SjrSrU=r$) TinyVitStageiaA basic TinyViT layer for one stage. Args: dim (int): Number of input channels. out_dim: the output dimension of the layer depth (int): Number of blocks. num_heads (int): Number of attention heads. window_size (int): Local window size. mlp_ratio (float): Ratio of mlp hidden dim to embedding dim. drop (float, optional): Dropout rate. Default: 0.0 drop_path (float | tuple[float], optional): Stochastic depth rate. Default: 0.0 downsample (nn.Module | None, optional): Downsample layer at the end of the layer. Default: None local_conv_size: the kernel size of the depthwise convolution between attention and MLP. Default: 3 act_layer: the activation function. Default: nn.GELU rrtNr_rrrrr r rrq downsampler r\cr>XS.n[TU]5 X0lX lU bU "SUUU S.UD6UlO![ R "5UlX:Xde[ R"[U5Vs/sH0n[SUUUUU[U[5(aXOUU U S.UD6PM2 sn6Ul gs snf)Nr*)rrr\)rrr r rrqr r\r ) r.r/rrr*r0ryrrrrrr)r9rrrrr r rrqr*r r\r+r,r:rr;s r<r/TinyVitStage.__init__s  /     !(# DO!kkmDO> !>mm5\ &#"  #'#*4Y*E*E),9 /#  " &# $ &#s37B4cURU5nURSSSS5nURU5nURSSSS5nU$r)r*rrrfs r<rhTinyVitStage.forwardsJ OOA  IIaAq ! KKN IIaAq !r>rc:SURSUR3$)Nr$z, depth=)rrrs r<r%TinyVitStage.extra_reprsdll^8DJJ<88r>)rrr*r)rOrPrQrRr'r0rrSrTr rrr rlr/rhrr%rVrWrXs@r<r)r)s. "3548#$)+.$.$.$ .$  .$  .$.$.$UDK/0.$!bii1.$!.$BII.$.$`9C99r>r)cz^\rSrSrSSSSSSSS S S S S S\R S S 4S\S\S\S\\S4S\\S4S\\S4S\\S4S\ S\ S\ S\ S\ S\S\ \R4U4Sjjjr S8S\ 4SjjrS8S \RS\ 4S!jjr\R"R$S"5r\R"R$S#5r\R"R$S9S$j5r\R"R$S8S%j5r\R"R$S&\R4S'j5rS:S\S\\4S(jjrS;S)\R4S*\\\\\4S+\ S,\ S-\S.\ S&\\\R4\\R4\\R4444S/jjrS[TU]5 UUS.nX lXlXPl[ U5UlXlXl[SUUSUS.UD6Ul [U [U55n[R"5UlURR nUSn/Ul[%UR 5HnUS:Xa['SUUUUUSUUU S.UD6nO`UUnU[USU5[USUS-5n [)SUUS- UUUUUUUURU U U [*US. UD6nUnUS-nURR-U5 U=R"[/UUSU3S 9/- slM US =UlUl[5[6S S 9n[9UR0U4UUS .UD6UlUR=SS9 g)Nr*r)r!r"r\)rrr\rqrr) rrrrr r rr rqr*r\r@zstages.)num_chs reductionmodulergh㈵>)rC) pool_typerF needs_resetr )r.r/r>r=rAr num_stagesr grad_checkpointingrZ patch_embedrsumr0rstagesr$ feature_inforrr)rrdict num_featureshead_hidden_sizerrrhead init_weights)r9r=r>r?r@rArrBr rCrDrErFr r\r+r,r:dprr$prev_dim stage_idxstager norm_layer_cfr;s r<r/TinyVit.__init__s9&  /&  f+""0% qM   (F Dmmo !!((a=t/IA~!  +'!"46)#45&9   %Y/!$S );%rMcJUR[URUS95 g)NrL)applyr _init_weights)r9rMs r<rXTinyVit.init_weightsYs 74--;GHr>rLc0[U[R5(aM[URSS9 UR b+[R RUR S5 ggU(a#[US5(aUR5 ggg)Ng{Gz?)stdrr) rr0rrr8r-r6r7hasattrr)r9rLrMs r<raTinyVit._init_weights\sk a # # !(( ,vv!!!!&&!," WQ(:;;   <[r>cS1$Nrr rs r<no_weight_decay_keywords TinyVit.no_weight_decay_keywordsds "##r>czUR5R5Vs1sH nSU;dM UiM sn$s snfrh) state_dictkeysrfs r<no_weight_decayTinyVit.no_weight_decayhs4??,113O3a7IQ7N3OOOs 88c0[SU(aSOSS/S9nU$)Nz ^patch_embedz^stages\.(\d+))z^stages\.(\d+).downsample)r)z^stages\.(\d+)\.\w+\.(\d+)N)stemr)rT)r9coarsematchers r< group_matcherTinyVit.group_matcherls' (.$455 r>cXlgre)rO)r9enables r<set_grad_checkpointingTinyVit.set_grad_checkpointingws"(r>rc.URR$re)rWfcrs r<get_classifierTinyVit.get_classifier{syy||r>cBXlURRXS9 g)N)rK)r>rWreset)r9r>r?s r<reset_classifierTinyVit.reset_classifiers&  ;r>rgindicesr stop_early output_fmtintermediates_onlycUS;dS5e/n[[UR5U5upURU5n[R R 5(dU(d URn OURSU S-n [U 5HeupUR(a/[R R 5(d [X5nOU "U5nX;dMTURU5 Mg U(aU$X4$)aForward features that returns intermediates. Args: x: Input image tensor indices: Take last n blocks if int, all if None, select matching indices if sequence norm: Apply norm layer to compatible intermediates stop_early: Stop iterating over blocks when last desired intermediate hit output_fmt: Shape of intermediate feature outputs intermediates_only: Only return intermediate features Returns: )NCHWzOutput shape must be NCHW.Nr) rrrRrPr5r is_scripting enumeraterOrr) r9rgrrrrr intermediates take_indices max_indexrRfeat_idxr\s r<forward_intermediatesTinyVit.forward_intermediatess*Y&D(DD& "6s4;;7G"Q    Q  99 ! ! # #:[[F[[)a-0F(0OH&&uyy/E/E/G/Gu(!H'$$Q' 1  r> prune_norm prune_headc[[UR5U5upEURSUS-UlU(aURSS5 U$)z?Prune layers not required for specified intermediates. Nrr)rrrRr)r9rrrrrs r<prune_intermediate_layers!TinyVit.prune_intermediate_layerssK#7s4;;7G"Q kk.9q=1   ! !!R (r>cURU5nUR(a;[RR 5(d[ UR U5nU$UR U5nU$re)rPrOr5rrrrRrfs r<forward_featuresTinyVit.forward_featuressX   Q   " "599+A+A+C+Ct{{A.A AAr> pre_logitscXU(aURXS9nU$URU5nU$)N)r)rW)r9rgrs r< forward_headTinyVit.forward_heads/3=DIIaI /DH99Q<r>cJURU5nURU5nU$re)rrrfs r<rhTinyVit.forwards'  ! !! $   a r>) rArSrOrWrVr=r r>rUrNrPrRrFre)NFFrF)rFT)$rOrPrQrRr0rrSrrrTrr rlr/rXrar5rignorerirnrtrxr|rrrr rrrrrrhrVrWrXs@r<rrs#$*=&2)7,9!!$'#(),#$)+#Y-Y-Y- Y- c3h Y- #s(O Y-S#XY- S/Y-Y-Y-"Y-!Y-"'Y-!Y-BIIY-Y-vII!ryy!t! YY$$ YYPP YY YY)) YY *   $r>c6SUR5;aUSnUR5n0nUR5HYupEURS5(aMSU;a2[ UR X$R SSS25R nXSU'M[ U$)Nmodelrrr)rmrlitemsendswithrTrF)rlr target_sdout_dictrrs r<checkpoint_filter_fnrs*//##(   "IH  " ::+ , ,   "/Y\5G5G"5MNPPA # Or>c 0US[[SSSSSSS. UE$) Nr2zpatch_embed.conv1.convzhead.fc)rr)r_rgffffff?z apache-2.0) urlr>meanrd first_conv classifier pool_size input_sizecrop_pctlicenser )rkwargss r<_cfgrs4%#.#    r>ztimm/iQU) hf_hub_idr>)r)r_r6r6)r9r9g?)rrrr)r_r)rsquash)rrrr crop_mode) ztiny_vit_5m_224.dist_in22kz"tiny_vit_5m_224.dist_in22k_ft_in1kztiny_vit_5m_224.in1kztiny_vit_11m_224.dist_in22kz#tiny_vit_11m_224.dist_in22k_ft_in1kztiny_vit_11m_224.in1kztiny_vit_21m_224.dist_in22kz#tiny_vit_21m_224.dist_in22k_ft_in1kztiny_vit_21m_224.in1kz#tiny_vit_21m_384.dist_in22k_ft_in1kz#tiny_vit_21m_512.dist_in22k_ft_in1kc lURSS5n[[UU4[SUS9[S.UD6nU$)N out_indices)rrr@r_T)flatten_sequentialr) feature_cfgpretrained_filter_fn)poprrrTr)variant pretrainedrrrs r<_create_tiny_vitr$sJ**]L9K  DkJ1     E Lr>c h[/SQ/SQ/SQ/SQSS9nURU5 [SU40UD6$)N)@i@r7)r@r^ r;rtr@rArrBrDtiny_vit_5m_224rTupdaterrr model_kwargss r<rr1s>&" L -z J\ JJr>c h[/SQ/SQ/SQ/SQSS9nURU5 [SU40UD6$)N)rrir7)r@r^rrr;r<rtiny_vit_11m_224rrs r<rr>s>&" L . Kl KKr>c h[/SQ/SQ/SQ/SQSS9nURU5 [SU40UD6$)Nr4r5r6i@r7r_r8r9r;g?rtiny_vit_21m_224rrs r<rrKs>& " L . Kl KKr>c h[/SQ/SQ/SQ/SQSS9nURU5 [SU40UD6$)Nrr7r)r9r9r:r9r<rtiny_vit_21m_384rrs r<rrX>& % L . Kl KKr>c h[/SQ/SQ/SQ/SQSS9nURU5 [SU40UD6$)Nrr7r)rr rr<rtiny_vit_21m_512rrs r<rrerr>)rr)?r'__all__r functoolsrtypingrrrrr r r r5torch.nnr0torch.nn.functional functionalr timm.datar r timm.layersrrrrrrr_builderr _featuresr _features_fxr _manipulaterr _registryrrrrrlrZrnrrrrrr)rrr default_cfgsrrrrrrr r>r<rsN +@@@ A^^^*+13<!uxx""!H. RYY F2994 @biiBjjZhM299hMV %I9299I9XPbiiPf   %"&# +/+!$($ ,0,"$($ ,0,",0 Hs, ,0 Hsh,[2&2 j  K K L L L L L L L Lr>