RЦiSrSSKrSSKJr SSKJrJrJrJrJ r J r SSK r SSK J r SSKJrJr SSKJrJrJrJrJr SSKJr SS KJr SS KJr SS KJr SS KJ r J!r!J"r" SS K#J$r$ S/r%\"SS\ RL55r'SSjr("SS\ RL5r)"SS\ RL5r*"SS\ RL5r+"SS\ RL5r,"SS\ RL5r-"SS\ RL5r.Sr/SSjr0SSjr1\!"0S \1"S!S"S#9_S$\1"S!S%S#9_S&\1"S!S'S(S)9_S*\1"S!S+S#9_S,\1"S!S-S#9_S.\1"S!S/S(S)9_S0\1"S!S1S#9_S2\1"S!S3S#9_S4\1"S!S5S(S)9_S6\1"S!S7S#9_S8\1"S!S9S#9_S:\1"S!S;S(S)9_S<\1"S!S=S#9_S>\1"S!S?S#9_S@\1"S!SAS(S)9_SB\1"S!SCS#9_SD\1"S!SES#9_0SF\1"S!SGS(S)9_SH\1"S!SIS#9_SJ\1"S!SKS#9_SL\1"S!SMS(S)9_SN\1"S!SOS#9_SP\1"S!SQS#9_SR\1"S!SSS(S)9_ST\1"S!SUS#9_SV\1"S!SWS#9_SX\1"S!SYS(S)9_SZ\1"S!S[S#9_S\\1"S!S]S#9_S^\1"S!S_S(S)9_S`\1"S!SaS#9_Sb\1"S!ScS#9_Sd\1"S!SeS(S)9_Sf\1"S!SgS#9_E\1"S!ShS#9\1"S!SiS(S)9\1"S!SjS#9\1"S!SkS#9\1"S!SlS(S)9\1"S!SmS#9\1"S!SnS#9\1"S!SoS(S)9Sp.E5r2\ SSq\.4Srjj5r3\ SSq\.4Ssjj5r4\ SSq\.4Stjj5r5\ SSq\.4Sujj5r6\ SSq\.4Svjj5r7\ SSq\.4Swjj5r8\ SSq\.4Sxjj5r9\ SSq\.4Syjj5r:\ SSq\.4Szjj5r;\ SSq\.4S{jj5r<\ SSq\.4S|jj5r=\ SSq\.4S}jj5r>\ SSq\.4S~jj5r?\ SSq\.4Sjj5r@\ SSq\.4Sjj5rA\ SSq\.4Sjj5rB\ SSq\.4Sjj5rC\ SSq\.4Sjj5rD\ SSq\.4Sjj5rE\ SSq\.4Sjj5rF\ SSq\.4Sjj5rG\ SSq\.4Sjj5rH\ SSq\.4Sjj5rI\ SSq\.4Sjj5rJ\ SSq\.4Sjj5rK\ SSq\.4Sjj5rL\ SSq\.4Sjj5rM\ SSq\.4Sjj5rN\""\O0SS$_SS&_SS,_SS._SS2_SS4_SS8_SS:_SS>_SS@_SSD_SSF_SSJ_SSL_SSP_SSR_SSV_SXS\S^SbSdSSSSSSS. E5 g)a\Cross-Covariance Image Transformer (XCiT) in PyTorch Paper: - https://arxiv.org/abs/2106.09681 Same as the official implementation, with some minor adaptations, original copyright below - https://github.com/facebookresearch/xcit/blob/master/xcit.py Modifications and additions for timm hacked together by / Copyright 2021, Ross Wightman N)partial)ListOptionalTupleUnionTypeAnyIMAGENET_DEFAULT_MEANIMAGENET_DEFAULT_STD)DropPath trunc_normal_ to_2tupleuse_fused_attnMlp)build_model_with_cfg)feature_take_indices)register_notrace_module) checkpoint)register_modelgenerate_default_cfgsregister_model_deprecations) ClassAttnXcitc`^\rSrSrSrS S\S\S\4U4SjjjrS\S\S \4S jrS r U=r $) PositionalEncodingFourier!z Positional encoding relying on a fourier kernel matching the one used in the "Attention is all you Need" paper. Based on the official XCiT code - https://github.com/facebookresearch/xcit/blob/master/xcit.py hidden_dimdim temperaturec>XES.n[TU]5 [R"US-U4SS0UD6UlS[ R -UlX0lXl X l SUl g)Ndevicedtype kernel_sizerư>) super__init__nnConv2dtoken_projectionmathpiscaler!rr eps)selfrr r!r$r%dd __class__s O/var/www/html/ai-image-ml/venv/lib/python3.13/site-packages/timm/models/xcit.pyr*"PositionalEncodingFourier.__init__)s` /  " *q.# S1 SPR S[ &$BHWc URRRnURRRn[R "SUS-US9R [R5RS5RSSU5n[R "SUS-US9R [R5RSUS5nXfSS2SS2SS24UR-- UR-nXwSS2SS2SS24UR-- UR-n[R "URUS9R [R5nURS[R"USSS9-UR- -nUSS2SS2SS2S4U- n USS2SS2SS2S4U- n [R"U SS2SS2SS2SSS24R!5U SS2SS2SS2SSS24R#5/SS 9R%S 5n [R"U SS2SS2SS2SSS24R!5U SS2SS2SS2SSS24R#5/SS 9R%S 5n [R&"X4S S 9R)SS SS5n URU R U55n U RUSSS5$) Nr)r$r&floor) rounding_moderr )r-weightr$r%torcharangetofloat32 unsqueezerepeatr1r0rr!divstacksincosflattencatpermute) r2r8r9r:r$r%y_embedx_embeddim_tpos_xpos_yposs r5forward!PositionalEncodingFourier.forward:s{&&--44%%,,22,,q!a%7::5==ISSTUV]]^_abdef,,q!a%7::5==IPPQRTUWXYQQY/$((:;djjHQ23Y/$((:;djjH T__V<?? N  Q5!7)S%SVZVeVe%ef1a &.1a &. U1aADqD=1557q!Q1}9M9Q9Q9STZ[\ddefg U1aADqD=1557q!Q1}9M9Q9Q9STZ[\ddefgiiA.66q!QB##CFF5M2zz!Q1%%r7)r r1rr0r!r-) i'NN) __name__ __module__ __qualname____firstlineno____doc__intfloatr*rV__static_attributes__ __classcell__r4s@r5rr!s^!!&  "&&&&&r7rc X4S.n[RR[R"X4SUSSS.UD6[R"U40UD65$)z3x3 convolution + batch normr#rArF)r'stridepaddingbias)rCr+ Sequentialr, BatchNorm2d) in_planes out_planesrer$r%r3s r5conv3x3rlLsP +B 88   )cQvqW\c`bc z(R( r7c ^\rSrSrSrSSSS\R SS4S\S \S \S \S \\R4 U4S jjjr Sr Sr U=r $)ConvPatchEmbedUzXgS.n[T U]5 [U5nUSU-USU--n XlX lXlUS:Xa{[ RR[X4S-S40UD6U"5[US-US-S40UD6U"5[US-US-S40UD6U"5[US-US40UD65Ul gUS:Xab[ RR[X4S-S40UD6U"5[US-US-S40UD6U"5[US-US40UD65Ul gSe) Nr#rrrqr&r?z=For convolutional projection, patch size has to be in [8, 16]) r)r*rrrrs num_patchesrCr+rhrlproj) r2rrrsrtrurvr$r%r3ryr4s r5r*ConvPatchEmbed.__init__XsH / X&{j0Xa[J5NO  $&  ++q.!:r:  Q Q@R@  Q Q@R@  Q 1;;DI1_++q.!:r:  Q Q@R@  Q 1;; DIR Rr7cURU5nURSURSp2URS5RSS5nXU44$)Nr&rAr)rzshaperM transpose)r2xHpWps r5rVConvPatchEmbed.forwardsL IIaLQWWQZB IIaL " "1a (r({r7)rrryrsrz)rZr[r\r]r^r+GELUr_rModuler*rVrarbrcs@r5rnrnUsuF   )+%S%S%S %S  %S BII %S%SNr7rnc ^\rSrSrSrS\R SSS4S\S\\S\ \RS\4U4S jjjr S \S \4S jr S r U=r$)LPIa  Local Patch Interaction module that allows explicit communication between tokens in 3x3 windows to augment the implicit communication performed by the block diagonal scatter attention. Implemented using 2 layers of separable 3x3 convolutions with GeLU and BatchNorm2d NrA in_features out_featuresrvr'c>>[T U]5 XVS.nU=(d UnUS-n[RR"X4XHUS.UD6UlU"5Ul[R"U40UD6Ul[RR"X4XHUS.UD6Ul g)Nr#r&)r'rfgroups) r)r*rCr+r,conv1actribnconv2) r2rrrvr'r$r%r3rfr4s r5r* LPI.__init__s  /#2{ "XX__ j2=Wbjfhj ;..33XX__ l3>Xdlhjl r7r9r:c8URupEnURSSS5RXFX#5nURU5nUR U5nUR U5nUR U5nURXFU5RSSS5nU$)Nrr&r)r}rOreshaperrrr)r2rr9r:r8NCs r5rV LPI.forwards''a IIaA  & &qQ 2 JJqM HHQK GGAJ JJqM IIaA  & &q!Q /r7)rrrr)rZr[r\r]r^r+rr_rrrr*rVrarbrcs@r5rrsw+/)+ ll#3-lBII l  ll*CCr7rc^\rSrSrSrSSSSS\R \RSSSS4 S\S \S \ S \ S \ S \ S\ S\ \RS\ \RS\ \ S\ 4U4SjjjrSrSrU=r$)ClassAttentionBlockzAClass Attention Layer as in CaiT https://arxiv.org/abs/2103.17239@F?Nr num_heads mlp_ratioqkv_bias proj_drop attn_drop drop_pathrv norm_layereta tokens_normcx>XS.n[TU]5 U "U40UD6Ul[U4UUUUS.UD6UlUS:a [ U5O[ R"5UlU "U40UD6Ul [SU[X-5UUS.UD6Ul US:a [ U5O[ R"5Ul U bg[ R"U [R "U40UD6-5Ul[ R"U [R "U40UD6-5UlOSuUlUlXlg)Nr#rrrrrrhidden_featuresrvdrop)rr)r)r*norm1rattnr r+Identity drop_path1norm2rr_mlp drop_path2 ParameterrConesgamma1gamma2r)r2r rrrrrrrvrrrr$r%r3r4s r5r*ClassAttentionBlock.__init__s+  / *r*        2;R(9-R[[]*r*  0    2;R(9-R[[] ?,,sUZZ-Br-B'BCDK,,sUZZ-Br-B'BCDK'/ $DK'r7c6URU5n[R"URU5USS2SS24/SS9nXR UR U-5-nUR (aURU5nO8[R"URUSS2SS245USS2SS24/SS9nUnUSS2SS24nURURU5-n[R"XQSS2SS24/SS9nX@RU5-nU$)Nrr@r) rrCrNrrrrrrrr)r2rx_norm1x_attnx_res cls_tokens r5rVClassAttentionBlock.forwards**Q-DIIg.12?QG  f 45 5    1 A 4::a1Q3i0!AqrE(;CAa1fI KK$((9"55 IIyAqrE(+ 3 OOA& &r7) rrrrrrrrr)rZr[r\r]r^r+r LayerNormr_r`boolrrrr*rVrarbrcs@r5rrsK ""!!!)+*,,,#% %.'.'.' .'  .'  .'.'.'BII.'RYY.'%.'.'.'`r7rc ^\rSrSr%\R R \\S'S S\ S\ S\S\ S\ 4 U4Sjjjr S r \R RS 5rS rU=r$) XCA fused_attnr rrrrc>XgS.n[T U]5 X l[SS9Ul[ R "[R"USS40UD65Ul [ R"XS-4SU0UD6Ul [ R"U5Ul [ R"X40UD6Ul[ R"U5Ulg)Nr#T) experimentalrrArg)r)r*rrrr+rrCrr!LinearqkvDropoutrrzr) r2r rrrrr$r%r3r4s r5r* XCA.__init__s / "(d;<< 9a(Ib(IJ99S'??B?I.IIc-"- I.r7cURup#nURU5RX#SURX@R-5R SSSSS5nUR S5upgnUR (a[RRRUSS9UR-n[RRRUSS9n[RRRXgUSS 9nO[RRRUSS9n[RRRUSS9nXgRS S5-UR-n U RSS9n URU 5n X-nUR SSSS5RX#U5nUR!U5nUR#U5nU$) NrAr&rr?rr<r@r)r0)r}rrrrOunbindrrCr+ functional normalizer!scaled_dot_product_attentionr~softmaxrrzr) r2rr8rrrqkvrs r5rV XCA.forward s''ahhqk!!!4>>1;NOWWXY[\^_abdef**Q-a ??##--aR-84;K;KKA##--aR-8A##@@qPS@TA##--aR-8A##--aR-8A B++t/?/??D<>$'DA IIaAq ! ) )! 2 IIaL NN1 r7cS1$)Nr!rr2s r5no_weight_decayXCA.no_weight_decay$s r7)rrrrzrrr!)rxFrrNN)rZr[r\r]rCjitFinalr__annotations__r_r`r*rVignorerrarbrcs@r5rrs %%"!!/// /  /  //(0 YYr7rc^\rSrSrSSSSS\R \R SSS4 S\S\S \S \ S \S \S \S\ \RS\ \RS\4U4Sjjjr S\S\4Sjr SrU=r$)XCABlocki)rFrrNr rrrrrrrvrrc B>XS.n [TU]5 U "U40U D6Ul[U4UUUUS.U D6UlUS:a [ U5O[ R"5UlU "U40U D6Ul [SXS.U D6Ul US:a [ U5O[ R"5Ul U "U40U D6Ul [SU[X-5UUS.U D6UlUS:a [ U5O[ R"5Ul[ R$"U [&R("U40U D6-5Ul[ R$"U [&R("U40U D6-5Ul[ R$"U [&R("U40U D6-5Ulg)Nr#rr)rrvrr)r)r*rrrr r+rrnorm3rlocal_mp drop_path3rrr_rrrrCrrgamma3r)r2r rrrrrrrvrrr$r%r3r4s r5r*XCABlock.__init__*sr / *r*        2;R(9-R[[]*r* GGBG 1:R(9-R[[]*r*  0    2;R(9-R[[]ll3C)>2)>#>? ll3C)>2)>#>? ll3C)>2)>#>? r7r9r:c |XRURURURU55-5-nXR UR UR URU5X#5-5-nXRURURURU55-5-nU$N) rrrrrrrrrrrr)r2rr9r:s r5rVXCABlock.forwardXs  dii 1 .F FG G  dmmDJJqM1.P PQ Q  dhhtzz!}.E EF Fr7) rrrrrrrrrrrr)rZr[r\r]r+rrr_r`rrrr*rVrarbrcs@r5rr)s ""!!!)+*,,,,@,@,@ ,@  ,@  ,@,@,@BII,@RYY,@,@,@\CCr7rc+8^\rSrSrSrS/S\\\\\44S\S\S\S\S\S \S \S \ S \ S \ S\ S\ S\ S\ S\ \ \ RS\ \ \ RS\S\ S\ S\ 4*U4SjjjrSr\R$R&S5r\R$R&S0Sj5r\R$R&S1Sj5r\R$R&S\ R4Sj5rS2S\S\ \4SjjrS3S \R2S!\ \\\\4S"\ S#\ S$\S%\ S\\\R2\\R2\\R2444S&jjrS4S!\\\\4S'\ S(\ 4S)jjrS*rS0S+\ 4S,jjrS-rS.r U=r!$)5riaz Based on timm and DeiT code bases https://github.com/rwightman/pytorch-image-models/tree/master/timm https://github.com/facebookresearch/deit/ rrrsrt num_classes global_poolrudepthrrr drop_rate pos_drop_rateproj_drop_rateattn_drop_ratedrop_path_ratervrcls_attn_layers use_pos_embedrrc8>[TU]5 UUS.nUS;de[U5nUSU-S:Xa USU-S:XdS5eU=(d [[R SS9nU=(d [R nX@lX0lU=Ul =Ul Ul XPl SUl [SUUUUUS.UD6UlUn[R "["R$"S S U40UD65UlU(a[)SS U0UD6UlOS Ul[R,"U S 9Ul[R0"[3U5Vs/sHn[5SUUU U U UUUUUS . UD6PM sn5Ul5Vs/sHnUSU3S9PM snUl[R0"[3U5Vs/sHn[=SUUU U U UUUUUS. UD6PM sn5UlU"U40UD6Ul [R,"U 5Ul!US:a"[RD"URU40UD6O[RF"5Ul$[KUR&SS9 URMURN5 g s snfs snfs snf)a Args: img_size (int, tuple): input image size patch_size (int): patch size in_chans (int): number of input channels num_classes (int): number of classes for classification head embed_dim (int): embedding dimension depth (int): depth of transformer num_heads (int): number of attention heads mlp_ratio (int): ratio of mlp hidden dim to embedding dim qkv_bias (bool): enable bias for qkv if True drop_rate (float): dropout rate after positional embedding, and in XCA/CA projection + MLP pos_drop_rate: position embedding dropout rate proj_drop_rate (float): projection dropout rate attn_drop_rate (float): attention dropout rate drop_path_rate (float): stochastic depth rate (constant across all layers) norm_layer: (nn.Module): normalization layer cls_attn_layers: (int) Depth of Class attention layers use_pos_embed: (bool) whether to use positional encoding eta: (float) layerscale initialization value tokens_norm: (bool) Whether to normalize all tokens or just the cls_token in the CA Notes: - Although `layer_norm` is user specifiable, there are hard-coded `BatchNorm2d`s in the local patch interaction (class LPI) and the patch embedding (class ConvPatchEmbed) r#avgtokenrz2`patch_size` should divide image dimensions evenlyr()r1F)rrrsrtrurvrr N)p) r rrrrrrrvrrzblocks.)num_chs reductionmodule) r rrrrrrvrrr{Gz?stdr)(r)r*rrr+rrrrt num_featureshead_hidden_sizerurgrad_checkpointingrn patch_embedrrCzerosrr pos_embedrpos_drop ModuleListrangerblocksdict feature_inforcls_attn_blocksnorm head_droprrheadrapply _init_weights)r2rrrsrtrrrurrrrrrrrrrvrrrrrr$r%r3r_ir4s r5r* Xcit.__init__hsh  /2222X& j(A-HQK*4LPQ4Q A @ ARB72<!!!&&!,-?' $r7c SS1$)Nrrrrs r5rXcit.no_weight_decays [))r7c[SSSS/S9$)Nz ^cls_token|pos_embed|patch_embedz^blocks\.(\d+))z^cls_attn_blocks\.(\d+)N)z^norm)i)stemrr )r)r2coarses r5 group_matcherXcit.group_matchers4$?AUV  r7cXlgr)r)r2enables r5set_grad_checkpointingXcit.set_grad_checkpointings"(r7returncUR$r)r rs r5get_classifierXcit.get_classifiers yyr7cXlUbUS;deX l[URS5(a URRR OSn[URS5(a URRR OSnUS:a&[R"URXUS9Ulg[R"5Ulg)NrrBrr#) rrhasattrr rBr$r%r+rrr)r2rrr$r%s r5reset_classifierXcit.reset_classifiers&  ""66 66* ,3DIIx,H,H!!((d*1$))X*F*F   &&D]hkl]lBIId//SXY rtr}r}r r7rindicesr  stop_early output_fmtintermediates_onlyc *US;dS5eUS:Hn/n[[UR5U5upURuppUR U5ununnUR bFUR XU5R U SURS5RSSS5nUU-nURU5n[RR5(dU(d URnOURSU S-n[U5HunnUR(a1[RR5(d[UXU5nO U"XU5nUU ;dMYURU(aUR!U5OU5 M U(aCUVs/sH6nUR XUS5RSS SS5R#5PM8 nnU(aU$[R$"UR&R)U SS5U4SS 9nUR*HMnUR(a1[RR5(d[UU5nMEU"U5nMO UR!U5nX4$s snf) aForward features that returns intermediates. Args: x: Input image tensor indices: Take last n blocks if int, all if None, select matching indices if sequence norm: Apply norm layer to all intermediates stop_early: Stop iterating over blocks when last desired intermediate hit output_fmt: Shape of intermediate feature outputs intermediates_only: Only return intermediate features Returns: )NCHWNLCz)Output format must be one of NCHW or NLC.r2Nr<rrr&rAr@)rlenrr}rrrrOrrCr is_scripting enumeraterrappendr  contiguousrNrexpandr )r2rr-r r.r/r0r intermediates take_indices max_indexr8rheightwidthrr pos_encodingrrblkys r5forward_intermediatesXcit.forward_intermediates s1*_,Y.YY,& "6s4;;7G"Q  ggf&&q) 8B >> %>>!4<> %>>!4<* "81$1 r7c SU;aUSn[USS5SLnUVs/sHo3RS5(dMUPM nnUH1nU(a%URU5XRSS5'M/X M3 SU;aSUR 5;a[ UR 5n[U5HnURSUS35nURS S URS 5n[S 5HupXxUSUS U S 3'M URSUS3S5n U cMuU RS S 5n [S 5HupXUSUS U S3'M M U$s snf)Nmodelrz pos_embeder.z pos_embed.z!cls_attn_blocks.0.attn.qkv.weightzcls_attn_blocks.0.attn.q.weightzcls_attn_blocks.z.attn.qkv.weightrAr<rz.attn.z.weightz.attn.qkv.biasz.bias) getattr startswithpopreplace state_dictr4r rrr}r6) r^rYrrpos_embed_keys num_ca_blocksr qkv_weightj subscriptrs r5checkpoint_filter_fnrds*( E;5TAM!+IA||K/HaNI  BL..QRBSJyy> ? +j8=^bgbrbrbt=tE112 }%A#*:1#=M(NOJ#++Ar:3C3CB3GHJ )% 0 MW] -aSykIJ!1!~~(8>&JDQH##++Ar2$-e$4LAOW{J!1!F9+UKL%5& )Js E$E$c lURSS5n[[UU4[[ USS9S.UD6nU$)N out_indicesrAgetter)rf feature_cls)pretrained_filter_fn feature_cfg)r\rrrdr)variant pretrained default_cfgkwargsrfrYs r5 _create_xcitrosJ**]A.K   2[hG     E Lr7c 4USSSSSS[[SSSS . UE$) NrS)rArprprbicubicTzpatch_embed.proj.0.0r z apache-2.0) urlr input_size pool_sizecrop_pct interpolationfixed_input_sizerNr first_conv classifierlicenser )rrrns r5_cfgr{s5=t)%.B,F   $*  r7zxcit_nano_12_p16_224.fb_in1kztimm/zhttps://dl.fbaipublicfiles.com/xcit/xcit_medium_24_p16_224.pthz#xcit_medium_24_p16_224.fb_dist_in1kzChttps://dl.fbaipublicfiles.com/xcit/xcit_medium_24_p16_224_dist.pthz#xcit_medium_24_p16_384.fb_dist_in1kzChttps://dl.fbaipublicfiles.com/xcit/xcit_medium_24_p16_384_dist.pthzxcit_large_24_p16_224.fb_in1kz=https://dl.fbaipublicfiles.com/xcit/xcit_large_24_p16_224.pthz"xcit_large_24_p16_224.fb_dist_in1kzBhttps://dl.fbaipublicfiles.com/xcit/xcit_large_24_p16_224_dist.pthz"xcit_large_24_p16_384.fb_dist_in1kzBhttps://dl.fbaipublicfiles.com/xcit/xcit_large_24_p16_384_dist.pthzxcit_nano_12_p8_224.fb_in1kz;https://dl.fbaipublicfiles.com/xcit/xcit_nano_12_p8_224.pthz xcit_nano_12_p8_224.fb_dist_in1kz@https://dl.fbaipublicfiles.com/xcit/xcit_nano_12_p8_224_dist.pthz xcit_nano_12_p8_384.fb_dist_in1kz@https://dl.fbaipublicfiles.com/xcit/xcit_nano_12_p8_384_dist.pthzxcit_tiny_12_p8_224.fb_in1kz;https://dl.fbaipublicfiles.com/xcit/xcit_tiny_12_p8_224.pthz xcit_tiny_12_p8_224.fb_dist_in1kz@https://dl.fbaipublicfiles.com/xcit/xcit_tiny_12_p8_224_dist.pthz xcit_tiny_12_p8_384.fb_dist_in1kz@https://dl.fbaipublicfiles.com/xcit/xcit_tiny_12_p8_384_dist.pthzxcit_tiny_24_p8_224.fb_in1kz;https://dl.fbaipublicfiles.com/xcit/xcit_tiny_24_p8_224.pthz xcit_tiny_24_p8_224.fb_dist_in1kz@https://dl.fbaipublicfiles.com/xcit/xcit_tiny_24_p8_224_dist.pthz xcit_tiny_24_p8_384.fb_dist_in1kz@https://dl.fbaipublicfiles.com/xcit/xcit_tiny_24_p8_384_dist.pthzxcit_small_12_p8_224.fb_in1kzB!RVXJ  fZ f4PZKe^dKe fE Lr7c P[SSSSSSS9n[S SU0[U40UD6D6nU$) Nrqr}rTrxrTrrl)xcit_small_12_p16_384rrs r5rrhrr7c P[SSSSSSS9n[S SU0[U40UD6D6nU$) Nrqrr?h㈵>Trrl)xcit_tiny_24_p16_224rrs r5rrp=B!SWYJ  eJ e$zJd]cJd eE Lr7c P[SSSSSSS9n[S SU0[U40UD6D6nU$) Nrqrrr?rTrrl)xcit_tiny_24_p16_384rrs r5rrxrr7c P[SSSSSSS9n[S SU0[U40UD6D6nU$) Nrqr}rrxrTrrl)xcit_small_24_p16_224rrs r5rr>B!SWYJ  fZ f4PZKe^dKe fE Lr7c P[SSSSSSS9n[S SU0[U40UD6D6nU$) Nrqr}rrxrTrrl)xcit_small_24_p16_384rrs r5rrrr7c P[SSSSSSS9n[S SU0[U40UD6D6nU$) NrqrrxrTrrl)xcit_medium_24_p16_224rrs r5rr>B!SWYJ  gj gDQ[Lf_eLf gE Lr7c P[SSSSSSS9n[S SU0[U40UD6D6nU$) NrqrrrxrTrrl)xcit_medium_24_p16_384rrs r5rrrr7c P[SSSSSSS9n[SSU0[U40UD6D6nU$) NrqrYrrTrrl)xcit_large_24_p16_224rrs r5rr>B"$TXZJ  fZ f4PZKe^dKe fE Lr7c P[SSSSSSS9n[SSU0[U40UD6D6nU$) NrqrYrrTrrl)xcit_large_24_p16_384rrs r5rrrr7c P[SSSSSSS9n[S SU0[U40UD6D6nU$) NrxrrTr?rFrrl)xcit_nano_12_p8_224rrs r5rr=2QVXJ  d: djIc\bIc dE Lr7c P[SSSSSSS9n[S SU0[U40UD6D6nU$) NrxrrTr?rFrrl)xcit_nano_12_p8_384rrs r5rrrr7c P[SSSSSSS9n[S SU0[U40UD6D6nU$) NrxrrTr?rTrrl)xcit_tiny_12_p8_224rrs r5rr=2QUWJ  d: djIc\bIc dE Lr7c P[SSSSSSS9n[S SU0[U40UD6D6nU$) NrxrrTr?rTrrl)xcit_tiny_12_p8_384rrs r5rrrr7c P[SSSSSSS9n[SSU0[U40UD6D6nU$) Nrxr}rTrTrrl)xcit_small_12_p8_224rrs r5rr=2QUWJ  eJ e$zJd]cJd eE Lr7c P[SSSSSSS9n[SSU0[U40UD6D6nU$) Nrxr}rTrTrrl)xcit_small_12_p8_384rrs r5rrrr7c P[SSSSSSS9n[S SU0[U40UD6D6nU$) Nrxrrr?rTrrl)xcit_tiny_24_p8_224rrs r5rr=2RVXJ  d: djIc\bIc dE Lr7c P[SSSSSSS9n[S SU0[U40UD6D6nU$) Nrxrrr?rTrrl)xcit_tiny_24_p8_384rrs r5rrrr7c P[SSSSSSS9n[SSU0[U40UD6D6nU$) Nrxr}rrTrrl)xcit_small_24_p8_224rrs r5rr=2RVXJ  eJ e$zJd]cJd eE Lr7c P[SSSSSSS9n[SSU0[U40UD6D6nU$) Nrxr}rrTrrl)xcit_small_24_p8_384rrs r5rrrr7c P[SSSSSSS9n[SSU0[U40UD6D6nU$) NrxrrrTrrl)xcit_medium_24_p8_224rrs r5rr>2RVXJ  fZ f4PZKe^dKe fE Lr7c P[SSSSSSS9n[SSU0[U40UD6D6nU$) NrxrrrTrrl)xcit_medium_24_p8_384rrs r5rr rr7c P[SSSSSSS9n[S SU0[U40UD6D6nU$) NrxrYrrqrTrrl)xcit_large_24_p8_224rrs r5rr=2SWYJ  eJ e$zJd]cJd eE Lr7c P[SSSSSSS9n[S SU0[U40UD6D6nU$) NrxrYrrqrTrrl)xcit_large_24_p8_384rrs r5rrrr7xcit_nano_12_p16_224_distxcit_nano_12_p16_384_distxcit_tiny_12_p16_224_distxcit_tiny_12_p16_384_distxcit_tiny_24_p16_224_distxcit_tiny_24_p16_384_distxcit_small_12_p16_224_distxcit_small_12_p16_384_distxcit_small_24_p16_224_distxcit_small_24_p16_384_distxcit_medium_24_p16_224_distxcit_medium_24_p16_384_distxcit_large_24_p16_224_distxcit_large_24_p16_384_distxcit_nano_12_p8_224_distxcit_nano_12_p8_384_distxcit_tiny_12_p8_224_distr~rrrrr) xcit_tiny_12_p8_384_distxcit_tiny_24_p8_224_distxcit_tiny_24_p8_384_distxcit_small_12_p8_224_distxcit_small_12_p8_384_distxcit_small_24_p8_224_distxcit_small_24_p8_384_distxcit_medium_24_p8_224_distxcit_medium_24_p8_384_distxcit_large_24_p8_224_distxcit_large_24_p8_384_dist)rNN)FN)rrU)Pr^r. functoolsrtypingrrrrrr rCtorch.nnr+ timm.datar r timm.layersr rrrr_builderr _featuresr _features_fxr _manipulater _registryrrrcaitr__all__rrrlrnrrrrrrdror{ default_cfgsrrrrrrrrrrrrrrrrrrrrrrrrrrrrrZrr7r5rs8   :: AOO*+1#YY ('& '&'&T.RYY.b$"))$NA"))AH5"))5p5ryy5pb299bJ : %B&"D J%LB& ( O*Q B&( O\i*kB&#D J%LB&( O*QB&"( O\i*k#B&(#D J%L)B&.( O*Q/B&4( O\i*k5B&:$T K&M;B&@)$ P+RAB&F)$ P]j+lGB&L$T K&MMB&R)$ P+RSB&X)$ P]j+lYB&^%d L'N_B&d*4 Q,SeB&j*4 Q^k,mkB&p$T K&MqB&v)$ P+RwB&|)$ P]j+l}B&F"4 I$KGB&L' N)PMB&R' N[h)jSB&X"4 I$KYB&^' N)P_B&d' N[h)jeB&j"4 I$KkB&p' N)PqB&v' N[h)jwB&|#D J%L}B&B( O*QCB&H( O\i*kIB&N#D J%LOB&T*. O*Q*. O\i*k&* K&M+/ P+R+/ P]j+l%) J%L*. O*Q*. O\i*kB&B J$$ttttttH '!D ' !D ' !D '  !D '  !D ' !D '!"F '!"F '!"F '!"F '"#H '"#H '!"F '!"F '$ B% '& B' '( B) '*!C B B!D!D!D!D"F"F!D!D? ' r7