RЦihvSrSSKrSSKJrJrJrJr SSKrSSKJ s J r SSKJ r SSK J r JrJrJrJrJr SSKJr SSKJr SS KJrJrJr SS KJrJrJrJrJ r J!r! SS K"J#r$ /r%SgS jr&ShSjr'SiSjr(SjSjr)\*"\"\&"SSSSS9\&"SSSSS9\'"SSSSSSSS9\'"SSSSS SSS9\'"SSSSSSSS94SSSSS S!S"9\"\&"SS#SS$9\&"SSSS$9\'"SSSSSSS%9\'"SSSS&S SS%9\'"SSSS'SSS%94SSSSS S(S"9\"\&"SS#SS$9\&"SSSS$9\'"SSSS'SSS%9\'"SS)SS*S SS%9\'"SS+SS,SSS%94SSSSS S-S"9\"\&"SS#SS$9\&"SSSS$9\'"SSSS'SSS%9\'"SS)SS*S SS%9\'"SS+SS,SSS%94SSSSS.\*"S/S09S-S19\)"S5\)"S25\)"S35\)"S5\)"S45\)"S55\)"S5S69 r+\"S7S8\ RX55r-"S9S:\ RX5r."S;S<\ RX5r/\"S=S>\ RX55r0\"S?\-5 \"S@\05 SkSAjr1SkSBjr2SlSCjr3\"0SD\3"SESF9_SG\3"SESF9_SH\3"SESF9_SI\3"SESJSK9_SL\3"SESJSK9_SM\3"SESJSK9_SN\3"SESJSK9_SO\3"SESJSK9_SP\3"SESJSK9_SQ\3"SESJSK9_SR\3"SESJSK9_SS\3"SESJSK9_ST\3"SESJSK9_SU\3"SESVSWSSX9_SY\3"SESVSWSSX9_SZ\3"SESVSWSSX9_5r4\SmS[\4S\jj5r5\SmS[\4S]jj5r6\SmS[\4S^jj5r7\SmS[\4S_jj5r8\SmS[\4S`jj5r9\SmS[\4Sajj5r:\SmS[\4Sbjj5r;\SmS[\4Scjj5r<\SmS[\4Sdjj5r=\SmS[\4Sejj5r>\"\?SRSSSTSUSYSZSf.5 g)na MobileViT Paper: V1: `MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer` - https://arxiv.org/abs/2110.02178 V2: `Separable Self-attention for Mobile Vision Transformers` - https://arxiv.org/abs/2206.02680 MobileVitBlock and checkpoints adapted from https://github.com/apple/ml-cvnets (original copyright below) License: https://github.com/apple/ml-cvnets/blob/main/LICENSE (Apple open source) Rest of code, ByobNet, and Transformer block hacked together by / Copyright 2022, Ross Wightman N)CallableTupleOptionalType)nn) to_2tuplemake_divisible GroupNorm1ConvMlpDropPath is_exportable)build_model_with_cfg)register_notrace_module)register_modelgenerate_default_cfgsregister_model_deprecations)register_block ByoBlockCfg ByoModelCfgByobNetLayerFn num_groups)Blockc .[SXUSU[SSS9S9$)NbottlerT) bottle_in linear_out)typedcsgsbr block_kwargs)rdictr r!r"r$s T/var/www/html/ai-image-ml/venv/lib/python3.13/site-packages/timm/models/mobilevit.py_inverted_residual_blockr)!s$  1rDT: <<c B[XX&S9[SSUS[UUUS9S94$)Nr' mobilevitr)transformer_dimtransformer_depth patch_size)rr r!r"r%r)rr&r r!r"r.r/r0r$s r(_mobilevit_blockr3(s: !1Q6Q! /"3%'   r*@?cD[XX%S9[SSUSUS[UUS9S94$)Nr' mobilevit2r)r/r0)rr r!r"r$r#r%r1)r r!r"r/r0r$transformer_brs r(_mobilevitv2_blockr:6s; !1Q6a1A"3%'  r*?c .SnUS:wa'[UVs/sHn[X -5PM sn5n[[SUSSSS9[SUSSSS9[ SUSSSS9[ SUS SS S9[ SUS SS S94[S U-5S S S SS9nU$s snf)N)@ir;rrr5r'r4)r r!r"r/r+ 3x3silu)blocksstem_chs stem_type stem_pool downsample act_layer)tupleintrr)r:) multiplierchsr!cfgs r(_mobilevitv2_cfgrQCs "CS#6#QS(#67  $qCFaC @ $qCFaC @ c!fQ G c!fQ G c!fQ G  R*_% C J7sBr'rA0r=r2P`rCrDrEi@)rFrGrHrIrJrK num_featuresrB)r r!r")r r!r"r.r/r0xr@r>iseg?)rd_ratio)rFrGrHrIrJ attn_layer attn_kwargsrWg?g?g?g?) mobilevit_xxs mobilevit_xs mobilevit_s semobilevit_smobilevitv2_050mobilevitv2_075mobilevitv2_125mobilevitv2_100mobilevitv2_150mobilevitv2_175mobilevitv2_200c&&^\rSrSrSrSSSSSSSSS S S S S S S S\R SS4S\S\\S\S\S\ S\\S\ \\4S\ S\\S\S\S\S\ S\S\ S\ S\ S\ \R4$U4S jjjrS!\R"S"\R"4S#jrS$rU=r$)%MobileVitBlockzFMobileViT block Paper: https://arxiv.org/abs/2110.02178?context=cs.LG NrArr;rrr5r4r+Fin_chsout_chs kernel_sizestride bottle_ratio group_sizedilation mlp_ratior.r/r0 num_heads attn_dropdrop no_fusiondrop_path_ratelayerstransformer_norm_layerc >UUS.n[TU]5 U=(d [5n[Xa5nU=(d UnU =(d [ XQ-5n UR "UU4UUUUSS.UD6Ul[R"X4SSS.UD6Ul [R"[U 5Vs/sH"n[U 4UU SU UUURUS.UD6PM$ sn6UlU"U 40UD6UlUR "X4SSS .UD6UlU(aSUlOUR "X-U4USS .UD6Ul[%U 5UlUR&SUR&S-Ulgs snf) NdevicedtyperrtrugroupsrxrFrtbiasT)ryrzqkv_biasr{ proj_drop drop_pathrK norm_layerrtru)super__init__rrr conv_norm_actconv_kxkrConv2dconv_1x1 SequentialrangeTransformerBlockact transformernorm conv_proj conv_fusionrr0 patch_area)selfrrrsrtrurvrwrxryr.r/r0rzr{r|r}r~rrrrkwargsddr_ __class__s r(rMobileVitBlock.__init__s0 / $79J/#V)R^LXVS.n[TU]5 Xl[R"SUSSU--USS.UD6Ul[R "U5Ul[R"SUUUSS.UD6Ul[R "U5Ul g)Nrrr4) in_channels out_channelsrrt) rrrrrqkv_projDropoutr{out_projout_drop) rrr{rrrrrrs r(rLinearSelfAttention.__init__-s / "  !a)m,     I.  !"      9- r*rcURU5nURSURUR/SS9up4n[R"USS9nUR U5nXF-R SSS9n[R"U5URU5-nURU5nURU5nU$)NrrrTrkeepdim) rsplitrrsoftmaxr{sumrelu expand_asrr) rrqkvquerykeyvaluecontext_scorescontext_vectorouts r(_forward_self_attn&LinearSelfAttention._forward_self_attnKsmmA  IIq$..$..&IqIQE5b17.33D3IffUmn66u==mmC mmC  r*x_prevcHURup4pVURSSupxXW:XdS5e[R"UURRSUR S-URR SUR S-S9n U RSUR /SS9up[R"UURRUR S-URR b&URR UR S-OSS9n [R"U SS9n URU 5n X-RSSS9n[R"U 5URU 5-nURU5nURU5nU$) NzJThe number of pixels in a patch for query and key_value should be the samer)weightrrrTr)rrconv2drrrrrrr{rrrrr)rrr batch_sizein_dim kv_patch_areakv_num_patches q_patch_area q_num_patchesqkrrrrrrs r(_forward_cross_attn'LinearSelfAttention._forward_cross_attncs=>GG9 M&'ggbcl#   ) X W X ) XX ==''(;!);<##$7T^^a%78 XXq$..1qX9  ==''(:;;?==;M;M;Y##DNNQ$67_c 5b17.33D3IffUmn66u==mmC mmC  r*cHUcURU5$URXS9$)N)r)rr)rrrs r(rLinearSelfAttention.forwards, >**1- -++A+= =r*)r{rrrr)rqrqTNNN)rrrrrrMrrrrrrjitignorerrrrrrs@r(rrs, #"... .  . ..<ELLU\\0 YY(U\\(8ELL;Q(]b]i]i((T>>x /E>QVQ]Q]>>r*rc^\rSrSrSrSS\S\S\S\S\S \\\ RS \\\ RS S4U4S jjjr SS \ RS\\ RS \ R4SjjrSrU=r$)LinearTransformerBlockia This class defines the pre-norm transformer encoder with linear self-attention in `MobileViTv2 paper <>`_ Args: embed_dim (int): :math:`C_{in}` from an expected input of size :math:`(B, C_{in}, P, N)` mlp_ratio (float): Inner dimension ratio of the FFN relative to embed_dim drop (float): Dropout rate. Default: 0.0 attn_drop (float): Dropout rate for attention in multi-head attention. Default: 0.0 drop_path (float): Stochastic depth rate Default: 0.0 norm_layer (Callable): Normalization layer. Default: layer_norm_2d Shape: - Input: :math:`(B, C_{in}, P, N)` where :math:`B` is batch size, :math:`C_{in}` is input embedding dim, :math:`P` is number of pixels in a patch, and :math:`N` is number of patches, - Output: same shape as the input Nrryr|r{rrKrrc X>XS.n [T U]5 U=(d [RnU=(d [nU"U40U D6Ul[ SXUS.U D6Ul[U5Ul U"U40U D6Ul [SU[X-5UUS.U D6Ul [U5Ulg)Nr)rr{r) in_featureshidden_featuresrKr|r)rrrSiLUr norm1rattnr drop_path1norm2r rMmlp drop_path2) rrryr|r{rrKrrrrrs r(rLinearTransformerBlock.__init__s / ( -:  0R0 'g)\`gdfg "9- 0R0 ! 56    #9-r*rrcBUc2XRURURU555-nO8UnURU5nURX5nURU5U-nXRUR UR U555-nU$r)r r r rrr )rrrress r(rLinearTransformerBlock.forwards >OODIIdjjm$<==AC 1 A !$A"S(A A 78 8r*)r r rrr r )r5rqrqrqNNNNr)rrrrrrMrrrrrrrrrrrrs@r(rrs $ #""3748... .  .  . RYY0.!bii1. ..<  x /E QVQ]Q]  r*rc ^\rSrSrSrSSSSSSSS S S S S S\SS4S \S \\S\S\S\\S\ \\4S\S\\S\S\S\S\S\S\ S\ \ R4U4SjjjrS\R S\R 4SjrSrU=r$) MobileVitV2Blockiz0 This class defines the `MobileViTv2 block <>`_ NrAr;rror5r4rprqrrrsrtrvrwrxryr.r/r0r{r|r~rrc >UUS.n[TU]5 U=(d [5n[XQ5nU=(d UnU=(d [ XA-5nUR "UU4USUUSS.UD6Ul[R"X4SSS.UD6Ul [R"[U 5Vs/sH n[U4UU U U URUS.UD6PM" sn6UlU"U40UD6UlUR "X4SSSS.UD6Ul[#U 5UlUR$SUR$S-Ul[)5Ulgs snf) NrrrrFr)ryr{r|rrKr)rtru apply_act)rrrrr rrrrrrrrrrrrrr0rr coreml_exportable)rrrrsrtrvrwrxryr.r/r0r{r|r~rrrrrrrrrs r(rMobileVitV2Block.__init__sy* / $79J/#V)R^LURX#XgX5RSSS S SS5nURX#X-X-5nURU5nU$) NrTrrrrrAr4r+r)upscale_factor)rr0rrrrrrrunfoldrpermuterr pixel_shuffler) rrrrrrrrrrrrrs r(rMobileVitV2Block.forwardsWW a??yy-71;9ORY9Yu#(#3U5E[!/ : aen:UYZA MM!  MM!  GGAJ  ! !'(:GCUVA ! kKSSTUWXZ[]^`acdeA IIaB ,   Q  IIaL  ! ! ![72KMA:A !+KSSTUWXZ[]^`acdeA ! 5{7LMA NN1 r*)rrrrrrr0r)rrrrrr rMrrrrrrrrrrrrrrs@r(rrs&* "%()(."-1%&!$&"6@%:1:1c]:1 :1  :1 ! :1CHo:1:1&c]:1 #:1:1:1:1":1:1 %)O!:1:1x##%,,##r*rr-r8c f[[X4U(d [UO[U[SS9S.UD6$NT)flatten_sequential) model_cfg feature_cfgrr model_cfgsr&variant cfg_variant pretrainedrs r(_create_mobilevitr,?<  -8*W%j>UD1    r*c f[[X4U(d [UO[U[SS9S.UD6$r"r&r(s r(_create_mobilevit2r/Gr-r*c $USSSSSSSSS S S S . UE$) Ni)rAr?r?)rprpg?bicubic)rqrqrq)r;r;r;z stem.convzhead.fcFzcvnets-license) url num_classes input_size pool_sizecrop_pct interpolationmeanstd first_conv classifierfixed_input_sizelicenser)r2rs r(_cfgr>Os54}SY)\!!#    r*zmobilevit_xxs.cvnets_in1kztimm/) hf_hub_idzmobilevit_xs.cvnets_in1kzmobilevit_s.cvnets_in1kzmobilevitv2_050.cvnets_in1kg"~j?)r?r6zmobilevitv2_075.cvnets_in1kzmobilevitv2_100.cvnets_in1kzmobilevitv2_125.cvnets_in1kzmobilevitv2_150.cvnets_in1kzmobilevitv2_175.cvnets_in1kzmobilevitv2_200.cvnets_in1kz$mobilevitv2_150.cvnets_in22k_ft_in1kz$mobilevitv2_175.cvnets_in22k_ft_in1kz$mobilevitv2_200.cvnets_in22k_ft_in1kz(mobilevitv2_150.cvnets_in22k_ft_in1k_384)rAr@r@) r@)r?r4r5r6z(mobilevitv2_175.cvnets_in22k_ft_in1k_384z(mobilevitv2_200.cvnets_in22k_ft_in1k_384rc [SSU0UD6$)Nr+)rar,r+rs r(raras  N Nv NNr*c [SSU0UD6$)Nr+)rbrBrCs r(rbrbs  M Mf MMr*c [SSU0UD6$)Nr+)rcrBrCs r(rcrcs  Lz LV LLr*c [SSU0UD6$)Nr+)rerBrCs r(rere  P: P PPr*c [SSU0UD6$)Nr+)rfrBrCs r(rfrfrGr*c [SSU0UD6$)Nr+)rhrBrCs r(rhrhrGr*c [SSU0UD6$)Nr+)rgrBrCs r(rgrgrGr*c [SSU0UD6$)Nr+)rirBrCs r(ririrGr*c [SSU0UD6$)Nr+)rjrBrCs r(rjrjrGr*c [SSU0UD6$)Nr+)rkrBrCs r(rkrkrGr*)mobilevitv2_150_in22ft1kmobilevitv2_175_in22ft1kmobilevitv2_200_in22ft1kmobilevitv2_150_384_in22ft1kmobilevitv2_175_384_in22ft1kmobilevitv2_200_384_in22ft1k)@)r+rT)r4r5r6)r;)NF)rD)F)@rrtypingrrrrrtorch.nn.functionalr functionalr timm.layersrr r r r r _builderr _features_fxr _registryrrrbyobnetrrrrrrvision_transformerrr__all__r)r3r:rQr&r'rrmrrrr,r/r> default_cfgsrarbrcrerfrhrgrirjrkrrr*r(r`s  22 __*1YY[[9 <  * $qB! < $qB! < qB!RSTabgj k qB!RSTabgj k qB!RSTabgj k    $qB! 4 $qB! 4 qB!RSTab c qB!STUbc d qB!STUbc d    $qB! 4 $qB! 4 qB!STUbc d qC1cUVcd e qC1cUVcd e    $qB! 4 $qB! 4 qB!STUbc d qC1cUVcd e qC1cUVcd e  #&"%S)$S)$T*$S)$S)$T*$S)QI XqRYYqqhy>"))y>x;RYY;|dryyddN{N+|-. %.&!8.&w 7.&tg6.& "4$ .&"4$.&"4$.&"4$.&""4$#.&("4$).&."4$/.&6+D-7.&<+D-=.&B+D-C.&J/ Hs1DK.&P/ Hs1DQ.&V/ Hs1DW.&. bOOONNNMwMMQ7QQQ7QQQ7QQQ7QQQ7QQQ7QQQ7QQH F F F$N$N$N'r*