RЦi_SrSSKrSSKrSSKJrJrJrJrJrJ r SSK r SSK J r SSK J s J r SSKJrJr SSKJr SSKJr \R,"\5r"S S \ R25r"S S \5rS"S \\S\S\S\4Sjjr\ R@r!\!4S\\\4S \\\4S\S\S\ RDS\ RFS\ RH4Sjjr%\!4S\ RHS\ RHS\\\4S\ RFS\ RFS\ RH4 Sjjr&S"S\ RHS \\S\S\S\4 Sjjr'"SS\ R25r("S S!\ R25r)g)#a>Image to Patch Embedding using Conv2d A convolution based approach to patchifying a 2D image w/ embedding projection. Based on code in: * https://github.com/google-research/vision_transformer * https://github.com/google-research/big_vision/tree/main/big_vision Hacked together by / Copyright 2020 Ross Wightman N)CallableDictListOptionalTupleUnion)nn)Formatnchw_to) to_2tuple)_assertc^\rSrSr%Sr\\S'\RR\ \S'SS\ \ \ \\ \ 44S\ S\ S\ S \ \S \ S\ \S \ S \ S\ 4U4S jjjrS\ \ \\ \ 444SjrSS\ \ \ \\ \ 44S\ \ \ \\ \ 444SjjrSS\ \\ \ 4\ 44SjjrS\\ \ 4S\\ \ 44SjrSrSrU=r$) PatchEmbed 2D Image to Patch Embedding output_fmtdynamic_img_padimg_size patch_sizein_chans embed_dim norm_layerflattenbiasstrict_img_sizec >XS.n [TU]5 [U5UlUR U5uUlUlUlUbSUl[U5Ul OX`l[RUl Xl Xl [R"X44X"US.U D6UlU(aU"U40U D6Ulg[R""5Ulg)NdevicedtypeF) kernel_sizestrider)super__init__r r_init_img_sizer grid_size num_patchesrr rNCHWrrr Conv2dprojIdentitynorm)selfrrrrrrrrrrrr dd __class__s V/var/www/html/ai-image-ml/venv/lib/python3.13/site-packages/timm/layers/patch_embed.pyr$PatchEmbed.__init__ s / #J/:>:M:Mh:W7 t~t'7  ! DL$Z0DO#L$kkDO..IIhnzcgnkmn 3=Jy/B/ 2;;= cUR(deUcg[U5n[[XR5VVs/sH up#X#-PM snn5nUSUS-nXU4$s snnf)N)NNNrr )rr tuplezip)r-rspr&r's r0r%PatchEmbed._init_img_sizeAsl  #X&c(OO.LM.Lda16.LMN lYq\1 K//NsA' c ~SnUb [U5nUGbLX0R:wGa<[R"5 [R "UR RUR RUUUR RSLUR RRUR RRS9nURR[UR RUSS95 UR Rb/URRUR R5 X@lSSS5 X0lU=(d URnXR:wdUb%UR!U5uUlUlUlgg!,(df  Na=f)N)r!r"rrr T)verbose)r rtorchno_gradr r)r* in_channels out_channelsrweightrr copy_resample_patch_embedrr%r&r')r-rrnew_patch_sizenew_projs r0set_input_sizePatchEmbed.set_input_sizeJsA   !&z2N  %.OO*K99II))II** .)t399++22))**00%%&:499;K;K^ei&jk99>>-MM'' 7$ !-O,t}} }} $(B>B>Q>QRZ>[ ;DM4>4+;)C!!s DF.. F<returncRU(a[UR5$UR$N)maxr)r- as_scalars r0 feat_ratioPatchEmbed.feat_ratiofs t' '?? "r2cUR(aR[R"USURS- 5[R"USURS- 54$USURS-USURS-4$)zGet grid (feature) size for given image size taking account of dynamic padding. NOTE: must be torchscript compatible so using fixed tuple indexing rr )rmathceilr)r-rs r0dynamic_feat_sizePatchEmbed.dynamic_feat_sizels   99Xa[4??1+==> (ST+X\XgXghiXjJj@kk kA;$//!"44hqkT__UVEW6WW Wr2c4URup#pEURbUR(a_[X@RS:HSUSURSS35 [XPRS:HSUSURSS35 OuUR(dd[X@R S-S:HSUSUR SS35 [XPR S-S:HSUSUR SS35 UR(a}UR SX@R S-- UR S-nUR SXPR S-- UR S-n[ R"USUSU45nURU5nUR(a"URS5RSS5nO3UR[R:wa[XR5nURU5nU$) NrzInput height (z) doesn't match model ().r z Input width (z%) should be divisible by patch size ()shaperrrrrFpadr*r transposerr r(r r,)r-xBCHWpad_hpad_ws r0forwardPatchEmbed.forwardusWW a == $##]]1--sBYZ^ZgZghiZjYkkm/no]]1--qcAXY]YfYfghYiXjjl/mn))**a/$QC'LT__]^M_L``bc**a/#A3&KDOO\]L^K__ab   __Q'!ooa.@*@@DOOTUDVVE__Q'!ooa.@*@@DOOTUDVVEa!UAu-.A IIaL << ! &&q!,A __ +??+A IIaLr2) rrr&rr,r'rrr*r) NTNTTFNN)NN)T)__name__ __module__ __qualname____firstlineno____doc__r __annotations__r;jitFinalboolrrintrrstrr$r%rDrKrPr`__static_attributes__ __classcell__r/s@r0rrsYY__T**?B  -1 (,$($)QuS%S/%9:;QQ Q  Q !* QQ! QQ"Q"QQB0uS%S/-A'B0?C@D\uS%S/%9:;\!sE#s(O';!<=\8#E%S/32F,G# X%S/XeCHoXr2rc^\rSrSr%Sr\\S'SS\\\ \ \ \ 44S\ S\ S\ S\\ S \ S\\ S \ 4U4S jjjrS \ \R \\ 44S jrSrU=r$)PatchEmbedWithSizerrrrrrrrrc 2>[T U]UUUUUUUUU U S9 g)N) rrrrrrrrrr )r#r$) r-rrrrrrrrrr r/s r0r$PatchEmbedWithSize.__init__s5 !!!  r2rFc6URup#pEURbd[X@RS-S:HSUSURSS35 [XPRS-S:HSUSURSS35 UR U5nURSSnUR (a"UR S5R SS5nO3UR[R:wa[XR5nURU5nX4$) NrzInput image height (z#) must be divisible by patch size (rSr zInput image width (rT) rUrrrr*rrXrr r(r r,)r-rYrZr[r\r] feat_sizes r0r`PatchEmbedWithSize.forwardsWW a == $ A**a/3GsJmnrn}n}~oAnBBD2E F A**a/3FqcIlmqm|m|}~mmAAC2D E IIaLGGBCL << ! &&q!,A __ +??+A IIaL|r2) rbrcrdreNTNTNN)rfrgrhrirjr rkrrrorrrnrpr$r;Tensorrr`rqrrrss@r0rurus?B  -1 (, uS%S/%9:;      !*   !    4 E%,,S "9:  r2runew_size interpolation antialiasr:c ^^^^ ^ ^SSKm SSKJn [ UR 5S:XdS5e[ T5S:XdS5eUR SSn[U5[T5:XaU$U(a)[RS UR S TS TS 35 UU4S jm U U 4SjnU"UT5n[R"T RRUR5URS9mUU4Sjn U"U"U SS5SS5n URn UR!5nU "U5nUR#U 5nU$![a SSKJn GNKf=f)aResample the weights of the patch embedding kernel to target resolution. We resample the patch embedding kernel by approximately inverting the effect of patch resizing. Code based on: https://github.com/google-research/big_vision/blob/b00544b81f8694488d5f36295aeb7972f3755ffe/big_vision/models/proj/flexi/vit.py With this resizing, we can for example load a B/8 filter into a B/16 model and, on 2x larger input image, the result will match. Args: patch_embed: original parameter to be resized. new_size (tuple(int, int): target shape (height, width)-only. interpolation (str): interpolation for resize antialias (bool): use anti-aliasing filter in resize verbose (bool): log operation Returns: Resized patch embedding kernel. rN)vmapzFour dimensions expectedrTzNew shape should only be hwrzzResize patch embedding z to z, w/ z interpolation.c>[R"U5Sn[R"X!TTS9SR 5nU$)N)NN.)sizemoder)rr.)r;r~rV interpolatenumpy)x_np _new_sizex_tf x_upsampledrrs r0resize(resample_patch_embed_old..resizesG||D!/2mm } KKTVV[V[V] r2c>/n[TRU55HNnTRU5nSUTRX05'UR T"XA5R S55 MP TR U5R$)Ng?)rangeprodzeros unravel_indexappendreshapestackT) _old_sizermati basis_vecnprs r0get_resize_mat0resample_patch_embed_old..get_resize_matssrwwy)*A+I8:Ib&&q4 5 JJvi3;;B? @+xx}r2)rcN>TURS5-nURT5$)Nr)r)kernelresampled_kernelrresize_mat_pinvs r0resample_kernel1resample_patch_embed_old..resample_kernels(*V^^B-??''11r2r )rr;r ImportError functorchlenrUr4_loggerinfotensorlinalgpinvrrr floatto) patch_embedrrrr:rold_sizer resize_matrv_resample_kernel orig_dtyperrrs ``` @@@r0resample_patch_embed_oldrsM4# {  !Q &B(BB & x=A <<<   %H X%/) .{/@/@.AhZuUbTccrst  (3Jll299>>*,,#? HZHZ[O2T/1a8!Q?""J##%K#K0K..,K M #""#sE EErrr rFcUupgUupXg-n X-n [R"XUS9n U RU SXg5n [R"U UUUSS9nUR S5R SSS5RX5nU$)zKComputes the resize matrix basis vectors and interpolates them to new_size.rr F)rrr align_cornersrTr)r;eyerrVrsqueezepermute)rrrrrr old_hold_wnew_hnew_w old_total new_total eye_matrixbasis_vectors_batchresized_basis_vectors_batch resize_matrixs r0_compute_resize_matrixr sLELE I I95AJ$,,Y5H"#--   #077:BB1aKSST]iM r2r pinv_matrixnew_size_tuplerintermediate_dtypecURtpVnURXVS5RUS9nURUS9nX-nUR"XV/UQ76RUS9nU$)zNSimplified resampling w/o vmap use. As proposed by https://github.com/stas-sl r)r )rUrr) rrrrrc_outc_in_resampled_patch_embeds r0_apply_resamplingr%sy"''OE!%%e2699@R9SK..'9.:K'5199%WWZZakZl  r2cz[UR5S:XdS5e[U5S:XdS5e[URSS5n[U5nXV:XaU$URnURn[ XVX#U[ 5n [RRU 5n [X Xh[ 5n U $)z4Standalone function (computes matrix on each call). rz/Input tensor should be 4D (out_ch, in_ch, h, w)rTz+New shape should only be hw (height, width)rzN) rrUr4rr rDTYPE_INTERMEDIATEr;rrr) rrrrr:old_size_tuplerrrrrrs r0rArA7s {  !Q &Y(YY & x=A LLL &+K,=,=bc,B&CN&+HoN'   F""J' &J\J,,##J/K-.>P ! r2c ^\rSrSrSrSS\\\4S\S\4U4Sjjjr \ 4S\\\4S\ RS \ RS \ R4S jjrS \ RS\\S \ R4S jrSrU=r$) PatchEmbedResamplerFixedOrigSizeiUzy Resample patch embedding weights from a fixed original size, caching the pseudoinverse matrix based on the target size. orig_sizerrc>[TU]5 [U[5(a[ U5S:XdS5eXlX lX0l0Ulg)z Args: orig_size (Tuple[int, int]): The expected original (height, width) of input patch_embed tensors. interpolation (str): Interpolation mode. antialias (bool): Use anti-aliasing filter in resize. rTz.`orig_size` must be a tuple of (height, width)N) r#r$ isinstancer4rrrr_pinv_cache_map)r-rrrr/s r0r$)PatchEmbedResamplerFixedOrigSize.__init__ZsP )U++I!0C = < =C"*";=r2rrr rFcUnURRU5nU(a=[X5(a-[X5nURU:XaUR U:XaU$[ URXRURX#5n[RRU5nSUSSUS3n[X5(a [X5 URXV5 XPRU'U$)zRRetrieves the cached pinv matrix or computes and caches it for the given new_size.pinv_rrYr )rgethasattrgetattrrr rrrrr;rrdelattrregister_buffer)r-rrr cache_key buffer_namerrs r0_get_or_create_pinv_matrix;PatchEmbedResamplerFixedOrigSize._get_or_create_pinv_matrixos **..y9 7455!$4K!!V+ 0A0AU0J##, NNH&8&8$..& ll'' 3 hqk]!HQK=9 4 % % T ' [6*5Y'r2rc~[UR5S:Xde[U5S:Xde[URSS5nX0R:XdSUSUR35e[U5nURU:XaU$URnUR nUR XE5n[XXF5nU$)zResamples the patch embedding weights to new_size. Args: patch_embed (torch.Tensor): Original weights (out_ch, in_ch, H_orig, W_orig). new_size (List[int]): Target [height, width]. Returns: torch.Tensor: Resampled weights. rrTrzNzInput patch_embed spatial size z0 does not match module's expected original size )rrUr4rrr rr) r-rr input_sizerrrrrs r0r`(PatchEmbedResamplerFixedOrigSize.forwards;$$%***8}!!!;,,RS12 ^^+ @-j\://3~~.> @ @++0/ >>^ + ## && 55nM !2+N g$$r2)rrrr)bicubicT)rfrgrhrirjrrorprnr$rr;rr r~rrr`rqrrrss@r0rrUs"+" >S#X>> >>2"4 CHoLL;;   <"%5<<"%49"%"%"%r2rc |^\rSrSrSrSS\\\4S\S\S\S\4 U4Sjjjr S \ RS \\\4S \ R4S jr S \ RS \\\4S \ R4S jr SS\ RS\ RS\\ RS\\\\4S\S \ R4 SjjrSrU=r$)PatchEmbedInterpolatoriaDynamically interpolates patch embedding weights for variable patch sizes. This module wraps patch embedding weight resampling functionality to support on-the-fly patch size variation during training. It handles both Conv2d and Linear patch embeddings. Args: base_patch_size: The original patch size the model was initialized with in_chans: Number of input channels embed_dim: Embedding dimension interpolation: Interpolation mode for resampling antialias: Whether to use antialiasing during interpolation base_patch_sizerrrrc^>[TU]5 XlX lX0lX@lXPlgrH)r#r$rrrrr)r-rrrrrr/s r0r$PatchEmbedInterpolator.__init__s+ . "*"r2r?target_patch_sizerFcZX R:XaU$URSnURupEUupgURX4XPR5nUR SSSS5n[ UXg/UR URSS9n U R SSSS5n U RUS5n U $)zResample linear patch embedding weights for a new patch size. Args: weight: Linear weight tensor of shape [embed_dim, patch_h * patch_w * in_chans] target_patch_size: Target (patch_h, patch_w) to resample to Returns: Resampled weight tensor rrdr rTFrrrr:r)rrUrrrrArr) r-r?rrbase_phbase_pw target_ph target_pw weight_convweight_conv_resampledweight_resampleds r0resample_linear_weight-PatchEmbedInterpolator.resample_linear_weights  4 4 4MLLO //0 nnY--P !))!Q15 !5 +,,nn ! 188Aq!D+33IrBr2c|X R:XaU$[U[U5URURSS9nU$)zResample conv2d patch embedding weights for a new patch size. Args: weight: Conv2d weight tensor of shape [embed_dim, in_chans, patch_h, patch_w] target_patch_size: Target (patch_h, patch_w) to resample to Returns: Resampled weight tensor Fr)rrAlistrr)r-r?rrs r0resample_conv_weight+PatchEmbedInterpolator.resample_conv_weightsI  4 4 4M0 +,,,nn   r2patches proj_weight proj_biasr is_linearcUc URnU(aX@R:wavURS:XdS5eURupgpn URX$5n UR XgS5n [ R RRXU5n U $URS:Xa"URupgpn UR XgS5n[ R RRXU5n U $X@R:wa=URX$5n [ R RRXUUSS9n U $[ R RRXUUSS9n U $)aApply patch embedding with dynamic weight resampling. Args: patches: Input patches - For linear mode with resampling: [B, N, Ph, Pw, C] - For linear mode without resampling: [B, N, Ph*Pw*C] - For conv mode: [B, C, H, W] proj_weight: Original projection weight proj_bias: Optional projection bias patch_size: Current patch size (if None, uses base_patch_size) is_linear: Whether using linear (True) or conv2d (False) projection Returns: Embedded patches z0Patches must be [B, N, Ph, Pw, C] for resamplingrr)r"padding) rndimrUrrr;r functionallinearrconv2d)r-rrrrrrZNPhPwr[r patches_flatoutputs r0r`PatchEmbedInterpolator.forwardse.  --J 111||q(\*\\(")--ba$(#>#>{#W  'qR8 ,,33LT]^* %<<1$&-mmOA"!%ooaB7G,,33G)T 111#'#<#<[#U ,,33y%q4 ,,33)%q4  r2)rrrrr)rdrerT)NNT)rfrgrhrirjrrorprnr$r;r~rrrr`rqrrrss@r0rrs- " !*" #"38_ # # #  #  # #( LL(  %S#X(   ( T LL  %S#X    @1548" :\\::  - : !sCx1 :  : ::r2r)rTF)*rjloggingrNtypingrrrrrrr;r torch.nn.functionalrrVformatr r helpersr trace_utilsr getLoggerrfrModulerrurorprnrfloat32rrr r~rrrArrr}r2r0rs  ?? #   H %ttn,,f' Cs)CC C  CL]]0 S/S/    {{  \\@+= !\\!\\!c3h!KK ! "KK !  \\ !*' !\\!s)!! !  !