RЦiSrSSKrSSKJrJrJrJr SSKrSSKJr SSK J r SSK J r SSK Jr SVS \S \S \S \\R$4SjjrSWS \S\S\S \\R$S\R(4 SjjrSSSSS\R,4S\\S\S\S\S\S \\R$S\R.S\R(4SjjrS\\S\\4SjrSSSSSSS SSS S\R,4 S\\S!\\R(S \S"\S\S \S#\S$\S%\\\S&\S'\S \\R$S\R.S\\R(4S(jjr"S)S*\R85rS+rS,\R(S\R(4S-jrSXS,\R(S.\R(S/\R(S0\S\R(4 S1jjr SXS,\\R(S.\R(S/\R(S0\S\\R(4 S2jjr!SXS,\R(S3\R(S0\S\R(4S4jjr"SXS,\R(S5\R(S6\R(S7\S\R(4 S8jjr#SSSSSS SSS S\R,4 S\\S!\\R(S\S"\S\S \S$\S%\\\S&\S'\S \\R$S\R.4S9jjr$"S:S;\R85r%"S<S=\R85r&SYS\R,S>.S?\S@\SA\S\SB\S\R(4 SCjjjr'\RPRR\ S S\R,4SD\\S'\S \\R$S\R.S\\R(\R(44 SEjj55r*SF\R(SG\R(SH\R(S\R(4SIjr+"SJSK\R85r,\RPRR\ SLS SSM\R,4SN\SO\SP\S'\S&\S \R$S\R.S\R(4SQjj55r-"SRSS\R85r.SZST\S\SA\S\R84SUjjr/g)[zuSin-cos, fourier, rotary position embedding modules and functions Hacked together by / Copyright 2022 Ross Wightman N)ListTupleOptionalUnion)nn)register_notrace_function)ndgrid)_assertT num_bandsmax_freq linear_bandsdevicec  U(a*[R"SUS- U[RUS9nOAS[R"S[R"US5S- U[RUS9-nU[R -$)N?dtyperrr)torchlinspacefloat32mathlogpi)r r rrbandss [/var/www/html/ai-image-ml/venv/lib/python3.13/site-packages/timm/layers/pos_embed_sincos.pypixel_freq_bandsrsh sHqL)5==Y_`U^^Atxx!'>"  2 2D(QIkk599T?EIIdO<)LTTUVWG ::E: ""s A E seqcV[U5S:aU$USUS/[USS5-$)Nrrr)lenlist)rBs r swap_shape_xyrFPs4 3x!| FCF d3qr7m ++rijrmax_res include_grid in_pixelsref_feat_shape grid_offset grid_indexingc Uc*U(a[U[U5UU S9nO+[UUSU S9nOU c URn U c URn U S:Xa[ U5nUb [ U5nU(a6UV s/sH(n [ R"SSX[ RS9PM* nn OSUV s/sHFn [ R"X[ RS9R[ R5U -PMH nn Ub*[XU5VVVs/sHunnnUU- U-PM nnnn[ R"[ R"XS 9S S 9nURS 5nUU-nUR!5RU S 9UR#5RU S 9nnU(aUUU/nU$UU/nU$s sn fs sn fs snnnf) a Args: feat_shape: Feature shape for embedding. bands: Pre-calculated frequency bands. num_bands: Number of frequency bands (determines output dim). max_res: Maximum resolution for pixel based freq. temperature: Temperature for non-pixel freq. linear_bands: Linear band spacing for pixel based freq. include_grid: Include the spatial grid in output. in_pixels: Output in pixel freq. ref_feat_shape: Reference feature shape for resize / fine-tune. grid_offset: Constant offset to add to grid for non-pixel freq. grid_indexing: Indexing mode for meshgrid ('ij' or 'xy') dtype: Output dtype. device: Output device. Returns: )rrrr0xygr)stepsrrr2indexingr1r3r4)rfloatr(rrrFrrrr$r%r&zipr5meshgridr8r9r:)r*rr rJr rrKrLrMrNrOrrr<txfrr=pospos_sinpos_cosouts rbuild_fourier_pos_embedr`VsF } $g) E' E >\\F =KKE":.  %*>:N   NN3!%-- P     LL = @ @ OR] ]  !&)!&H I&H71aQUQY&H I ;;u~~a@b ID >>" D ,Cwwy||%|0#''),,U,2KWG&24' "C J:A'8JC J)   Js6/F3,A F8F=ch^\rSrSrS S\S\4U4SjjjrS SjrS SjrS SjrSr S r U=r $) FourierEmbedrJr c >[TU]5 XlX lX0lX@lUR S[R"X%US9SS9 UR5 g)Nrr2F persistent) super__init__rJr concat_grid keep_spatialregister_bufferremptyreset_parameters)selfrJr rirjrr __class__s rrhFourierEmbed.__init__sV  "&( Wekk)RW&Xejk rc$UR5 gz"Initialize parameters and buffers.N _init_buffersrns rrmFourierEmbed.reset_parameters rcvURR[URUR55 g.Compute and fill non-persistent buffer values.N)rcopy_rr rJrus rrtFourierEmbed._init_bufferss$ )$..$,,GHrc$UR5 gz"Initialize non-persistent buffers.Nrsrus rinit_non_persistent_buffers(FourierEmbed.init_non_persistent_buffersrwrc URSSup#URSSn[UURURURUR S9n[ R"USS9nURSS5R[U55nU4SURS- --nUR(aH[ R"XRS5RU5RSS SS5/SS9nU$[ R"URSSS S5URS5RU5/SS9nUR!X$R#5S5nU$) Nr)rKrrr1r3r1rr)shaper`rrirrrcatr7r6rDndimrjr8expandpermutereshapenumel)rnrYBCr*emb batch_expands rforwardFourierEmbed.forwards=wwr{WWQR[ %  JJ))''88  ii$mmB#++C O<teqvvz22     1mmA.55lCKKAqRSUVWX^_`A  199Q1a0#--2B2I2I,2WX^`aA !--/4Ar)rirjrJr )rGr)TFNNr"N) __name__ __module__ __qualname____firstlineno__intrhrmrtrr__static_attributes__ __classcell__ros@rrbrbsM    &Irrbc[R"USSSS24*USSSS24/S5RUR5$)N.rrr1)rr5rr)rYs rrotrsE ;;319 qcc{3R 8 @ @ IIrrYcVURSSS9up[R"U*U/SS9$)Nrr1r3)chunkrr)rYx1x2s rrope_rotate_halfrs/WWQBW FB 99rc2YB ''rsin_embcos_embhalfc\U(aX-[U5U--$X-[U5U--$N)rr)rYrrrs rapply_rot_embedrs9 {-a07::: {SVg---rc[U[R5(aU/nU(a$UVs/sHoDU-[U5U--PM sn$UVs/sHoDU-[ U5U--PM sn$s snfs snfr) isinstancerTensorrr)rYrrrrXs rapply_rot_embed_listrs| !U\\"" C FGGQG .q1G;;QGG 9::1G c!fw..:: H ;s A2A7rcURSS5up4U(aX-[U5U--$X-[U5U--$)Nrr1)rrr)rYrrrrs rapply_rot_embed_catrsL yyB'G {-a07::: {SVg---r pos_embed keep_indicespos_embed_has_batchcU(a[URS:S5 OY[URS:S5 URS4SUR--nURS5R U5nURS4SURS- --URSS4-nUR U5n[ UR5nSUS 'UR U5nURS U5$) aApply keep indices to different ROPE shapes Expected pos_embed shapes: * [seq_len, pos_embed_dim] --> output [batch_size, seq_len, pos_embed_dim] * [num_heads, seq_len, pos_embed_dim] --> output [batch_size, num_heads, seq_len, pos_embed_dim] * [depth, num_heads, seq_len, pos_embed_dim] --> output [batch_size, depth, num_heads, seq_len, pos_embed_dim] And all of the above with leading batch dimension already present if `pos_embed_has_batch == True` rzIncorrect number of dimensionsrrr)rrr1r)r rrr8rviewrEgather)rYrrr expand_shape keep_shape keep_expands rapply_keep_indices_nlcr,s  !#%EF  !#%EF }uy~~'== ''*11,? $$Q')DINNQ4F,GG[TU]5 XlX lX0lXPlX@lX`lXplXl Xl USLUl Uc9US-4n URS[R"XU S9SS9 SUlSUlOaSUlSn UHnX-n M X4nURS[R"XU S9SS9 URS[R"XU S9SS9 UR%5 g) Nr/rr2Frer pos_embed_sin pos_embed_cos)rgrhr+rJr rrLr*rMrNrO_use_cached_embedrkrrlrrrrmrnr+rJr rLrr*rMrNrOrr bands_shapenum_posr< emb_shaperos rrhRotaryEmbedding.__init__s   &("$,&*",4!7  !8+K  %++kX]*^kp q!%D !%D DJG   I  %++i^c2dqv w  %++i^c2dqv w rc$UR5 grrrsrus rrm RotaryEmbedding.reset_parametersrwrc UR(d*URRUR55 gUR UR 5upUR RU5 URRU5 gry)rrr{_compute_bands_get_pos_embed_valuesr*rr)rnemb_sinemb_coss rrtRotaryEmbedding._init_buffersse%% JJ  T002 3#99$//J G    $ $W -    $ $W -rcUR(a6[URS-[UR5UR S9nO"[ URS-URSS9nURXS9$zCompute frequency bands.r/)rr)r r!r2 rLrr+rUrJrr(r r&rnrrrs rrRotaryEmbedding._compute_bandsi >>$A dll#!..E A  ,,E xxvx33rc[UURURURURUR UR URURUUS9 upEXE4$)N r*r+rJr rrLrMrNrOrr) rr+rJr rrLrMrNrO)rnr*rrrrs rr%RotaryEmbedding._get_pos_embed_valuesse1!LL((**nn..((,,  rc$UR5 gr~rsrus rr+RotaryEmbedding.init_non_persistent_buffersrwrcURb{XR:wakURceURceURUURRURR S9uUlUlXlgggNr2)r*rrrrrrnr*s rupdate_feat_shape!RotaryEmbedding.update_feat_shapes ?? &:+H%%1 11%%1 11595O5O))00((..6P6 2D  2 )O,I &rrc UbMURb@[UURURURURUR S9$UR b%URbUR UR4$S5e)NrLrMrNrOzQget_embed() requires pre-computed pos embeds or valid shape w/ pre-computed bands)rrrLrMrNrOrr)rnrs r get_embedRotaryEmbedding.get_embeds  !7) ..#22 ,,"00     +0B0B0N%%t'9'99 9 mm m5rcZURURSS5up#[XU5$Nr)rrr)rnrYrrs rrRotaryEmbedding.forwards*>>!''!"+6q733r) rrr+r*rOrNrLrrJrrrMr rGi'TFNNrHrINNrNNr)rrrr__doc__boolrrrrUstrrhrmrtrrrrrrrrrrrs@rrrs!&.226!#!%-  - !c+- %T#Y/- - - - ^.4 CGemm S   )DI )nxS 2n 44rrc~^\rSrSrSrSS\S\S\S\S\S \\ \S \\ \S \S \ 4U4S jjjr SSjr SSjr SSjrS\R 4S \ \4SjjrSSjrS \ \4SjrSS\\ \4SjjrSS\ \\\4S\\S\\R.\ \R.44SjjrSrSrU=r$) RotaryEmbeddingCatia Rotary position embedding w/ concatenatd sin & cos The following impl/resources were referenced for this impl: * https://github.com/lucidrains/vit-pytorch/blob/6f3a5fcf0bca1c5ec33a35ef48d97213709df4ba/vit_pytorch/rvt.py * https://blog.eleuther.ai/rotary-embeddings/ Nr+rJr rLrr*rMrNrOc >[TU]5 XlX lX0lX@lXPlX`lXplXl Xl USLUl Uc2US-4n URS[R"XU S9SS9 SUlO@SUlSn UHnX-n M XS-4nURS[R"XU S9SS9 UR#5 g) Nr/rr2Frerrr)rgrhr+rJr rLrr*rMrNrOrrkrrlrrrmrs rrhRotaryEmbeddingCat.__init__s  &"($,&*",4!7  !8+K  %++kX]*^kp q!DNDJG   '*I  ekk)Z_.`mr s rr"c$UR5 grrrsrus rrm#RotaryEmbeddingCat.reset_parametersKrwrcUR(d*URRUR55 gURRUR UR 55 gry)rrr{rrrr*rus rrt RotaryEmbeddingCat._init_buffersOsG%% JJ  T002 3 NN !;!;DOO!L MrcUR(a6[URS-[UR5UR S9nO"[ URS-URSS9nURXS9$rrrs rr!RotaryEmbeddingCat._compute_bandsVrrc[UURURURURUR UR URURUUS9 n[R"US5$)Nrr1) rr+rJr rrLrMrNrOrr)rnr*rrembedss rr(RotaryEmbeddingCat._get_pos_embed_valuesfsj'!LL((**nn..((,,  yy$$rc$UR5 gr~rsrus rr.RotaryEmbeddingCat.init_non_persistent_buffersvrwrcURbdXR:waTURceURUURRURRS9UlXlgggr)r*rrrrrs rr$RotaryEmbeddingCat.update_feat_shapezsi ?? &:+H>>- --!77~~,,nn**8DN )O,I &rrc UbdURbW[UURURURURUR S9n[ R"US5$URb UR$S5e)Nrr1zPget_embed() requires pre-computed pos embed or valid shape w/ pre-computed bands) rrrLrMrNrOrrr)rnrrs rrRotaryEmbeddingCat.get_embedsy  !7+ ..#22 ,,"00 F99VR( ( ^^ '>> ! ll l5rshapesseq_lenc U(d/$URc [S5e[SU55n[SU55n[X44URURUR UR URS9upV[R"XV/SS9RX4S5nUbz[R"[U5X'RS5RU5n[U5H-un upX-n USU 2SU 24R!U S5XSU 24'M/ U$UV V s/sH"upUSU 2SU 24R!X-S5PM$ n n n U $s sn n f)a{Generate ROPE embeddings for multiple grid shapes efficiently. Computes embeddings for the maximum grid size once, then extracts and flattens the relevant portions for each requested shape. Args: shapes: List of (H, W) tuples representing different grid sizes Returns: List of concatenated sin/cos embeddings for each shape, where each tensor has shape (H*W, dim) NzMBatch embedding generation requires cached bands, not pre-computed embeddingsc3*# UH upUv M g7fr.0hws r 6RotaryEmbeddingCat.get_batch_embeds..)&$!A&c3*# UH upUv M g7frrrs rrrr r )r*rrLrMrNrOr1r3)r RuntimeErrormaxrrLrMrNrOrrrzerosrDrtype_as enumerater)rnrrmax_hmax_wrr rope_embed_2d flat_embedsirrsrc_lenflat_embeds_lists rget_batch_embeds#RotaryEmbeddingCat.get_batch_embedssp"I :: no o)&)))&))2~**nn..((,,   7"4"=BB5QST  ++c&k7> U1XfEMMB U1XfEMMBLE ((5/ ! ! #C ((5/ ! ! #C 8Orfreqsr-r.cURnUR5nURS5USRS5-nURS5USRS5-nXE-n[R"U5R SS5n[R "U5R SS5n[R"Xx/SS9n U RU5$)z&Compute mixed (learnable) frequencies.r1rrrrr3) rrUr8rr9rr:rr&) r0r-r.rfreqs_xfreqs_ycombinedrr rope_embedss rget_mixed_freqsr6s KKE KKME}}R 58#5#5b#99G}}R 58#5#5b#99G Hii!33Ar:Gii!33Ar:G))W.B7K >>%  rc `^\rSrSrSrSS\S\S\S\S\\\S\ 4 U4S jjjr SS jr SS jr S\\\4S jr S\\\4SjrSSjrSS\\\S \R"4SjjrSS\\\\4S\\S \\R"\\R"44SjjrSrSrSrU=r$)RotaryEmbeddingMixediajRotary position embedding with depth-dependent learnable frequencies. This implementation supports mixed (learnable) ROPE. In mixed mode, each transformer block has its own set of learnable frequency parameters. Based on 'Rotary Position Embedding for Vision: https://arxiv.org/abs/2403.13298)' Compatible with original at https://github.com/naver-ai/rope-vit r+r r!r r*rOc >[T U]5 XlX lX0lX@lXPlX`lX-n U S-S:Xd SU 35e[U UUUSUUS9n [R"U 5Ul UbhSn UHn X-n M URS[R"XUS 9S S 9 URS [R"XUS 9S S 9 UR5 gS=UlUlg) aZInitialize rotary embeddings. Args: dim: Embedding dimension (should be divisible by 4) depth: Number of transformer blocks num_heads: Number of attention heads temperature: Base for frequency computation feat_shape: Spatial dimensions [H, W] if known in advance grid_indexing: How to index grid positions ('xy' or 'ij') r/rz%head_dim must be divisible by 4, got T)r r"rrNrr-r2Frer.)rgrhr+r r!r r*rOr)r Parameterr0rkrrlrtr-r.)rnr+r r!r r*rOrrrr0rr<ros rrhRotaryEmbeddingMixed.__init__"s*  "&$*#!|q T$I("TT $   # \\%(  !G    GRW(Xej k   GRW(Xej k    "& &DHtxrr"cURbTURUR5upURRU5 URRU5 ggry)r*_get_grid_valuesr-r{r.)rnr-r.s rrt"RotaryEmbeddingMixed._init_buffersYsG ?? &,,T__=HC HHNN3  HHNN3  'rc$UR5 grrrsrus rrm%RotaryEmbeddingMixed.reset_parameters`rwrc^[UURURRS9up#X#4$)NrOr)r/rOr0rrnr*r-r.s rr=%RotaryEmbeddingMixed._get_grid_valuesds0! ,,::$$  xrcURbXR:waURceURceURU5up#UR URR URR 5UlUR URR URR 5UlXlgggr)r*r-r.r=r&rrrCs rr&RotaryEmbeddingMixed.update_feat_shapels ?? &:+H88' ''88' '',,Z8HCvvdhhootxx~~>DHvvdhhootxx~~>DH(O ,I &rc$UR5 gr~rsrus rr0RotaryEmbeddingMixed.init_non_persistent_buffersurwrrcUb,[UURURRS9up#O9URb%UR bURUR p2OS5e[ URX#5$)zGenerate rotary embeddings for the given spatial shape. Args: shape: Spatial dimensions [H, W] Returns: Tensor of shape (depth, H*W, dim) containing concatenated sin/cos embeddings rBz8get_embed() requires pre-computed t_x/t_y or valid shape)r/rOr0rr-r.r6)rnrr-r.s rrRotaryEmbeddingMixed.get_embedysq  %"00zz((HC XX !dhh&:xx TT T5tzz344rrrc U(d/$[SU55n[SU55n[X4/URURRS9upV[ URXV5nUR uppURXX4U 5n Ub[U5n [R"XXXRRURRS9n[U5HAununnU SS2SS2SU2SU24RXUU-U 5nUU-nUXSS2SS2SU24'MC U$/nUH<unnU SS2SS2SU2SU24RXUU-U 5nURU5 M> U$)a!Generate ROPE embeddings for multiple grid shapes efficiently. Computes embeddings for the maximum grid size once, then extracts and flattens the relevant portions for each requested shape. Args: shapes: List of (H, W) tuples representing different grid sizes seq_len: If provided, return padded tensor of this length. Otherwise return list. Returns: If seq_len is provided: Padded tensor of shape (len(shapes), depth, num_heads, seq_len, dim) Otherwise: List of tensors with shape (depth, num_heads, H*W, dim) for each shape c3*# UH upUv M g7frrrs rr8RotaryEmbeddingMixed.get_batch_embeds..r r c3*# UH upUv M g7frrrs rrrMr r rBNr2)r r/rOr0rr6rrrDrrrrrappend)rnrrrrr-r. max_embedr r!_r+ max_embed_2drpaddedrrr embed_slice actual_lenresultss rr%RotaryEmbeddingMixed.get_batch_embedss$I)&)))&))" N,,::$$  $DJJ9 $-?? ! ~~ecJ  F A[[9s::K\K\dhdndndtdtuF&v. 6Aq*1a!RaR<8@@STWXSXZ]^ U /:!Q  +, / MG1*1a!RaR<8@@STWXSXZ]^ {+NrcTURURSS5n[X5$rrrs rrRotaryEmbeddingMixed.forwardrrcS1$)z/Exclude frequency parameters from weight decay.r0rrus rno_weight_decay$RotaryEmbeddingMixed.no_weight_decays yr) r r+r*r0rOr!r-r.r )$@NrQNNrr)rrrrrrrUrrrrhrtrmr=rrrrrrrrrr[rrrs@rr8r8s"&.2!%5'5'5' 5'  5' !c+ 5'5'5'n 8DI+>)HT#Y,?)5xS 25ell52&*6sCx)6c]6 u||T%,,// 0 6p1 rr8separatecpuheightwidthnormalize_coordsc[R"SX[RS9U-n[R"SX[RS9U-nUS:Xa[[ X55n U n U n OJUS:Xa[[ X55n U n U n O+US:Xa[U5n [U5n O[ SU35eXz- nX- nURU5nURU5nUS:Xa.[R"XSS9up[R"X/S S 9nO([R"[R"XxS S9S S 9nURS S 5nSU-S- nU$)zsMake coordinate grid matching offset and normalization of original. Returns: coords with shape (HW, 2) in [-1, 1]. g?r2r minr^zUnknown normalize_coords: rQrSr1r3rIrr@r) rr$rrUr rd ValueErrorr&rWr5r6)r`rarbrOrNrrcoords_hcoords_wdenomh_denomw_denomgrid_wgrid_hcoordss rmake_coords_dinov3rosI||CemmL{ZH||CU]]KkYH5 c&() U "c&() Z '-,56F5GHII!H!H{{5!H{{5!HTJf-26U^^HNTVW ^^Aq !F 6\C F Mrc^\rSrSrSrS$S\S\\S\\S\\S\\\S\ S \S \ S \ S \\S \\S\\4U4Sjjjr S%Sjr S%Sjr S\R4S\R S\R"S\R$4SjjrS\R$S\R$4SjrS\R$S\\R$\R$44SjrS&S\\S\ S\R$4SjjrS\\4SjrS\\4SjrS%SjrS'S\\\S\R$4S jjrS!\R$S\R$4S"jrS#rU=r$)(RotaryEmbeddingDinoV3iakRoPE for timm DinoV3 port, numerically matching original. Math is aligned to original DinoV3 RopePositionEmbedding at https://github.com/facebookresearch/dinov3: - 0.5-centered coords normalized by H/W (or min/max), mapped to [-1,1] - training-time augmentations (shift/jitter/rescale) - periods schedule equals Rope's temperature (base) or min/max period r+r min_period max_periodr*rbrNrO rotate_half shift_coords jitter_coordsrescale_coordsc L>[TU]5 XlXl[ U5UlX0lX@lX`lXl Xl Xl [URURUR4Vs/sHoSLPM sn5Ul XPlXplXlUS-4nUR#S[$R&"UXS9SS9 Ub7USUS-nUUS-4nUR#S [$R&"UXS9SS9 OSUlUR+5 gs snf) Nr/periodsr2Frerrrpos_embed_cached)rgrhr+rtrUr rrrsrbrurvrwany aug_activer*rNrOrkrrlrzrm)rnr+r rrrsr*rbrNrOrtrurvrwrra periods_shaperrros rrhRotaryEmbeddingDinoV3.__init__ s/" &!-$$!1(*,t7H7H$J\J\^b^q^q6rs6r}6rst%&*  Y M&(^kpq  ! mjm3G #'*I  !3U[[SY5gty z$(D ! )ts: D!r"c$UR5 grrrsrus rrm&RotaryEmbeddingDinoV3.reset_parametersCrwrcURRUR55 URbDURb6UR URSS9nURRU5 ggg)rzNTno_aug)ryr{_compute_periodsr*rz _create_embed)rn rope_embeds rrt#RotaryEmbeddingDinoV3._init_buffersGsh 40023 ?? &4+@+@+L++DOOD+IJ  ! ! ' ' 3,M &rr_rrcURS-nURb]URbP[R"SSUS[R S9nURURUR- U--nO^UR c [S5eS[R"US[R S9-URS-- nUR U-nURXS9$) z5Construct periods from either min/max or temperature.r/rrr_r2z0Provide either min/max periods or `temperature`.rer) r+rrrsrrrr rfr$r&)rnrrr+ exponentsrys rr&RotaryEmbeddingDinoV3._compute_periodsNshh!m ?? &4??+Fq!SemmTIoo$//DOO*KPY)YZG' !STTell3uEMMRRVZV^V^bcVcdI&&)3Gzzz55rrnc6UR(aUR(dU$URnURnURbF[ UR5n[ R"SX#S9RU*U5nXSSS24-nURb{[ UR5nUS::a [S5e[R"U5n[ R"SX#S9RU*U5R5nXSSS24-nURbt[ UR5n U S::a [S5e[R"U 5n [ R"SX#S9RU *U 5R5n X-nU$)z4Apply shift/jitter/rescale train time augmentations.Nrr2rzAjitter_coords must be > 0 (interpreted as multiplicative factor).zBrescale_coords must be > 0 (interpreted as multiplicative factor).r)trainingr|rrrurUrrluniform_rvrfrrr'rw) rnrnrrshiftshift_hw jitter_factor jitter_max jitter_hwrescale_factor rescale_maxrescales r_apply_coord_augs'RotaryEmbeddingDinoV3._apply_coord_augs_se}}DOOM     ($++,E{{1VAJJE6SXYHtQw//F    )!$"4"45M! !dee-0J AfBKKZKYcdhhjIa00F    *"4#6#67N" !eff((>2Kkk!F@II;,XcdhhjG%F rc&URS-nURRnURRnURR 5U:XdeUSS2SS2S4R X4S9nS[ R-U-URSSSS24- nURS5nUR(aURS5nOURSSS9n[R"U5n[R"U5nXg4$)zEReturn sin/cos embeddings with either 'half' or 'interleaved' layout.r/Nr2rrr1r3)r+ryrrrr&rrr6rttilerrr9r:)rnrnr+rrr&r9r:s r_get_pos_embed_from_coords0RotaryEmbeddingDinoV3._get_pos_embed_from_coordsshh!m$$ ""||!!#s***1d #&&f&BTWWv% T4](CC"   [[^F--aR-8FiiiixrrcUup4[X4URURURS9nU(dUR U5nUR U5upg[ R"Xg/SS9nU$)N)rbrOrNr1r3)rorbrOrNrrrr) rnr*rHWrnr9r:rs rr#RotaryEmbeddingDinoV3._create_embedsr # !22,,((   ++F3F226:YYzr2 rcRURUSS9nURSUSS9 Xlg)NTrrzFre)rrkr*)rnr*rs r _cache_embed"RotaryEmbeddingDinoV3._cache_embeds2'' 4'@  /N$rcbURb"XR:waURU5 gggr)r*rrs rr'RotaryEmbeddingDinoV3.update_feat_shapes, ?? &:+H   j ),I &rc$UR5 gr~rsrus rr1RotaryEmbeddingDinoV3.init_non_persistent_buffersrwrrc@UbURU5nU$URSL=(d UR=(a URnU(a1URcS5eURUR5nU$URceURnU$)zGenerate rope_embed matching DINOv3 RopePositionEmbedding numerics. Returns: (HW, num_heads, 2 * head_dim) with last dim = [sin, cos] cat. Nz&feature shape must be cached on create)rrzrr|r*)rnrr need_creates rrRotaryEmbeddingDinoV3.get_embeds  ++E2J//47^DMM<]dooK2\4\\2!//@ ,,888!22 rrYcfURURSS5n[XURS9$)z$Get and apply rotary embeddings to xrN)r)rrrrtrs rrRotaryEmbeddingDinoV3.forwards/NN17712;/ "1d6F6FGGr)r|r+r*rOrNrvrsrrrbrzrwrtrur ) gY@NNNr^rHrITNNNNNrFr)rrrrrrrrUrrrrhrmrtrrrrrrrrrrrrrrrrrrs@rrqrqs,1*.*..2$.!$!% $,0-1.25 5 "%5 ! 5 ! 5 !c+ 5 "5 5 5 5 #5/5 $E?5 %UO5 5 n473#3>v>> e  =$'!Ac&6A&AA g  ;% #T*#KKFKK h  ;% #T*$D)9DVDD.yk:;;r)gl@TN)rrNr)r]T)ri )0rrtypingrrrrrr_fxr r=r trace_utilsr rrUrrrrr(rrrArFrr`Modulerbrrrrrrrrrr)r'wrapr/r6r8rorqrrrrrs| // * !)-    & $)- &   \\ ###()-"]]&#I&# &#&# &# ! &# & &#{{&# \\&#R,tCy,T#Y,)-#"".2!)-"]]RIR %RR R  R  RRR!c+RRR&R{{R %,,Rj62996rJ ( (( . <<... .  \\ .. ;  ;;; ;  %,, ;0. <<. \\.. \\ ..%* ". <<".<<".ll"." ".  \\ ".N)-#".2!)-"]]5I5 %55 5  5  55!c+555&5{{5pJ4biiJ4Zx1x1~" (mm(((( (  ( \\(D")-"]] Cy&{{   5<< %& $!||! \\!\\! \\ !$u299up!+!$"]]---- -  -  -{{- \\--`MHBIIMHb%<%< %<%< YY %