ёiHZ~SSKJr SSKJr SSKJrJr SSKJr SSK r SSK J r SSK J r SSK JrJrJr \ "\S S S 9rSS KJr \(aSS K JrJr SSKJr 0qSrS+Sjr\"SS55r\"SS9S,Sj5r\"SS9S-S.Sjj5r\"SS9S/Sj5rS0Sjr S0Sjr!\"SS9S1Sj5r"S0Sjr#S0Sjr$S0Sjr%\"S S9S2S!j5r&\"SS9S3S"j5r'\"SS9S1S#j5r(S0S$jr)S4S%jr*S4S&jr+S5S4S'jjr,S5S4S(jjr-S6S)jr.S7S8S*jjr/g)9) annotations) dataclass)cached_property lru_cache) TYPE_CHECKINGN)_C_ops) get_logger) SDPBackend_get_backend_priority_get_enabled_backendsINFOz&%(asctime)s-%(levelname)s: %(message)s)fmt)_math_attention)Tensordtype)PlacecSS[RRSS9(a [R[R4O[R4S.SS[RRSS9(a/[R[R[R 4O[R[R 4S.S.qg)N)r) F)including_emulation)MINIMUM_SM_VERSIONMAXIMUM_SM_VERSIONsupport_dtypes)r) flash_attnmem_efficient_attn)paddledeviceis_bf16_supportedfloat16bfloat16float_configY/var/www/html/banglarbhumi/venv/lib/python3.13/site-packages/paddle/nn/functional/sdpa.py init_configr(*s#)")}}..5.I &~~v?.."  #)") }}..5.I   ..&,,/  Gr&c0US:XaX4$URS5URS5pURSSSUS/5URSSSUS/5pURSS5R5URSS5R5pX4$)z Repeat key and value tensors along the num_heads(3) dimension. The layout of key and value should be [batch_size, seq_len, num_heads, head_dim]. r) unsqueezeexpandflatten contiguous)keyvalue num_repeatss r' _repeat_kvr4Bs azq!5??1#5 BB R01 b"b+r23  Aq$$& a&&(  :r&c\rSrSr%S\S'S\S'S\S'S\S'S\S 'S \S 'S \S 'S \S'S\S'S\S'\SSj5r\SSj5r\SSj5r\SSj5r \SSj5r Sr g) SDPParamsVz paddle.Size query_shape key_shape value_shapezpaddle.Size | Noneattn_mask_shaper#dropoutbool is_causal float | Nonescalequery_stop_gradientztuple[dtype, dtype, dtype]rztuple[Place, Place, Place]placecZURSURSURS4$)Nrr8r9r:selfs r' batch_sizeSDPParams.batch_sizec."DNN1$5t7G7G7JJJr&cZURSURSURS4$)NrrDrEs r'seq_lenSDPParams.seq_lengrIr&cZURSURSURS4$)Nr,rDrEs r' num_headsSDPParams.num_headskrIr&cZURSURSURS4$)Nr+rDrEs r'head_dimSDPParams.head_dimos.#T^^B%79I9I"9MMMr&c>[SUR55nU$)Nc3n# UH+oR5(aUR5OSv M- g7f)r+N) is_gpu_place gpu_device_id).0pls r' &SDPParams.device_id..us+ FP//"3"3B    ;js35)tuplerB)rFrets r' device_idSDPParams.device_idss$ FJjj   r&r%N)returnztuple[int, int, int])r_ztuple[int, ...]) __name__ __module__ __qualname____firstlineno____annotations__rrGrKrNrQr]__static_attributes__r%r&r'r6r6Vs'' NO  %% %%KKKKKKNNr&r6r)maxsizecbUS:ag[RRRU5$)Nr)rr)rrcudaget_device_capabilityr]s r'riri{s'1} ==   3 3I >>r& cJ[U5up4X44nXs=:*=(a U:*$s $Nri)min_smmax_smr]majorminorcurrents r'check_sm_versionrts.)3LEnG  & & && &&r&rcv[R"5=(a [RR5$rm)ris_compiled_with_cudarh is_availabler%r&r'check_cuda_is_availablerxs#  ' ' ) Hfkk.F.F.HHr&c URSR5(dhURSR5(dF[R SURSSURSSURS35 gg) z7 Check all input tensors are placed on the GPU device. rzLAll input tensors should be placed on GPU or custom place, but query place: z , key place: rz, value place: r,FT)rBrUis_custom_place_loggerdebugparamss r'check_all_tensors_on_devicers}  Q$$&&&,,q/*I*I*K*K  "LLO,M||Av||A.? A  r&cURupnUS:d X:wdX#:wa[RSUSUSU35 gUS-S:wa[RSU35 gg ) z] Check the dimension of head in query, key, and value should be equal and all less than 256. zfThe dimension of head in query, key, and value should be equal and all less than 256, but q_head_dim: , k_head_dim: , v_head_dim: FrrzZThe dimension of head in query, key, and value should be a multiple of 8, but q_head_dim: T)rQr{r|)r~ q_head_dim k_head_dim v_head_dims r'check_head_dim_size_flashrs*0&JJC:3z7O  )l. NS]R^ ` A~  )l ,  r&c [R(a;[R"[RR 5S5(ag[ 5(d[RS5 g[[SS[SSU5(d<[RS[SSS [SSS [U535 gg) zJ Check flash attention requires CUDA support and SM between 8.0 and 12.1. rTz&Flash attention requires CUDA support.Frrrz$Flash attention requires SM between and , but found SM ) r FLASH_ATTENTIONris_compiled_with_custom_devicerget_all_device_typerxr{r|rtr$rirjs r'&check_flash_attention_hardware_supportrs !!f&K&K ))+A.'' " $ $ >?  23 23    27<3HI]3^2_7<()=>?$Y/0 2  r&cUR(dgURupnX:Xag[RSURSUR 35 g)zk Check flash attention only supports causal attention when the sequence length of query and key are equal. TzxFlash attention only supports causal attention when the sequencelength of query and key are equal, but got query shape: z , key shape: F)r>rKr{r|r8r9)r~q_lenk_len_s r'%check_flash_causal_non_square_seqlensrsZ   nnOE! ~ MM C    mF,<,<+= ? r&c URupnX:wdX2:wdU[SS;a,[RSUSUSUS[SS35 gg) 9 check QKV share the same dtype and are supported dtype. rrzoFlash attention requires query, key, and value to be of the same dtype and support dtype, but got query dtype: , key dtype: , value dtype: . Supported dtypes are: FTrr$r{r|r~q_dtypek_dtypev_dtypes r'check_dtypes_low_precision_farsu!' Gg   ',/0@A A  'y gYgY&>|$%567  9  r&c URupnX:wdX2:wdU[SS;a,[RSUSUSUS[SS35 gg) rrrzrMem_efficient_attn requires query, key, and value to be of the same dtype and support dtype, but got query dtype: rrrFTrrs r'-check_dtypes_low_precision_mem_efficient_attnrsx!' Gg   '"678HI I  'y gYgY&>+,-=>?  A  r&r,c<[U5up#US:agUS:XaU$g)NrTFrn)is_halfr]rqrs r'use_tensor_coresrs'$Y/HE z z r&cU[R[R4;n[X!5n[ U5upEUS:aSOSnU(aSOSnU(a[ USU- 5nU$)Nrrrk)rr!r"rrimax)rr]ruse_tcrqrmatmul_alignment_mnbits_per_scalars r'minimum_gemm_alignmentrsb88G g 1F$Y/HE$qy!a#bO !"5s_7LM r&c [5(d[RS5 g[[SS[SSU5(d<[RS[SSS[SSS[ U535 gg ) zR Check mem_efficient attention requires CUDA support and SM between 5.0 and 12.1. z.Mem efficient attention requires CUDA support.Frrrz,Mem efficient attention requires SM between rrT)rxr{r|rtr$rirjs r'$check_mem_efficient_hardware_supportr$s # $ $ FG $%&:;$%&:;    :7CW;XYm;n:o7/01EFG$Y/0 2  r&c URSURSURSp2n[URSUR S5nX-S:wdX$-S:wdX4-S:wa"[ RSUSUSUSU35 gg) Nr+rz:Mem efficient attention requires head dim size aligned to z, but found q_head_dim: rrFT)r8r9r:rrr]r{r|)r~rrr alignments r'!check_head_dim_size_mem_efficientr;s22'J 'v||A8H8H8KLI!#  !Q &  !Q & H T%%/Lzl.YcXd f  r&cUR(agURcgURSnUS-S:wa[RSUS35 gg)NTr+rrztMem efficient attention requires attn_mask last dimension to be divisible by 8 to satisfy vector alignment, but got z!. Falling back to other backends.F)r>r;r{r|)r~last_dims r'check_attn_mask_alignmentrOsa  %%%b)H!|q 44<:>. .  r&cJURcg[RS5 g)NTz/Paddle's FAV2 does not support scale parameter.F)r@r{r|r}s r'check_scale_is_Nonercs || MMCD r&c[[[[[/nUHnU"U5(aM g [ UR S5(dggNFrT)rrrrrrr])r~general_constraints constraints r'can_use_flash_attentionrjsQ#!-% * &!!* 2&2B2B12E F F r&c[[[[/nUHnU"U5(aM g [ UR S5(dggr)rrrrrr])r~ constraintsrs r'can_use_mem_efficient_attentionr}sM#)!5 K " &!!" 00@0@0C D D r&cd[R"5nSU;ag[5n[5nUHpnXB;aM U[R :Xa[ U5(a gM2U[R:Xa[U5(a gMZU[R:XdMp g [S5e)Nxpur mem_efficientmathz@No available backend for scaled_dot_product_attention was found.) r get_devicer r r rrEFFICIENT_ATTENTIONrMATH RuntimeError)paramrBenabled_backendspriority_orderbackends r'select_sdp_for_sdpars    E ~,.*,N!  *  j00 0&u--#.  66 6.u55&6   '" J r&c \ UR5S:Hn U (d3URS5nURS5nURS5nURSURSURSpn U (a0U S:XdX-S:XdSU SU S35eX:XdSU SUS35eOXs=:XaU:XdO SU S U S US 35eURunnnnURunnnnUS :XaUR5(a|UR5(agUR5(aRUb SUS35e[R R RRRUUUUUU5nU$[RR5(d#[RR54S-nO#URURUR4n[URURURUb UROS UUUUR UR"UR"UR"4US9 n[%[&5S:Xa [)5 URR+5S:H=(dA URR+5S:H=(d URR+5S:HnUbzUR"[R,:Xa\[R."U[R0"SUR"S9[R0"[3S5*UR"S95nU(aSnO [5U5n9SU-5 US:XaS nSnSnUbUUR:S:Xa UR=US/URQ5nO%UR:S:Xa[R"USS9n[>R@"UUUUUUUUU(+U5 un nOUS:XaSSK!J"nJ#n X-n![IXU!5upU(aU"5nOyUbvUR5S:XabURSS:waOURSU:wa<UR=URSUURSURS/5nU "UUUUUUUS9nOEUS:Xa1U S:waX-OSn![IXU!5up[KUUUUUUSUU5 SnO[MSU35eU (d[RN"USS9nU$) a The equation is: .. math:: result=softmax(\frac{ Q * K^T }{\sqrt{d}}) * V where : ``Q``, ``K``, and ``V`` represent the three input parameters of the attention module. The dimensions of the three parameters are the same. ``d`` represents the size of the last dimension of the three parameters. Warning: This API only verifies inputs with dtype float16 and bfloat16, other dtypes may fall back to math implementation, which is less optimized. Warning: If is_causal is set to True, the causal mask should not be provided, otherwise the provided mask will be ignored. Note: This API differs from :ref:`api_paddle_compat_nn_functional_scaled_dot_product_attention` in that: 1. The QKV layout of this API is [batch_size, seq_len, num_heads, head_dim] or [seq_len, num_heads, head_dim]. If you need num_heads before seq_len layout, please use ``paddle.compat.nn.functional.scaled_dot_product_attention``. Args: query(Tensor): The query tensor in the Attention module. 4-D tensor with shape: [batch_size, seq_len_key, num_heads, head_dim]. 3-D tensor with shape: [seq_len_key, num_heads, head_dim]. The dtype can be float16 or bfloat16. key(Tensor): The key tensor in the Attention module. 4-D tensor with shape: [batch_size, seq_len_key, num_heads, head_dim]. 3-D tensor with shape: [seq_len_key, num_heads, head_dim]. The dtype can be float16 or bfloat16. value(Tensor): The value tensor in the Attention module. 4-D tensor with shape: [batch_size, seq_len_value, num_heads, head_dim]. 3-D tensor with shape: [seq_len_value, num_heads, head_dim]. The dtype can be float16 or bfloat16. attn_mask(Tensor, optional): The attention mask tensor. The shape should be broadcastable to [batch_size, num_heads, seq_len_key, seq_len_query]. The dtype can be bool or same type of query. The bool mask indicates the positions should take part in attention. The non-bool mask will be added to attention score. dropout_p(float, optional): The dropout ratio. is_causal(bool, optional): Whether enable causal mode. training(bool, optional): Whether it is in the training phase. backend(str, optional): Specify which backend to compute scaled dot product attention. Currently only support "p2p" for distribution usage. scale(float, optional): The scaling factor used in the calculation of attention weights. If None, scale = 1 / sqrt(head_dim). enable_gqa(bool, optional): Whether enable GQA(Group Query Attention) mode. Default is True. name(str|None, optional): The default value is None. Normally there is no need for user to set this property. For more information, please refer to :ref:`api_guide_Name`. Returns: out(Tensor): The attention tensor. 4-D tensor with shape: [batch_size, seq_len, num_heads, head_dim]. 3-D tensor with shape: [seq_len, num_heads, head_dim]. The dtype can be float16 or bfloat16. Examples: .. code-block:: python >>> # doctest: +SKIP('bfloat need V100 compile') >>> import paddle >>> q = paddle.rand((1, 128, 2, 16), dtype=paddle.bfloat16) >>> output = paddle.nn.functional.scaled_dot_product_attention(q, q, q, None, 0.9, False) >>> print(output) >>> # doctest: -SKIP rrr,zThe number of groups in query(z3) must be divisible by the number of groups in key(z) if GQA enabled.zThe number of groups in key(z1) must be equal to the number of groups in value(z/) must be equal to the number of groups in key(z$) and the number of groups in value(z) if GQA disabled.p2pNzBackend z not support scale parameter.r*) r8r9r:r;r<r>r@rArrB)rinfrzSelected backend:rFr)axisr)LowerTriangularMaskmemory_efficient_attention) attn_biaspr@trainingzInvalid backend )(dimr-shapeis_distr distributed auto_parallelring_attentionRingFlashAttentionapplybasein_dygraph_mode framework_current_expected_place_rBr6 stop_gradientrlenr$r(numelr=where to_tensorr#rr{r|ndimr.rr-paddle.incubate.nn.memory_efficient_attentionrrr4r ValueErrorsqueeze)"queryr1r2 attn_mask dropout_pr>rrr@ enable_gqaname is_batchedk_headsq_headsv_headsbs seq_len_q num_heads_q head_dim_qr seq_len_k num_heads_k head_dim_kout qkv_placer is_zero_size sdp_func_namefixed_seed_offsetreturn_softmaxrng_namerrrepeatss" r'scaled_dot_product_attentionrs(p!J "mmA" !  A AG !|w0A5 ,WI5hiphqrC D 5! *7)3deldmm~  !,W, ,WI5deldmn118 9K M ,.3[[*B ; ,/II)Ay+z 5 MMOO KKMM MMOO}O 1NOO}  ..==PPVV         ;; & & ( (%%>>@BQF [[#))U[[9 KK))KK+4+@ d!//{{CIIu{{3 E 7|q   q  $ 99??  ! $ ;;   ! #  ??fkk )   EKK8  %,ekkBI  +E2  MM% 56 $   ~~"%,,b!-Fioo-FG 1$",,YQ? ((        L   Q1 / ) $G4  +-I  " 1$OOA&!+OOA&+5%,,!*#!*!*  )     & (/1 '$!G4               +G9566 nnSq) Jr&)r1rr2rr3int)r]rr_tuple[int, int])r)rorrprr]rr_r=)r_r=)r~r6)r]r)rr=r]rr_r=)rrr]r)r~r6r_r=)F)rr6r_str)NrFTNNTN)rrr1rr2rrz Tensor | Nonerr#r>r=rr=r str | Noner@r?rr=rr r_r)0 __future__r dataclassesr functoolsrrtypingrrrpaddle.base.log_helperr paddle.nn.attention.sdpar r r r`r{$paddle.nn.functional.flash_attentionrrrpaddle.base.corerr$r(r4r6rirtrxrrrrrrrrrrrrrrrrr%r&r'rs#!0 -  fB A$& 0( !! !H 1??  2GH' '%4'AD' '' 1II  * 16&** 1 2 1,((& H $z z z z z  z  zzz zz z zr&