͑ijSSKJr SSKJr SSKJrJr SSKrSSKJ r \(aSSKJ r Sr "SS 5r "S S \ 5r "S S 5r"SS\5r"SS\5rSr\SSSjj5r\SSSjj5r\SS Sjj5r\SS!Sjj5rS"Sjr\SS#Sjj5r\SS$Sjj5rS"SjrSrS"Sjrg)%) annotations)Sequence) TYPE_CHECKINGoverloadN) framework)Tensorc[U[R5(aU$[U[5(a [ U5$U$N) isinstancerVariablertuple)xss X/var/www/html/banglarbhumi/venv/lib/python3.13/site-packages/paddle/autograd/autograd.py as_tensorsrs6"i(()) B ! !Ry c\rSrSrSrSSSjjr\SSj5rSrSSjr Sr Sr S r S r S rS rS rSrSrSrSrSrSrSrSrSrg)Jacobian#aComputes the Jacobian matrix of given xs and ys. Once the Jacobian ``J`` is constructed, you can use a multidimensional index to retrieve the submatrix of ``J``, as same as slicing a Tensor. The submatrix is lazily evaluated along row axis, and will be cached once evaluated. you can retrieve the submatrix by following methods: * J[:], retrieving the full matrix. * J[:, :, j], retrieving the partial derivatives w.r.t. the j'th input variable. * J[:, i, :], retrieving the partial derivatives w.r.t. the i'th output variable. * J[:, i, j], retrieving the partial derivatives w.r.t. the i'th output variable and the j'th input variable. Notes: Ellipsis index is not supported currently. Args: ys (Tensor|Tuple[Tensor, ...]): The output derived from xs . xs (Tensor|Tuple[Tensor, ...]): The input tensor(s) . is_batched (bool): If true, the first axis is batch axis. Defaults to False. Returns: Jacobian (Object): A python object retains the Jacobian matrix. clU(dS[UR5s=::aS::d#O [S[UR535eS[UR5s=::aS::d#O [S[UR535e[X5UlgS[UR5s=::aS::d#O [S[UR535eS[UR5s=::aS::d#O [S[UR535e[ X5Ulg)Nrz7xs.ndim should be 0 or 1 when is_batched=False but got z7ys.ndim should be 0 or 1 when is_batched=False but got z6ys.ndim should be 1 or 2 when is_batched=True but got z6xs.ndim should be 1 or 2 when is_batched=True but got )lenshape ValueError_JacobianNoBatch _jacobian_JacobianBatchFirst)selfysr is_batcheds r__init__Jacobian.__init__Gs  BHH **  #BHH 0BHH **  #BHH 0.b5DNBHH **  #BHH 0BHH **  #BHH 018DNrc.URR$)z'The shape of flattened Jacobian matrix.)rrrs rrJacobian.shapefs~~###rc URU$r r)rindexess r __getitem__Jacobian.__getitem__ks~~g&&rcUS:Xa[URU5$US:Xa[URU5$[URR5U5$)Nr _evaluate_all)getattrrr,)r_Jacobian__names r __getattr__Jacobian.__getattr__nsN W 4>>62 2 _ $4>>62 2t~~335v>>rcxUR5n[U[5(aUR5OUnX#-$r r,r rrotherlhsrhss r__add__Jacobian.__add__u4  "'1%'B'Be!!#yrcxUR5n[U[5(aUR5OUnX#- $r r2r3s r__sub__Jacobian.__sub__zr9rcxUR5n[U[5(aUR5OUnX#-$r r2r3s r__mul__Jacobian.__mul__r9rcxUR5n[U[5(aUR5OUnX#- $r r2r3s r__div__Jacobian.__div__r9rcxUR5n[U[5(aUR5OUnX#- $r r2r3s r __truediv__Jacobian.__truediv__r9rcxUR5n[U[5(aUR5OUnX#-$r r2r3s r__pow__Jacobian.__pow__s3  "'1%'B'Be!!#xrcxUR5n[U[5(aUR5OUnX#-$r r2r3s r__mod__Jacobian.__mod__r9rcxUR5n[U[5(aUR5OUnX#-$r r2r3s r __floordiv__Jacobian.__floordiv__4  "'1%'B'Be!!#zrcxUR5n[U[5(aUR5OUnX#-$r r2r3s r __matmul__Jacobian.__matmul__r9rcxUR5n[U[5(aUR5OUnX#:H$r r2r3s r__eq__Jacobian.__eq__rOrcxUR5n[U[5(aUR5OUnX#:g$r r2r3s r__ne__Jacobian.__ne__rOrcxUR5n[U[5(aUR5OUnX#:$r r2r3s r__lt__Jacobian.__lt__r9rcxUR5n[U[5(aUR5OUnX#:*$r r2r3s r__le__Jacobian.__le__rOrcxUR5n[U[5(aUR5OUnX#:$r r2r3s r__gt__Jacobian.__gt__r9rcxUR5n[U[5(aUR5OUnX#:$r r2r3s r__ge__Jacobian.__ge__rOrr'N)F)rrrrr boolreturnNone)rfz list[int])r.str)__name__ __module__ __qualname____firstlineno____doc__r!propertyrr)r/r7r;r>rArDrGrJrMrQrTrWrZr]r`rc__static_attributes__rrrr#s!N! 9 9 9 9  9>$$'?              rrc\rSrSrSrg)HessianrpN)rirjrkrlrorprrrrrrsrrrc\\rSrSrSrSr\S5rSrSr SSjr Sr S r S r S rS rg ) _JacobianaThe base class for computing Jacobian matrix. ``_Jacobian`` implements the core logic of multidimensional index and lazy evaluation for Jacobian matrix, subclass only need to overwrite following methods: * ``_lazy_axis()``, return the axis along which will be lazy evaluating. * ``_flatten(xs)``, flattens the inputs ``xs``. * ``_evaluate(index)``, evaluates one slice along ``_lazy_axis`` . Notes: Because currently PaddlePaddle only support reverse differentiation by ``paddle.grad``, so lazy evaluation is only supported along the row of Jacobian matrix, which means that slicing along row will get better performance. chURUlURUlX lXl[ URR5S:Xa2UR (d!URRS/5Ul[ URR5S:Xa3UR (a"URRSS/5UlUR[UR55Ul UR[UR55Ul 0Ul g)Nrr) roriginal_xs_shapeoriginal_ys_shape_xs_ysrr reshape_flattenr _flatten_xs _flatten_ys_cache)rrrs rr!_Jacobian.__init__s!#!# txx~~ ! #DOOxx''DH txx~~ ! #xx''Q0DH==DHH)=>==DHH)=> rc[e)z"The axis of lazily evaluated.NotImplementedErrorr$s r _lazy_axis_Jacobian._lazy_axiss "!rcXRn[U[5(aU4$[[ UR UR UR55$r )rr intr rangestartstopstep)rr(idxs r _lazy_indexes_Jacobian._lazy_indexessMoo&#s##F uSYY#((;< rc[er rrrs rr~_Jacobian._flattens!!rcXRn[U[5(aSO [SUS5n/USURQUPXRS-SQ7$)Nrr)rr rslice)rr(lazy_axis_sizershifted_lazy_axis_idxs r_shifted_indexes_Jacobian._shifted_indexessloo&C%%A5NA+F  &t ' ! __q(* +  rc0URSLa_[UR5S:Xa [S5e[UR5S:Xa![UR5S:XaSU4OUS4nO[UR5S:XaUSS4nOm[UR5S:XaT[ U[ 5(aU[ SSS54nO/[UR5S:Xa USSUS4O USUSS4n[XR5n[ XR[5(aAUSURXRS-S-nURXR5U$URU5n[UR5n[U5X@R'[R"UVs/sHoPRU5PM snURS9R!U5nX`R#U[U55n[UR5[UR5:aL[%[UR5[UR5- 5HnUR'S5nM U$s snf)NFrz0-D tensor can not be indexed.rr)axisrx)r rr IndexErrorrzr r _multi_index inner_shaperr_cached_evaluaterlistpaddleconcatr}rrsqueeze) rr( other_indexes lazy_indexesripart_jacresult_s rr)_Jacobian.__getitem__sN ??e #4::!# !ABBTZZA%4112a7L!1 4::!#"Aq/TZZA%gu--&dD$(?@Gt556!;!Q 3%aj'!*a8 w(8(89 goo. 4 4)$//*W__q5H5J-KK (()AB ))'2 T%%&!$\!2oo==/; <|! " "1 %| <  '%. //\9JKL v|| s4:: .3v||,s4::>?+@  =sJcUc URS5R/5$URRU5nUcUR U5nX RU'U$Nr)rr}rget _evaluate)rkvs rr_Jacobian._cached_evaluate=sX 9((+33B7 7 KKOOA  9q!AKKNrc[e)z&Evaluate one slice at along lazy axis.r)rindexs rr_Jacobian._evaluateFs!!rc`[UR5S:XaURS5$USS$r)rrrr$s rr,_Jacobian._evaluate_allJs, tzz?a ((. .7Nr)rrrr{r|ryrzN)r)rirjrkrlrmr!rnrrr~rr)rrr,rorprrrurusD($"" "  4l"rrucH^\rSrSrSrU4Sjr\S5rSrSr Sr U=r $)riQzCompute Jacobian matrix without batch dimension. Suppose the mapping is :math:`f: R^M o R^N`, the output shape is ``(N, M)`` . c>SUl[TU] X5 /URRSSQUR RSSQUl/URSSQURSSQUlg)NFrr) r superr!rrrrrzryrrr __class__s rr!_JacobianNoBatch.__init__Ws   $$Qq) $$Qq)  $$Qq) $$Qq)  rcgrrpr$s rr_JacobianNoBatch._lazy_axisdrc[U[5(dURS5$[R"[ SU555$)Nrxc3B# UHoRS5v M g7f)rN)r}.0xs r ,_JacobianNoBatch._flatten..ks"@R99U#3#3Rs)r rr}rrr rs rr~_JacobianNoBatch._flattenhs8"h''::e$ $}}U"@R"@@AArcfUR[URUUR55$r r~_grad_for_jacobianrr{r row_indexs rr_JacobianNoBatch._evaluatems0}}   +   rrr r rirjrkrlrmr!rnrr~rro __classcell__rs@rrrQs1  B   rrcH^\rSrSrSrU4Sjr\S5rSrSr Sr U=r $)rivzCompute Jacobian matrix with batch at first axis. Suppose the mapping is :math:`f: R^{B,M} o R^{B,N}`, the output shape is ``(B, N, M)`` . cR>SUl[TU] X5 /URRSSQUR RSSQURRSSQUl/URRSSQURSSQURSSQUlg)NTrrr) r rr!rrrrrzryrs rr!_JacobianBatchFirst.__init__|s   $$Qq) $$Qq) $$Qq)   $$Qq) $$Qq) $$Qq)  rcg)Nrrpr$s rr_JacobianBatchFirst._lazy_axisrrc[U[5(d URURSS45$[R "[ S[U555S5$)Nrrxc3`# UH$oRURSS45v M& g7f)rrxN)r}rrs rr/_JacobianBatchFirst._flatten..s'F~!))QWWQZ,--~s,.r)r rr}rrrr rrs rr~_JacobianBatchFirst._flattensP"h''::rxx{B/0 0}} Fz"~F F  rcnUR[URSS2U4UR55$r rrs rr_JacobianBatchFirst._evaluates0}} t//9 =txx H  rrrrs@rrrvs0     rrc L[U[5(aUOU4n[SU55(a [S5eU[ SSS54[ U5[ U5- --n/n[ U5GH#up4[U[5(a[ UR=(d SUR=(d XUR=(d S5nUR[ URS:aURX-O URURS:aURX-O URUR55 M[U[5(a!URUS:aXAU-OU5 GM[SUS35e [U5$)a.A tool for parsing N-dimensional index into a standard format. Currently supporting following input format: * ([positive|negative|slice], ...), the right-most elements can be omitted. The standard format after converted is slice tuple which contains N elements: * ([positive|slice], ..., [positive|slice]) Notes: Ellipsis indexes such as ``(..., i), (i, ...)`` is not supported. Args: indexes (tuple): The input indexes. shape (tuple): The input shape. Returns: tuple: The standard format index as the above description. c3T# UHn[U[[55v M g7fr )r typeEllipsis)rrs rr_multi_index..s :'Q:ah ( ('s&(z*Ellipsis index currently is not supported.rNrzNot supported index type .)r ranyrrr enumeraterrrappendr TypeErrorr )r(rpositive_indexesrrs rrrs[($GX66gWJG :' :::EFFq$-/3u:G 3LMMGg& eU # #  q%**"8%**/E  # #.3kkAoEKK%(*5;;-2ZZ!^EJJ)JJ  s # #  # # E!H$4u M7wa@A A#'$ ! ""rcgr rprr batch_axiss rjacobianrs rcgr rprs rrrs (+rcgr rprs rrr rcgr rprs rrrrrc^^^UbUS:wa[SUS35eUSLm[T[5(a-[T[5(a[UU4SjT55nU$[T[5(a-[T[5(d[UU4SjT55nU$[T[5(d-[T[5(a[UU4SjT55nU$[ TTT5nU$)aT Computes the Jacobian of the dependent variable ``ys`` versus the independent variable ``xs``. Where ``ys`` represents the output of ``xs`` after a certain operation, ``ys`` and ``xs`` can be Tensor or tuple of Tensors, ``batch_axis`` indicates the position of the batch dimension of the parameter data. When the input is a tuple Tensors, the returned result is a ``Jacobian`` object with the same number of nesting levels as ``xs``, and each Jacobian has the same shape as The ``xs`` tuples are identical in one-to-one correspondence. - When ``batch_axis=None``, only 0-dimensional Tensor or 1-dimensional Tensor is supported, assuming the shape of ``xs`` is ``[N, ]``, the shape of ``ys`` is ``[M, ]``, then the output Jacobian matrix shape is ``[M, N]``. - When ``batch_axis=0``, only 1-dimensional Tensor or 2-dimensional Tensor is supported, assuming the shape of ``xs`` is ``[B, N]``, The shape of ``ys`` is ``[B, M]``, then the output Jacobian matrix shape is ``[B, M, N]``. After the ``Jacobian`` object is created, the actual calculation process does not occur, but the lazy evaluation method is used for calculation. It can be multi-dimensional indexed to obtain the entire Jacobian matrix or sub-matrix, and the actual calculation will be performed at this time the value is calculated and the result is returned. At the same time, in the actual evaluation process, the calculated sub-matrix will be cached to avoid duplicate calculations in the subsequent indexing process. For example, assuming ``Jacobian`` instance ``J`` has shape ``[B, M, N]``, assuming ``M > 4`` , then ``J[:, 1:4:1, :]`` means to get the values from row ``1`` to row ``3`` of ``J``. In actual calculation, only the rows ``1`` to ``3`` are evaluated, and the calculation results of ``1`` to ``3`` will be cached at the granularity of the row, and will be used next time. When obtaining one or more rows of results above, the already calculated parts will not be recalculated. Args: ys (Union[paddle.Tensor, Tuple[paddle.Tensor, ...]]): Output or tuple of outputs derived from xs. xs (Union[paddle.Tensor, Tuple[paddle.Tensor, ...]]): Input or tuple of inputs. batch_axis (Optional[int], optional): Index of batch axis. Defaults to None. Returns: Union[Tuple[Tuple[Jacobian, ...], ...], Tuple[Jacobian, ...], Jacobian]: Jacobian(s) of ys derived from xs. Examples: .. code-block:: pycon >>> import paddle >>> x1 = paddle.randn([3]) >>> x2 = paddle.randn([3]) >>> x1.stop_gradient = False >>> x2.stop_gradient = False >>> y = x1 + x2 >>> J = paddle.autograd.jacobian(y, (x1, x2)) >>> J_y_x1 = J[0][:] # evaluate result of dy/dx1 >>> J_y_x2 = J[1][:] # evaluate result of dy/dx2 >>> print(J_y_x1.shape) paddle.Size([3, 3]) >>> print(J_y_x2.shape) paddle.Size([3, 3]) Nrz(batch_axis should be None or 0, but got rc3R>^# UHm[UU4SjT55v M g7f)c3>># UHn[TUT5v M g7fr r)rr{r|r s rr%jacobian...>s?BS(3Z00BN)r rr|r rs @rrjacobian..=s" KMCE?B? ? ?2s#'c3>># UHn[UTT5v M g7fr rrs rrrAsF2C(3J772rc3>># UHn[TUT5v M g7fr r)rr{r rs rrrCsF2C(2sJ772r)rr rr r)rrrrr s`` @rrrsR*/6zl! D  4'J"hJr8$<$< KM    B ! !*R*B*BF2FF  H % %*R*B*BF2FF  RZ0 rcgr rprs rhessianrJs rcgr rprs rrrRs '*rc^^TcCUR5S:a[SUR5S35eURS5nO[T[5(aWUSR5S:a[SUR5S35eTS:wa [S5eURS 5nO[S [ T5S 35e[ UTT5n[T[5(d[ UTT5n[Ul U$[UU4S jU55n[[U55H1n[[US55Hn[XEUl M M3 U$) a Computes the Jacobian of the dependent variable ``ys`` versus the independent variable ``xs``. Among them, ``ys`` means the output of ``xs`` after a certain operation, ``ys`` can only be a single Tensor, ``xs`` can be a Tensor or a Tensor tuple, and ``batch_axis`` means The position of the batch dimension of the parameter data. When the input ``xs`` is a Tensor tuple, the returned result is a ``Hessian`` tuple, assuming that the internal shape of the ``xs`` tuple is composed of ``([M1, ], [M2, ])``, the shape of the returned result consists of ``(([M1, M1], [M1, M2]), ([M2, M1], [M2, M2]))`` - When ``batch_axis=None``, only 0-dimensional Tensor or 1-dimensional Tensor is supported, assuming that the shape of ``xs`` is ``[N, ]``, and the shape of ``ys`` is ``[ ]`` (0-dimensional Tensor), the final output is a single Hessian matrix whose shape is ``[N, N]``. - When ``batch_axis=0``, only 1-dimensional Tensor or 2-dimensional Tensor is supported, assuming that the shape of ``xs`` is ``[B, N]``, and the shape of ``ys`` is ``[B, ]``, the final output Jacobian matrix shape is ``[B, N, N]``. After the ``Hessian`` object is created, the complete calculation process does not occur, but a partial lazy evaluation method is used for calculation. It can be multi-dimensionally indexed to obtain the entire Hessian matrix or sub-matrix. At this time, the actual Evaluates the computation and returns the result. At the same time, in the actual evaluation process, the calculated sub-matrix will be cached to avoid repeated calculations in the subsequent indexing process. Args: ys (paddle.Tensor): Output derived from xs which contain one element. xs (Union[paddle.Tensor, Tuple[paddle.Tensor, ...]]): Input or tuple of inputs. batch_axis (Optional[int], optional): Index of batch axis. Defaults to None. Returns: Union[Tuple[Tuple[Hessian, ...], ...], Tuple[Hessian, ...], Hessian]: Hessian(s) of ys derived from xs. Examples: .. code-block:: pycon >>> import paddle >>> x1 = paddle.randn([3]) >>> x2 = paddle.randn([4]) >>> x1.stop_gradient = False >>> x2.stop_gradient = False >>> y = x1.sum() + x2.sum() >>> H = paddle.autograd.hessian(y, (x1, x2)) >>> H_y_x1_x1 = H[0][0][:] # evaluate result of ddy/dx1x1 >>> H_y_x1_x2 = H[0][1][:] # evaluate result of ddy/dx1x2 >>> H_y_x2_x1 = H[1][0][:] # evaluate result of ddy/dx2x1 >>> H_y_x2_x2 = H[1][1][:] # evaluate result of ddy/dx2x2 >>> print(H_y_x1_x1.shape) paddle.Size([3, 3]) >>> print(H_y_x1_x2.shape) paddle.Size([3, 4]) >>> print(H_y_x2_x1.shape) paddle.Size([4, 3]) >>> print(H_y_x2_x2.shape) paddle.Size([4, 4]) rzOnly support ys.numel()(z)==1 when batch_axis is None.rprzOnly support ys[0].numel()(z)==1 when batch_axis is intzOnly support batch_axis=0 yet.rz*batch_axis should be None or int, but got rc3>># UHn[UTT5v M g7fr )r)r_jrrs rrhessian..sIyR44yr) numelrr}r rrrrrrrr rr)rrrrrrjs `` rrrZsXJ 88:>*288:,6ST ZZ^ J $ $ a5;;=1 -bhhj\9TU  ?=> > ZZ 8j9I8J! L  R,I b( # #9b*5$ NIyIIs7|$A3wqz?+*1 1 ',% Nrc^Uc)[R"T5nTRUlU$[U[5(a[ U4Sj[ U555$U$)Nc3F># UHup[UTU5v M g7fr )_replace_none_with_zero_tensor)rrrrefss rr1_replace_none_with_zero_tensor..s% CP41 *1d1g 6 6=s!)r zeros_like stop_gradientr rr r)rrs `rrrs] z   t $-- B ! ! CLR=    rc[R"5(at[R"XUSSS9n[U[RR R 5(a)[U[5(a[U5S:aUSnOh[RRXU5n[U[ R 5(a)[U[5(a[U5S:aUSn[X15$)aA gradient function that can be used in dynamic graph and static graph. The ``grad`` combines ``paddle.grad`` used in dynamic graph and ``paddle.static.gradients`` used in static graph, and do following changes: * The ``allow_unused`` flag is removed and set defaults to true internally, none in outputs will be replaced by zero tensor. * The ``create_graph`` flag is removed and set defaults to true internally, only makes sense in dynamic graph. * When xs is a single Tensor, ``paddle.grad`` returns a list which only contains one Tensor. It may confuse users, thus in this case we improve to return a single Tensor in _grad_for_jacobian interface. Args: ys (Tensor|Sequence[Tensor]): The output tensor or tensor sequence of the graph to compute gradients. xs (Tensor|Sequence[Tensor]): The input tensor or tensor sequence of the graph to compute gradients. The returned values of this API are the gradients of inputs . v (Tensor|Sequence[Tensor]|None,optional): The initial gradient values of outputs . If grad_outputs is None, the initial gradient values of outputs would be Tensors filled with 1; if grad_outputs is not None, it must have the same length as outputs , and in this case, the initial gradient value of the i-th outputs would be: (1) a Tensor filled with 1 when the i-th element of grad_outputs is None; (2) the i-th element of grad_outputs when the i-th element of grad_outputs is a Tensor. Default None. Returns: Tensor|tuple[Tensor]: Tensor or a tuple of Tensors, whose length is the same as the Tensor number inside inputs, and the i-th returned Tensor is the sum of gradients of outputs with respect to the i-th inputs. T) create_graph allow_unusedr) rin_dynamic_modegradr baserr rrstatic gradientsr)rrrxs_grads rrrsF++badN r6;;0099 : :7H--G q ajG--))"!4 r9-- . .7H--G q ajG )' 66r).)rrrrr int | Nonerfr)rSequence[Tensor]rrrrrfz tuple[tuple[Jacobian, ...], ...])rrrrrrrftuple[Jacobian, ...])rrrrrrrfrr )rrrrrrrfrr)rrrrrrrfztuple[tuple[Hessian, ...], ...]) __future__rcollections.abcrtypingrrr paddle.baserrrrrrrurrrrrrrrprrrs#$* ![[| h IIX" y" J# )# L-#` !   !++++& + + !   !  [| !   !****% * *gT 67r