ϑi5SSKJr SSKJrJr SSKrSSKrSSKJ r SSK J r J r J r \(a SSKJr SSKJr S S S jjrg) ) annotations) TYPE_CHECKINGLiteralN) strong_wolfe)_value_and_gradient&check_initial_inverse_hessian_estimatecheck_input_type)Callable)Tensorc ^^^^^^^^ ^ ^^T S;a[ST S35eSn [USU 5 Uc#[R"URST S9mO[USU 5 [ U5 Um[R "UR55n [TU 5up[R"S /SS S 9n[R"S /S S S 9n[R"S /S S S 9n[R"S /S S S 9n[R"/TS S 9m[R"S /S S S 9n[R"S /SS S 9nURSn[R"TS -U4T S9n[R"TS -U4T S9n[R"TS -S 4T S9n[R"TS -S 4T S9mU4SjnUUU UU UUUUU4 Sjn[RRRUUUUUUUU UUUUUU/ S9 UUXU4$)ah Minimizes a differentiable function `func` using the L-BFGS method. The L-BFGS is a quasi-Newton method for solving an unconstrained optimization problem over a differentiable function. Closely related is the Newton method for minimization. Consider the iterate update formula: .. math:: x_{k+1} = x_{k} + H_k \nabla{f_k} If :math:`H_k` is the inverse Hessian of :math:`f` at :math:`x_k`, then it's the Newton method. If :math:`H_k` is symmetric and positive definite, used as an approximation of the inverse Hessian, then it's a quasi-Newton. In practice, the approximated Hessians are obtained by only using the gradients, over either whole or part of the search history, the former is BFGS, the latter is L-BFGS. Reference: Jorge Nocedal, Stephen J. Wright, Numerical Optimization, Second Edition, 2006. pp179: Algorithm 7.5 (L-BFGS). Args: objective_func: the objective function to minimize. ``objective_func`` accepts a 1D Tensor and returns a scalar. initial_position (Tensor): the starting point of the iterates, has the same shape with the input of ``objective_func`` . history_size (Scalar): the number of stored vector pairs {si,yi}. Default value: 100. max_iters (int, optional): the maximum number of minimization iterations. Default value: 50. tolerance_grad (float, optional): terminates if the gradient norm is smaller than this. Currently gradient norm uses inf norm. Default value: 1e-7. tolerance_change (float, optional): terminates if the change of function value/position/parameter between two iterations is smaller than this value. Default value: 1e-9. initial_inverse_hessian_estimate (Tensor, optional): the initial inverse hessian approximation at initial_position. It must be symmetric and positive definite. If not given, will use an identity matrix of order N, which is size of ``initial_position`` . Default value: None. line_search_fn (str, optional): indicate which line search method to use, only support 'strong wolfe' right now. May support 'Hager Zhang' in the future. Default value: 'strong wolfe'. max_line_search_iters (int, optional): the maximum number of line search iterations. Default value: 50. initial_step_length (float, optional): step length used in first iteration of line search. different initial_step_length may cause different optimal result. For methods like Newton and quasi-Newton the initial trial step length should always be 1.0. Default value: 1.0. dtype ('float32' | 'float64', optional): data type used in the algorithm, the data type of the input parameter must be consistent with the dtype. Default value: 'float32'. name (str, optional): Name for the operation. For more information, please refer to :ref:`api_guide_Name`. Default value: None. Returns: output(tuple): - is_converge (bool): Indicates whether found the minimum within tolerance. - num_func_calls (int): number of objective function called. - position (Tensor): the position of the last iteration. If the search converged, this value is the argmin of the objective function regarding to the initial position. - objective_value (Tensor): objective function value at the `position`. - objective_gradient (Tensor): objective function gradient at the `position`. Examples: .. code-block:: python :name: code-example1 >>> # Example1: 1D Grid Parameters >>> import paddle >>> # Randomly simulate a batch of input data >>> inputs = paddle. normal(shape=(100, 1)) >>> labels = inputs * 2.0 >>> # define the loss function >>> def loss(w): ... y = w * inputs ... return paddle.nn.functional.square_error_cost(y, labels).mean() >>> # Initialize weight parameters >>> w = paddle.normal(shape=(1,)) >>> # Call the bfgs method to solve the weight that makes the loss the smallest, and update the parameters >>> for epoch in range(0, 10): ... # Call the bfgs method to optimize the loss, note that the third parameter returned represents the weight ... w_update = paddle.incubate.optimizer.functional.minimize_bfgs(loss, w)[2] ... # Use paddle.assign to update parameters in place ... paddle.assign(w_update, w) .. code-block:: python :name: code-example2 >>> # Example2: Multidimensional Grid Parameters >>> import paddle >>> def flatten(x): ... return x. flatten() >>> def unflatten(x): ... return x.reshape((2,2)) >>> # Assume the network parameters are more than one dimension >>> def net(x): ... assert len(x.shape) > 1 ... return x.square().mean() >>> # function to be optimized >>> def bfgs_f(flatten_x): ... return net(unflatten(flatten_x)) >>> x = paddle.rand([2,2]) >>> for i in range(0, 10): ... # Flatten x before using minimize_bfgs ... x_update = paddle.incubate.optimizer.functional.minimize_bfgs(bfgs_f, flatten(x))[2] ... # unflatten x_update, then update parameters ... paddle.assign(unflatten(x_update), x) )float32float64z?The dtype must be 'float32' or 'float64', but the specified is .minimize_lbfgsinitial_positionrdtype initial_inverse_hessian_estimaterint64shape fill_valuerFboolc >UT :U)-$N) kdone is_convergenum_func_callsvaluexkg1sk_vecyk_vecrhok_vecheadtail max_iterss j/var/www/html/banglarbhumi/venv/lib/python3.13/site-packages/paddle/incubate/optimizer/functional/lbfgs.pycondminimize_lbfgs..condsI $&&c 4> ^^^ ^ ^ ^^"U5n [R"/T S- RT5SS9n U 4SjnUU UU4Sjn[RR R XXT/S9 [R"TU 5n[R"/T S-SS9n U 4SjnUUU UU4Sjn[RR R XU U/S9 U*nT!S :Xa[T#UUT"T TS 9unnnnO[S T!S 35e[R"UU-U5 UU-nUU- n[R"UU5m[RR RTS :HU4SjU4Sj5n[R"5(aUTT 'UTT 'UT T 'Oc[RRTT U5m[RRTT U5m[RRT T U5m T S-T-m SmR RT T :HU U4SjS5 UU-nUnUS- n[RRU[ R"S9n[RRU[ R"S9n[R"UUT%:-UT$:-U5 [R"X5 [R"UUS :H-U5 UUUUUUUTTT T T / $)Nrrrc>UT:g$rr)iqai_vecr)s r+r,*minimize_lbfgs..body..cond 9 r.c D>[R"5(a#TU[R"TUU5-X 'O>[RR X TU[R"TUU5-5nXUTU-- nUS- R T5nXU4$Nr)paddlein_dynamic_modedotstaticsetitemmod)r1r2r3 history_sizer'r%r&s r+body*minimize_lbfgs..body..bodys%%''$QK&**VAY*BB ..x{VZZq 1-EE1Iq ))AQ L)A< r.r,r? loop_varsc>UT:g$rr)r1rr(s r+r,r4r5r.c>TU[R"TUU5-nUTUTUU- --nUS-RT5nX4$r7)r8r:r=)r1rDbetar3r>r'r%r&s r+r?r@sUA;F1Iq!99DF1IT!122AQ L)A4Kr.r)fr#pkr*initial_step_lengthrzNCurrently only support line_search_fn = 'strong_wolfe', but the specified is ''gc2>[R"S/STS9$)Nrg@@r)r8fullrsr+.minimize_lbfgs..body..sFKKqcfEJr.c>ST- $)N?r)rhok_invsr+rMrNs C(Nr.c8[R"US-U5 gr7)r8assign)r)s r+true_fn-minimize_lbfgs..body..true_fn s MM$(D )r.c>T"T5$rr)r)rTsr+rMrN$s GDMr.)p)r8rSrLr=r;nn while_loopmatmulrNotImplementedErrorr:r,r9r<linalgnormnpinf)&rrr r!r"r#r$r%r&r'r(r)r2r1r,r?rDrHalphag2 ls_func_callsskykrhokgnormpk_normrQrTH0r3rr>rIline_search_fnmax_line_search_itersobjective_functolerance_changetolerance_grads& ````` @@r+r?minimize_lbfgs..bodys" MM"  KK$( !=W     ##Q6N $  MM"a  KKbTAXW E     ##QF#KR ^ +.: /$7 / +E5"m&`ao`ppqr   n}4nERZ "W::b"%}}$$ O J "   ! ! # #F4LF4L!HTN]]**64r*rmrlrrirjrIrnameop_namer#r"r$rrr r!r(r)rr%r&r'r,r?rhr3s` ```` ```` @@r+rr$sH **MeWTU V  G%'97C'/ ZZ(..q1 ? , .  //OP - '..0 1B#NB7IE 1#!7;A ;;aSU& AD++QCEHK[[sqHN;;RLPL ;;aSQg >D ;;aSQg >D  " "1 %E \\rwNrrvrPrN)rkzCallable[[Tensor], Tensor]rr r>intr*rxrmfloatrlryrz Tensor | NonerizLiteral['strong_wolfe']rjrxrIrxrzLiteral['float32', 'float64']rsz str | Nonereturnz(tuple[bool, int, Tensor, Tensor, Tensor]) __future__rtypingrrnumpyr^r8 line_searchrutilsrr r collections.abcr r rrr.r+rs#) % (  "6:.