ϑi)(SSKrSSjrSSjrg)Nc:UbUupxO X::aX4OX04upxX%-SX- -X- - - n U S-X%-- n U S:a_U R5n X::aX3U- X[-U - XR- SU --- -- n OXU- X+-U - X%- SU --- -- n [[X5U5$Xx-S- $)a-Cubic interpolation between (x1, f1, g1) and (x2, f2, g2). Use two points and their gradient to determine a cubic function and get the minimum point between them in the cubic curve. Reference: Jorge Nocedal, Stephen J. Wright, Numerical Optimization, Second Edition, 2006. pp59: formula 3.59 Args: x1, f1, g1: point1's position, value and gradient. x2, f2, g2: point2's position, value and gradient. bounds: bounds of interpolation area Returns: min_pos: the minimum point between the specified points in the cubic curve. rg@)sqrtminmax) x1f1g1x2f2g2bounds xmin_bound xmax_boundd1 d2_squared2min_poss m/var/www/html/banglarbhumi/venv/lib/python3.13/site-packages/paddle/incubate/optimizer/line_search_dygraph.py_cubic_interpolaters$!' J-/X"B8 1=BG, ,BAIA~ ^^  8G2"'AF:J(KLLGG2"'AF:J(KLLG3w+Z88'3..c UR5R5n UR5nU"XU5upSn[R"X5n[R "SUR S9UUU4unnnnSnSnUU :aXXr-U--:d US:a%U U:aUU/nUU /nUU R5/nUU/nO[R"U5U*U-::a U/nU /nU /nSnOUS:aUU/nUU /nUU R5/nUU/nOgUSUU- --nUS-nUn[UUUUU UUU4S9nUnU nU R5nUnU"XU5upUS- nU R U5nUS- nUU :aMUU :Xa SU/nXL/nX]/nSnWSUS ::aS OS unnU(GdUU :Ga[R"WSUS- 5U -U :aGO[USUSWSUSUSUS5nS [U5[U5- -n [[U5U- U[U5- 5U :aU(dU[U5:dU[U5::af[R"U[U5- 5[R"U[U5- 5:a[U5U - nO[U5U -nSnOSnOSnU"XU5upUS- nU R U5nUS- nXXr-U--:d U UU:a6UUU'U UU'U R5WU'UUU'USUS::aS OS unnOu[R"U5U*U-::aSnO2UUUUU- -S:a UUUU'UUUU'WUUU'UUUU'UUU'U UU'U R5WU'UUU'U(d UU :aGMWUnUUn WUn XX.4$) aL Implements of line search algorithm that satisfies the strong Wolfe conditions using double zoom. Reference: Jorge Nocedal, Stephen J. Wright, Numerical Optimization, Second Edition, 2006. pp60: Algorithm 3.5 (Line Search Algorithm). Args: obj_func: the objective function to minimize. ```` accepts a multivariate input and returns a scalar. xk (Tensor): the starting point of the iterates. alpha (Scalar): the initial step size. d (Tensor): search direction. loss (scalar): the initial loss grad (Tensor): the initial grad c1 (Scalar): parameter for sufficient decrease condition. c2 (Scalar): parameter for curvature condition. tolerance_change (Scalar): terminates if the change of function value/position/parameter between two iterations is smaller than this value. max_ls(int): max iteration of line search. alpha_max (float): max step length. Returns: loss_new (Scaler): loss of obj_func at final alpha. grad_new, (Tensor): derivative of obj_func at final alpha. alpha(Tensor): optimal step length, or 0. if the line search algorithm did not converge. ls_func_evals (Scaler): number of objective function called in line search process. Following summarizes the essentials of the strong Wolfe line search algorithm. Some notations used in the description: - `func` denotes the objective function. - `obi_func` is a function of step size alpha, restricting `obj_func` on a line. obi_func = func(xk + alpha * d), where xk is the position of k'th iterate, d is the line search direction(decent direction), and a is the step size. - alpha : substitute of alpha - a1 is alpha of last iteration, which is alpha_(i-1). - a2 is alpha of current iteration, which is alpha_i. - a_lo is alpha in left position when calls zoom, which is alpha_low. - a_hi is alpha in right position when calls zoom, which is alpha_high. Line Search Algorithm: repeat Compute obi_func(a2) and derphi(a2). 1. If obi_func(a2) > obi_func(0) + c_1 * a2 * obi_func'(0) or [obi_func(a2) >= obi_func(a1) and i > 1], alpha= zoom(a1, a2) and stop; 2. If |obi_func'(a2)| <= -c_2 * obi_func'(0), alpha= a2 and stop; 3. If obi_func'(a2) >= 0, alpha= zoom(a2, a1) and stop; a1 = a2 a2 = min(2 * a2, a2) i = i + 1 end(repeat) zoom(a_lo, a_hi) Algorithm: repeat aj = cubic_interpolation(a_lo, a_hi) Compute obi_func(aj) and derphi(aj). 1. If obi_func(aj) > obi_func(0) + c_1 * aj * obi_func'(0) or obi_func(aj) >= obi_func(a_lo), then a_hi <- aj; 2. 2.1. If |obi_func'(aj)| <= -c_2 * obi_func'(0), then alpha= a2 and stop; 2.2. If obi_func'(aj) * (a2 - a1) >= 0, then a_hi = a_lo a_lo = aj; end(repeat) r)dtypeFTg{Gz? )r)rr)rrg?) absrclonepaddledot to_tensorrrr)!obj_funcxkalphadlossgradgtdc1c2tolerance_changemax_lsd_normloss_newgrad_new ls_func_evalsgtd_newt_prevf_prevg_prevgtd_prevdonels_iterbracket bracket_f bracket_g bracket_gtdmin_stepmax_steptmpinsuf_progresslow_poshigh_posepss! r _strong_wolferD6slUUW[[]F ::222:"      h' !%b3 ,,q/1 a F f&e*$ $ N"+A,)B-"?VGXw' ::gaj71:- . 7:J J # AJ aL N AJ aL N  "S\CL01 s7|e#US\%9 :S @#g,!6%3w<:O::ec'l23fjjCL(7 L3.EL3.E!&!%"N%b3 ,,q/1  rzC// 09W--!&GH "*Ih "*.."2Ih $+K !#A,)A,6F GXzz'"rcCi/GH-0@@AQF$+G$4!&/&8 (#&/&8 (#(3G(< H% %GG !)Ig !)!1Ig #*K Sw'X G E!H!H u 33r)N)g-C6?g?g& .>)r rrDrrrGs$!/X   s4r