{-jZddlZddlmZddlmZmZddlmZmZddlm Z d dZ d dZ dS) N)_C_ops) check_typecheck_variable_and_dtype)Variablein_dynamic_or_pir_mode) LayerHelpercltrtj||\}}||fStd it }|d}t |dttfd|D]}t|dgdd||d}||d}| d||||fS) ay Check if input X contains all finite data, if yes, scale it by input Scale. $$Out = X / scale$$ If any tensor in X contains Inf or Nan, the Out will generate a indicator. FoundInfinite will be 1 (True), and Out will not be scaled. In this case, the data of Out should not be used, and its data may not be deterministic. Otherwise, FoundInfinite will be 0 (False). Args: x(list|tuple): The input tensors of check_finite_and_unscale operator. scale: The scale of check_finite_and_unscale operator. float_status(Tensor): (Only used on NPU) The float status to check overflow. check_finite_and_unscalebool)dtypexfloat16float32float64uint16)XScale)Out FoundInfinite)typeinputsoutputs)r ) rrcheck_finite_and_unscale_rlocals"create_variable_for_type_inferencertuplelistr append_op) r scalename float_status found_infhelpererrs X/var/www/html/banglarbhumi/venv/lib/python3.11/site-packages/paddle/static/amp/amp_nn.pyr r s"75AA 9)|  @ @vxx @ @F999GGIq#t}&@AAA     7 7 7 &     u % %F)44G  ' i<Fc vtrtj|||||||||| |St|dddgdt |dt t fd|D]o} t| dgdd| jtj tj fvr |jtj ks JdU|j| jks Jdptdit} |||||d } ||||d }||||d }t| tr| | d <n| |d <| d| |||S)a Update loss scaling according to overall gradients. If all gradients is finite after incr_every_n_steps, loss scaling will increase by incr_ratio. Otherwise, loss scaling will decrease by decr_ratio after decr_every_n_nan_or_inf steps and each step some gradients are infinite. Args: x(list|tuple): The input tensors of update_loss_scaling operator. found_inf (Variable): A boolean variable indicates whether there is any infinite gradient. prev_loss_scaling (Variable): Previous loss scaling. num_good_steps (Variable): A variable accumulates good steps in which all gradients are finite. num_bad_steps (Variable): A variable accumulates bad steps in which some gradients are infinite. incr_every_n_steps (int): A variable represents increasing loss scaling every n consecutive steps with finite gradients. decr_every_n_nan_or_inf (int): A variable represents decreasing loss scaling every n accumulated steps with nan or inf gradients. incr_ratio(float): The multiplier to use when increasing the loss scaling. decr_ratio(float): The less-than-one-multiplier to use when decreasing loss scaling. prev_loss_scalingrrupdate_loss_scalingr rz\The dtype of prev_loss_scaling should be float32 when the dtype of x is float16 or bfloat16.zAThe dtype of prev_loss_scaling should be equal to the dtype of x.)rrPrevLossScaling InGoodSteps InBadSteps)r LossScaling OutGoodSteps OutBadSteps)incr_every_n_stepsdecr_every_n_nan_or_inf incr_ratio decr_ratio StopUpdate stop_update)rrrattrs)r*)rrupdate_loss_scaling_rrrrr paddlerbfloat16rrr isinstancerr)r r#r)num_good_steps num_bad_stepsr1r2r3r4r6r!r%r$rrr7s r&r*r*BsN #      #     I  q#t}&;<<<   7 7 7 !    7v~v7 7 7$*fn<<<n=<<<%*ag555S6555 ; ;&(( ; ;F",%# F(&$ G1#:    E+x((+*|*m  "67% Hr')NN)FN) r9rpaddle.base.data_feederrrpaddle.base.frameworkrrpaddle.base.layer_helperrr r*r'r&rBs HHHHHHHH100000&&&&f n n n n n n r'