͑iT&SSKJr SSKrSSKJrJr SSKJr SSKJr SS/r "SS 5r "S S\ 5r "S S\ 5r g) ) annotationsN)_C_opspir) framework)in_dynamic_or_pir_modeL1DecayL2Decayc>\rSrSrSrSSjrS SjrSrSrg) WeightDecayRegularizeravBase class for weight decay regularizers Defines the common interface of weight-decay regularizers. Weight-decay regularizers are added only during the backward pass for faster regularization. They add operations to the network that correspond to gradient of the regularization function. Users should not use this class directly, but need to use one of its implementations cgNselfs R/var/www/html/banglarbhumi/venv/lib/python3.13/site-packages/paddle/regularizer.py__init__WeightDecayRegularizer.__init__%s c[e)z8Add corresponding weight decay operations to the networkNotImplementedError)rparamgradblocks r__call__WeightDecayRegularizer.__call__(s "!rc[e)z Debug stringrrs r__str__WeightDecayRegularizer.__str__.s!!rrN)returnNoner paddle.Tensorrr$rz pir.Block) __name__ __module__ __qualname____firstlineno____doc__rrr__static_attributes__rrrr r s. """*7"@I" "rr cT^\rSrSrSrSSU4SjjjrS SjrS SjrSrU=r $) r3a Implement the L1 Weight Decay Regularization, which encourages the weights to be sparse. It can be set in :ref:`api_paddle_ParamAttr` or ``optimizer`` (such as :ref:`api_paddle_optimizer_Momentum` ). When set in ``ParamAttr`` , it only takes effect for trainable parameters in this layer. When set in ``optimizer`` , it takes effect for all trainable parameters. When set together, ``ParamAttr`` has higher priority than ``optimizer`` , which means that for a trainable parameter, if regularizer is defined in its ParamAttr, then the regularizer in Optimizer will be ignored. Otherwise the regularizer in Optimizer will be used. In the implementation, the loss function of L1 Weight Decay Regularization is as follows: .. math:: loss = coeff * reduce\_sum(abs(x)) Args: coeff(float, optional): regularization coeff. Default:0.0. Examples: .. code-block:: python :name: code-example1 >>> # Example1: set Regularizer in optimizer >>> import paddle >>> from paddle.regularizer import L1Decay >>> linear = paddle.nn.Linear(10, 10) >>> inp = paddle.rand(shape=[10, 10], dtype="float32") >>> out = linear(inp) >>> loss = paddle.mean(out) >>> beta1 = paddle.to_tensor([0.9], dtype="float32") >>> beta2 = paddle.to_tensor([0.99], dtype="float32") >>> momentum = paddle.optimizer.Momentum( ... learning_rate=0.1, ... parameters=linear.parameters(), ... weight_decay=L1Decay(0.0001)) >>> back = out.backward() >>> momentum.step() >>> momentum.clear_grad() .. code-block:: python :name: code-example2 >>> # Example2: set Regularizer in parameters >>> # Set L1 regularization in parameters. >>> # Global regularizer does not take effect on my_conv2d for this case. >>> from paddle.nn import Conv2D >>> from paddle import ParamAttr >>> from paddle.regularizer import L1Decay >>> my_conv2d = Conv2D( ... in_channels=10, ... out_channels=10, ... kernel_size=1, ... stride=1, ... padding=0, ... weight_attr=ParamAttr(regularizer=L1Decay(coeff=0.01)), ... bias_attr=False) c8>Uce[TU]5 Xlgrsuperr_coeffrcoeff __class__s rrL1Decay.__init__q     rc[U[R[R[R R 45(de[U[R[R45(de[5(a8[R"U5n[R"X@RSS5$URURURUR S9nURURURUR S9nUR#SSU0SU0S9 UR#SSU0SU0SUR0S 9 U$) aAdd L1 weight decay ops to network Adds L1 weight decay ops. L1WeightDecay = reg_coeff * sign(parameter) Args: param: parameter variable for which regularization is applied block: block in which variable is to be created Returns: new variable for weight decay Tdtypeshape lod_levelsignXOut)typeinputsoutputsscaler?r@rAattrs) isinstancerVariablerValuecore ParameterMetaBlockrrr<rBr0 create_varr9r:r; append_op)rrrrr<decays rrL1Decay.__call__vs6$ I&& 3883I3IJ    %)//399!=>>>> ! # #;;u%D<<kk3= =##kk$D$$kk%E OOS%L5$-   OOT{ ,   Lrc"SURS3$)NzL1Decay, coeff=fr0rs rrL1Decay.__str__ Q00rrQr7r2floatr!r"r#r!str r%r&r'r(r)rrrr* __classcell__r3s@rrr3s?;z --- -^11rcT^\rSrSrSrSSU4SjjjrS SjrS SjrSrU=r $) r a Implement the L2 Weight Decay Regularization, which helps to prevent the model over-fitting. It can be set in :ref:`api_paddle_ParamAttr` or ``optimizer`` (such as :ref:`api_paddle_optimizer_Momentum` ). When set in ``ParamAttr`` , it only takes effect for trainable parameters in this layer. When set in ``optimizer`` , it takes effect for all trainable parameters. When set together, ``ParamAttr`` has higher priority than ``optimizer`` , which means that for a trainable parameter, if regularizer is defined in its ParamAttr, then the regularizer in Optimizer will be ignored. Otherwise the regularizer in Optimizer will be used. In the implementation, the loss function of L2 Weight Decay Regularization is as follows: .. math:: loss = 0.5 * coeff * reduce\_sum(square(x)) Args: coeff(float, optional): regularization coeff. Default:0.0 Examples: .. code-block:: python :name: code-example1 >>> # Example1: set Regularizer in optimizer >>> import paddle >>> from paddle.regularizer import L2Decay >>> linear = paddle.nn.Linear(10, 10) >>> inp = paddle.rand(shape=[10, 10], dtype="float32") >>> out = linear(inp) >>> loss = paddle.mean(out) >>> beta1 = paddle.to_tensor([0.9], dtype="float32") >>> beta2 = paddle.to_tensor([0.99], dtype="float32") >>> momentum = paddle.optimizer.Momentum( ... learning_rate=0.1, ... parameters=linear.parameters(), ... weight_decay=L2Decay(0.0001)) >>> back = out.backward() >>> momentum.step() >>> momentum.clear_grad() .. code-block:: python :name: code-example2 >>> # Example2: set Regularizer in parameters >>> # Set L2 regularization in parameters. >>> # Global regularizer does not take effect on my_conv2d for this case. >>> from paddle.nn import Conv2D >>> from paddle import ParamAttr >>> from paddle.regularizer import L2Decay >>> my_conv2d = Conv2D( ... in_channels=10, ... out_channels=10, ... kernel_size=1, ... stride=1, ... padding=0, ... weight_attr=ParamAttr(regularizer=L2Decay(coeff=0.01)), ... bias_attr=False) c8>Uce[TU]5 Xlgrr.r1s rrL2Decay.__init__r5rc[U[R[R[R R 45(de[U[R[R45(de[5(a"[R"XRSS5$URURURURS9nUR!SSU0SU0SUR0S9 U$)a Add L2 weight decay ops to network Adds L2 weight decay ops. L2WeightDecay = reg_coeff * parameter Args: param: parameter variable for which regularization is applied block: block in which variable is to be created Returns: new variable for weight decay r7Tr8rBr=r>rC)rErrFrrGrHrIrJrrrBr0rKr9r:r;rL)rrrrrMs rrL2Decay.__call__s$ I&& 3883I3IJ    %)//399!=>>>> ! # #<<{{C> >$$kk%E OOU| ,  Lrc"SURS3$)NzL2Decay, coeff=rPrQrs rrL2Decay.__str__rSrrQrTrUr#rWrYr[s@rr r s?:x &&& &P11r) __future__rpaddlerr paddle.baserpaddle.base.frameworkr__all__r rr rrrrisI # !8 i ""2s1$s1lk1$k1r