ёi}sSSKrSSKrSSKJr SSKJr SSKJr SSKJ r J r SSK J r J r JrJr SS KJrJr S rS rS r"S S5r"SS5r"SS5rg)N)IrGraph)core) quant_layers)QuantWeightPassReplaceFakeQuantDequantPass)_get_input_name_index_get_op_input_var_names_get_output_name_index$move_persistable_var_to_global_block) fuse_utilsutils.pdmodelz .pdiparamsc\SSKJn URRRR US'URRRR US'URURR 5 URURR 5 X4$)Nr)fleetColumnParallelLinearRowParallelLinear)paddle.distributedr meta_parallelparallel_layers mp_layersrrappend)layer_name_mapfake_quant_input_layersrs b/var/www/html/banglarbhumi/venv/lib/python3.13/site-packages/paddle/quantization/imperative/qat.pylazy_import_fleetr&s( ++55JJ)* ++55GG&'""5#6#6#H#HI""5#6#6#K#KL  22c \^\rSrSrSr/SQSSSSSSS S S S S4 U4S jjrS rSS jrS rU=r $)ImperativeQuantAware4zB Applying quantization aware training (QAT) to the dygraph model. )Conv2DLinearConv2DTransposerrabs_maxmoving_average_abs_max?FNc >[TU]5 XplUUUUUUUU U U S. n [S0U D6Ul[ XeU 5Ulg)a The constructor for ImperativeQuantAware. Args: quantizable_layer_type(list[str | layer]): List the type of layers that will be quantized. Default is ['Conv2D', 'Linear']. weight_quantize_type(str): quantization type for weights, which supports 'abs_max' and 'channel_wise_abs_max'. activation_quantize_type(str): quantization type for activations, which supports 'abs_max' and 'moving_average_abs_max' now. If using 'abs_max' mode, the quantization scale will be calculated dynamically each step in both training and testing period. If using 'moving_average_abs_max', the static quantization scale will be calculated during training and used in inference. weight_bits(int): quantization bit number for weights, whereas the bias is not quantized. activation_bits(int): quantization bit number for activations. moving_rate(float): the parameter for 'moving_average_abs_max' quantization. fuse_conv_bn(bool): Whether to fuse conv and bn, default is False. weight_preprocess_layer(paddle.nn.Layer, optional): A paddle Layer that defines how to preprocess weight before quantization. Using this can quickly test if user's preprocess method works or not. The input is non-quantized weight and function returns processed weight to be quantized. If None, the weight will be quantized directly. Default is None. act_preprocess_layer(paddle.nn.Layer, optional): A paddle Layer that defines how to preprocess activation before quantization. Using this can quickly test if user's preprocess method works or not. The input is non-quantized activation and function returns processed activation to be quantized. If None, the activation will be quantized directly. Default is None. weight_quantize_layer(paddle.nn.Layer, optional): A paddle Layer that defines how to quantize weight. Using this can quickly test if user's quantization method works or not. In this layer, user should both define quantization method and dequantization method, that is, the function's input is non-quantized weight and returns dequantized weight. If None, will use quantization op defined by 'weight_quantize_type'. Default is None. act_quantize_layer(paddle.nn.Layer, optional): A paddle Layer that defines how to quantize activation. Using this can quickly test if user's quantization method works or not. In this layer, user should both define quantization method and dequantization method, that is, the function's input is non-quantized activation and returns dequantized activation. If None, will use quantization op defined by 'activation_quantize_type'. Default is None. onnx_format (bool, optional): Whether to export the quantized model with format of ONNX. Default is False. Note: If user sets attribute 'skip_quant' to a Layer that support dynamic quantization and sets it to true, the layer would not be quantized during training. If this attribute is not sets or the attribute is false, the Layer would be quantized in training. Examples: .. code-block:: python >>> import paddle >>> from paddle.static.quantization import ( ... ImperativeQuantAware, ... ) >>> from paddle.vision.models import ( ... resnet, ... ) >>> model = resnet.resnet50(pretrained=True) >>> imperative_qat = ImperativeQuantAware( ... weight_quantize_type='abs_max', ... activation_quantize_type='moving_average_abs_max') >>> # Add the fake quant logical. >>> # The original model will be rewrite. >>> # The outscale of outputs in supported layers would be calculated. >>> imperative_qat.quantize(model) >>> # Fine-tune the quantized model >>> # ... >>> # Save quant model for the inference. >>> imperative_qat.save_quantized_model( ... layer=model, ... model_path="./resnet50_qat", ... input_spec=[ ... paddle.static.InputSpec( ... shape=[None, 3, 224, 224], dtype='float32')]) .. code-block:: python >>> import paddle >>> from paddle.static.quantization import ( ... ImperativeQuantAware, ... ) >>> class ImperativeModel(paddle.nn.Layer): ... def __init__(self): ... super().__init__() ... # self.linear_0 would skip the quantization. ... self.linear_0 = paddle.nn.Linear(784, 400) ... self.linear_0.skip_quant = True ... # self.linear_1 would not skip the quantization. ... self.linear_1 = paddle.nn.Linear(400, 10) ... self.linear_1.skip_quant = False ... def forward(self, inputs): ... x = self.linear_0(inputs) ... x = self.linear_1(inputs) ... return x >>> model = ImperativeModel() >>> imperative_qat = ImperativeQuantAware( ... weight_quantize_type='abs_max', ... activation_quantize_type='moving_average_abs_max') >>> # Add the fake quant logical. >>> # The original model will be rewrite. >>> # >>> # There is only one Layer(self.linear1) would be added the >>> # fake quant logical. >>> imperative_qat.quantize(model) >>> # Fine-tune the quantized model >>> # ... >>> # Save quant model for the inference. >>> imperative_qat.save_quantized_model( ... layer=model, ... model_path="./imperative_model_qat") ) quantizable_layer_typeweight_quantize_typeactivation_quantize_type weight_bitsactivation_bits moving_rateweight_preprocess_layeract_preprocess_layerweight_quantize_layeract_quantize_layerN)super__init__ fuse_conv_bnImperativeQuantizeInputs_quantize_inputsImperativeQuantizeOutputs_quantize_outputs)selfr*r+r,r-r.r/r7r0r1r2r3 onnx_formatkwargs __class__s rr6ImperativeQuantAware.__init__9sbz ('=$8(@&.&'>$8%:"4  !9 B6 B!: +" rc [U[RR5(dS5eUR(a[ R"U5 UR RU5 URRU5 U$)al According to weights' and activations' quantization types, the model will be added some fake quant ops, such as fake_quantize_dequantize_moving_average_abs_max, fake_quantize_dequantize_abs_max and so on. At the same time, the out_scale value of outputs would be calculated. Args: model(paddle.nn.Layer): the model to be quantized. Returns: None Examples: .. code-block:: python >>> import paddle >>> from paddle.static.quantization import ( ... ImperativeQuantAware, ... ) >>> class ImperativeModel(paddle.nn.Layer): ... def __init__(self): ... super().__init__() ... # self.linear_0 would skip the quantization. ... self.linear_0 = paddle.nn.Linear(784, 400) ... self.linear_0.skip_quant = True ... # self.linear_1 would not skip the quantization. ... self.linear_1 = paddle.nn.Linear(400, 10) ... self.linear_1.skip_quant = False ... def forward(self, inputs): ... x = self.linear_0(inputs) ... x = self.linear_1(inputs) ... return x >>> model = ImperativeModel() >>> imperative_qat = ImperativeQuantAware( ... weight_quantize_type='abs_max', ... activation_quantize_type='moving_average_abs_max') >>> # Add the fake quant logical. >>> # The original model will be rewrite. >>> # >>> # There is only one Layer(self.linear1) would be added the >>> # fake quant logical. >>> imperative_qat.quantize(model) 2The model must be the instance of paddle.nn.Layer.) isinstancepaddlennLayerr7rr9applyr;)r<models rquantizeImperativeQuantAware.quantizespb%11 @ 1     # #E * ##E* $$U+ rc [RR5 URR"XU40UD6 SSS5 g!,(df  g=fN)rD pir_utils OldIrGuardr;save_quantized_model)r<layerpath input_specconfigs rrO)ImperativeQuantAware.save_quantized_model(sA    ( ( *  " " 7 7Z +1 + * *s A A)r9r;r7rL) __name__ __module__ __qualname____firstlineno____doc__r6rIrO__static_attributes__ __classcell__r?s@rr r 4sG  '!9 $!"'q f:xrr c T^\rSrSrSr/SQSSSSSSSSS4 U4S jjrS rS rS rU=r $) r8i/zt Based on the input params, add the quant_dequant computational logic both for activation inputs and weight inputs. )r"r#r$r%r&r'r(Nc >^[TT]5 [[R[R 5uTlTl[ U4SjU55TlTRH2n [U [5(dU TR ;aM*U S35e 1Skn SS1n US:waX,;d SUS35eX=;d SUS 35eS nU"U5(dS 5eU"U5(dS 5eS nU"U5(dS5eU"U5(dS5eU"U 5(dS5eU"U 5(dS5eUUUUUUUU U S. Tl g)zb The constructor for ImperativeQuantizeInputs. Please refer to the args of ImperativeQuantAware. c3f># UH&nUTR;aTRUOUv M( g7frL)r).0rPr<s r 4ImperativeQuantizeInputs.__init__..Ls?- 0D///##E*0s.1 is unsupported to be quantized.>r% lsq_weightchannel_wise_abs_maxr&channel_wise_lsq_weightr&lsq_actz"Unsupported weight_quantize_type: z1. It can only be abs_max or channel_wise_abs_max.z&Unsupported activation_quantize_type: z7. It can only be moving_average_abs_max or lsq_act now.cR[U[5=(a US:=(a US:*$)Nr)rCint)bitss r3ImperativeQuantizeInputs.__init__..ps D#.K419KKrz%weight_bits should be 1, 2,... or 16.z)activation_bits should be 1, 2,... or 16.c`USL=(d$ [U[RR5$rL) issubclassrDrErF)methods rrlrmws&Vt^& z FIIOO8 & rz%weight_preprocess should be nn.Layer.z"act_preprocess should be nn.Layer.z#weight_quantize should be nn.Layer.z act_quantize should be nn.Layer.) r+r,r-r.r/weight_pre_layer act_pre_layerweight_quant_layeract_quant_layerN) r5r6rrrrtuple_quantizable_layer_typerCstr_kwargs)r<r*r+r,r-r.r/r0r1r2r3rP quantize_typeact_quantize_type bits_check layer_checkr?s` rr6!ImperativeQuantizeInputs.__init__5s$  N Nrr8cX^\rSrSrSrS U4SjjrSrS SjrSrSr Sr S r S r U=r $) r:iz0 Calculate the output scales for target layers. cF>[TU]5 XlX lX0lg)a The constructor for ImperativeQuantizeOutputs. Args: moving_rate(float): The decay coefficient of moving average. The default value is 0.9. activation_bits(int, optional): quantization bit number for activation. Default is 8. N)r5r6 _moving_rate_activation_bits _onnx_format)r<r/r.r=r?s rr6"ImperativeQuantizeOutputs.__init__s" ' /'rc[U[RR5(dS5eUR 5Hup#SU;aM UR U5(dM%[ R"X5upESn[U[[ R55(a [R"X0RUS9nO[R"X0RUS9n[XEU5 M g)a Insert the `moving_average_abs_max_scale` layers to calculate the output scales for specific layers in the dygraph model. Args: model(paddle.nn.Layer): The target model which would be calculate the output quantization scale. Returns: None rB_act_preprocessN) reduce_type)rCrDrErFr_is_target_layerrrrufake_quant_output_layersrFakeQuantMAOutputScaleLayerrMAOutputScaleLayerr)r<rHcur_namerrrrrs rrGImperativeQuantizeOutputs.applys%11 @ 1$)#8#8#: H H,((33%*%I%I& "LK)U5+I+I%JKK"."J"J00k##/"A"A00k# LO <+$;rc [U[RR5(dS5eU(a[RR XS9 [RR "SXUS.UD6 Sn[R"5(aSn[R"5 [R"5n[RR5n[RRU5n[RR!U5n [RR#U5n U [$-n U [&-n [RR)U UU U S9un nnUR*(dUR-XU5 [/[R0"U R25SS9nUR55HQnUR75H*nUR95S:XdMUR;U5 M, UR=5 MS UR?5n URAU 5 SnO[/[R0"U R25SS9n[CXvURDS 9nUR55HnSUl#URIU5 M [KXv5nUR55HnSUl#URIU5 M UR?5n Sn[MU 5 S nU cS nO.U ROS 5(aU RQS S5SnOU n[RRSU U5nUVs/sH"nU RU5RWU5PM$ nn[RRYUUUUU R[5US9 U(a[R\"5 g g s snf)a Save the quantized model for the inference. Args: model (Layer): The model to be saved. path (str): The path prefix to save model. The format is ``dirname/file_prefix`` or ``file_prefix``. input_spec (list[InputSpec|Tensor], optional): Describes the input of the saved model's forward method, which can be described by InputSpec or example Tensor. If None, all input variables of the original Layer's forward method would be the inputs of the saved model. Default None. **config (dict, optional): Other save configuration options for compatibility. We do not recommend using these configurations, they may be removed in the future. If not necessary, DO NOT use them. Default None. The following options are currently supported: (1) output_spec (list[Tensor]): Selects the output targets of the saved model. By default, all return variables of original Layer's forward method are kept as the output of the saved model. If the provided ``output_spec`` list is not all output variables, the saved model will be pruned according to the given ``output_spec`` list. Returns: None rB)rR)rPrQrRFT)executormodel_filenameparams_filename)for_testmoving_average_abs_max_scale) quant_bitsNrHr.r r)rprogram clip_extrar4)/rCrDrErFjit to_staticsavein_dynamic_mode enable_staticrCPUPlacestatic global_scopeExecutorosrQdirnamebasenameINFER_MODEL_SUFFIXINFER_PARAMS_SUFFIXload_inference_modelr_gather_scalesrGraphdescall_sub_graphs all_op_nodesrsafe_remove_nodesresolve_hazard to_program_set_skip_quant_attrrr _for_testrGrr endswithrsplitjoin global_blockvarsave_inference_modelclonedisable_static)r<rHrQrRrSis_dynamic_modeplacescopeexerrrr infer_programfeed_target_names fetch_targetsgraph sub_graph_oprtransform_passquant_weight_pass model_name path_prefixr feed_varss rrO.ImperativeQuantizeOutputs.save_quantized_modelsM8%11 @ 1  JJ  > Pe:PP  ! ! # #"O  "  **,mm$$U+''//$'77##D)!$66"%88 MM . . )+ /         m DDJJ}'9'9:UKE"113 $113Cxxz%CC!33C84((* 4 ",,.M  % %m 4JDJJ}'9'9:UKE8)>)>N#113 &* #$$Y/4!0 = "113 &* #!'' 24",,.MJ,];  ! J  $ $Z 0 0'..sA6q9J'Jggll7J7 ?P ?PtM & & ( , ,T 2?P    **   !'')! +    ! ! #  s!)Oc [U[RR5(dgUR(a,[U[ [ R55(aS$S$Sn[ R"U5(a*[U[ [ R55(dSn[U[ [ R55(aSn[U[RRR5(aSnU$)z5 Whether the layer needs to calculate output scales. FT) rCrDrErFrrurfake_quant_wrap_layers is_leaf_layerfake_quant_leaf_layersquantFloatFunctionalLayer)r<rPflags rr*ImperativeQuantizeOutputs._is_target_layerqs %11   eU5+G+G%HII      u % %j 5556/ / D eU5#?#?@ A AD eVYY__AA B BD rcD^^^UU4SjnUUU4SjnU"5 U"5 g)zu Get all scales from fake ops, save them into the corresponding ops and delete all moving_average_abs_max_scale ops. c>/n/[RQSPnT RH9nURH&nURU;dMUR U5 M( M; UHn[ U5Hn[R"URU5nUcM)SUR;dURS:XdMKURS5Sn[R"T U5n[R"U5n[X45upURU[U 5-S-U5 URSS5 M M g)Nrquantize_dequantizeOutScaler _thresholdwith_quant_attrT)r!fake_quantize_dequantize_op_typesblocksopstyperr find_previous_opblockoutputload_variable_datafp_numpy_to_naiver _set_attrrw) target_opsskip_opsrop in_var_name previous_op scale_namein_scaleargnameindexrrs r_gather_input_scaleEImperativeQuantizeOutputs._gather_scales.._gather_input_scales$J88.H!))Bwwh."))"-$( !#:2#>K"'"8"8;"OK".-1A1AA&++/MM%0%7%7 %CA%F #(#;#;E:#N#(#:#:8#D)>r)O #c%j0<? %6=$?!rc~>/nTRH9nURH&nURS:XdMURU5 M( M; UGHjnUR S5SnUR S5SnUR n[R"X5n[R"X5nUR S5Sn[R"TU5n[R"U5nURS:waW[XS5n U bIU upURU [U 5-S-U5 URSU5 URS S 5 UHXn U RXC5 [![#T55H,n TU R$U:XdMUR'U5TU 'M. MZ GMm g) NrXrOutrfeedr out_thresholdrT)rrrrinputrrrr find_next_opsrrr rrw _rename_inputrangelenrr)rrrr out_var_namernext_opsout_scale_name out_scaleresrrnext_opirrrs r_gather_output_scaleFImperativeQuantizeOutputs._gather_scales.._gather_output_scalesJ ))Bww"@@"))"-$( ! hhsmA. !yy/2 #44UH  ..uC!#:!6q!9!44UNK !33I> ##v-0JC),#--#c%j0<?$--oyI#--.?F'G)),D#3}#56(+00L@/4yy/EM!,7 (+!rNr4)r<rrrrrs ``` rr(ImperativeQuantizeOutputs._gather_scaless  >8" FH rcURHRnURH?nURX#5(dMURSS5 URSS5 MA MT g)z Label the skip quantized ops. rTrN)rr_is_skip_quant_opr)r<rrrs rr.ImperativeQuantizeOutputs._set_skip_quant_attrsN^^Eii))%44LLt4LL!2D9 $rc/SQnURU;agURVs/sHn[R"X5PM nn[ SU55$s snf)z The input op should be skipped quantization. 1. the type of input op should be conv2d, depthwise_conv2d or matmul 2. the previous ops of the input op are not fake_quantize_dequantize ops )conv2ddepthwise_conv2dmatmulconv2d_transposeFc3r# UH-nUSL=(a UR[R;v M/ g7frL)rrr)r`rs rra>ImperativeQuantizeOutputs._is_skip_quant_op..s; # dN GuFFF G"s57)rinput_arg_namesrrany)r<rin_optarget_op_typesarg_name previous_opss rr+ImperativeQuantizeOutputs._is_skip_quant_opsp   ::_ ,"11 1  " "5 31   #    s A)rrr)r(r'FrL)rUrVrWrXrYr6rGrOrrrrrZr[r\s@rr:r:s5 (%=Nu$n:GR:  rr:)rrDpaddle.base.frameworkrpaddle.frameworkrpaddle.nn.quantr%static.quantization.quantization_passrrstatic.quantization.utilsr r r r rrrrrr r8r:r4rrr,sc )!(  " 3xxvNNNNbx x r