ёi VSSKrSSKrSSKrSSKrSSKrSSKJrJrJ r SSK J r SSK J r SSKJrJrJrJr SSKJr \ "\\R,S S 9r\R0R2R4\R0R2R6\R0R2R8/rS rS rS r Sr!Sr"Sr#Sr$\ S5r%Sr&SSjr'SSjr(SSjr)SSjr*g)N)core framework global_scope) get_logger)signature_safe_contextmanager) _rename_arg_rename_op_inputfind_true_post_opfind_true_prev_op)AutoMixedPrecisionListsBF16z&%(asctime)s-%(levelname)s: %(message)s)fmt __use_bf16__c[R"U5n[R"S[R/S9"UR5n[R "XR 5$)Ncf[R"S[R"SU55SS- $)Nz)convert_float_to_uint16..3s$&--fkk$&:;A>"D)otypes)npasarray vectorizeuint16flatreshapeshape)in_listouts rconvert_float_to_uint16r&0sMjj!G ,,D { ll C ::c== ))rc.U[R:Xagg)zh Convert specific variable type to its corresponding string. Args: dtype (VarType): Variable type. bf16fp32)paddlebfloat16)dtypes r _dtype_to_strr-9s rc SnURGHwnU[R:XaURS;aUS;aM0UR U5GH1nUR U5nUR[ ;dURU:XaM;URU:XaURS-[U5-n URRU 5n U bU RU:waOURU USURS9n URUSSU0S U 0URU RS .S 9 US - n[XRU R5 GMUR!S 5(dGMUR#S U5 GM4 GMz U[R:XGa*U[R$:XGaUR&GHn URS;aU S:waMUR)U 5Hn UR U 5n U R[ ;aM*U R[R:XdMJU R*R-[.R0R2R45 UR!S5(dMUR#S[.R0R2R45 M GM U$)a Insert cast op and rename args of input and output. Args: block (Program): The block in which the operator is. op (Operator): The operator to insert cast op. idx (int): The index of current operator. src_dtype (VarType): The input variable dtype of cast op. dest_dtype (VarType): The output variable dtype of cast op. Returns: num_cast_op (int): The number of cast ops that have been inserted. r) batch_normfused_bn_add_activation layer_normXZ.cast_Fnamer, persistable stop_gradientcastr3Outin_dtype out_dtypetypeinputsoutputsattrsr r=Yr>) input_namesr*float32r@inputvar _valid_typesr,r7r-varsget create_varr9 _insert_opr has_attr _set_attrr+ output_namesoutputdesc set_dtyperVarDescVarTypeBF16) blockopidx src_dtype dest_dtype num_cast_opsin_name in_var_namein_var cast_nameout_varout_name out_var_names r_insert_cast_oprdFs9L>>  &2777 , j(88G,KYY{+F{{,.&,,*2L||y("KK(2]:5NN **..3?gmmz&A#..&($)&,&:&: /G$$# #V}!& 0(. )0 % !A%LB W\\:;;z**LLZ8;-"LFNN"zV__'DHJKO " ( 3 ))L1<<|3==FNN2LL**4<<+?+?+D+DE{{;// [$,,2F2F2K2KL!4( rc fSnURU5nUR[;dURU:XaU$URU:Xd*S[ UR5S[ U5S35eUR S-[ U5-n UR RU 5n U bU RU:watURU USURS9n URUSS U0S U 0URU RS .S 9 US - nU R X`RUR 'U$)NrzThe real dtype(z ) is not equal to the src dtype()r5Fr6r:r3r;r<r?r ) rHr@rIr,r-r7rJrKrLr9rMrY) rWrXrYrZr[ target_nameop_var_rename_mapr\ target_varr`cast_vars r_insert_cast_post_oprksFL;'Jl*j.>.>*.L   y ( - (8(89::Z[hir[sZttuv ((*]:-FFIzz~~i(H8>>Z7##$22 $   $H%)//hnnM    8@ ))$Z__5 rcUR(dgURHnX!R;dM g URHnX1R;dM g gNFT) fp32_varnamesinput_arg_namesoutput_arg_names)rX amp_listsr]rbs r_is_in_fp32_varnamesrrsS  " "%% -- -&'' .. .( rcLURU;ag/nUR[UR55 UR[UR55 UH nSU;dM g U(a1UR S5(a[ URS5;aggg)NT learning_rate op_namescopeF)r@extendlistrorprN_bf16_guard_patternattr)rXunsupported_op_listuse_bf16_guardin_out_arg_namesr7s r_need_keep_fp32r}s ww%%D!3!345D!4!456  d "! ;;~ & & 277>#: :rc#x# [R"[S9 Sv SSS5 g!,(df  g=f7f)a2 As for the pure bf16 training, if users set `use_bf16_guard` to True, only those ops created in the context manager `bf16_guard` will be transformed as float16 type. Examples: .. code-block:: python >>> import numpy as np >>> import paddle >>> import paddle.nn.functional as F >>> paddle.enable_static() >>> data = paddle.static.data(name='X', shape=[None, 1, 28, 28], dtype='float32') >>> conv2d = paddle.static.nn.conv2d(input=data, num_filters=6, filter_size=3) >>> with paddle.static.amp.bf16.bf16_guard(): ... bn = paddle.static.nn.batch_norm(input=conv2d, act="relu") ... pool = F.max_pool2d(bn, kernel_size=2, stride=2) ... hidden = paddle.static.nn.fc(pool, size=10) ... loss = paddle.mean(hidden) )prefixN)r name_scoperxrr bf16_guardrs$.   %8 9  : 9 9s:) : 7:c8UHnUH nX1;dM g M grmr)post_ops keep_fp32_opspost_oprXs rare_post_ops_bf16rs$B" rc "UR5RnUGHn[UR5UR;dM)Sn/n /n UR Hwn UR U 5H_n URU 5n [X7U S5nU bU R[;aSn MQU RU5 U RU 5 Ma My U(dM[X5(dMU Hn U R[R:Xa=U RR!["R$R&R(5 UcMcU R*U;dMuUR-U R*5 M UR/S5(dGMUR1S5["R$R&R2:XdGMUR5S["R$R&R(5 GM g)NTFr,) global_blockopsstrr@bf16_initializer_listrPrQrHr rIappendrr,r*rFrRrSrrTrUrVr7removerNryFP32rO) startup_progrqrWall_opsrto_bf16_var_names prepend_opsrX change_op op_post_ops op_out_varsrbrcrars rcast_initializers_to_bf16rs{++-11K rww<9:: :IKKOO$&IIh$7L#ii 5G/\4PG',,l*J$) &&w/&&w/%8,y.{JJ*G}}6 ..t||/C/C/H/HI)5#LL,==)00> +KK(((DLL,@,@,E,EELL$,,*>*>*C*CD;rc  Uc [5nUR5n[5n[5n[5n/nURHn UR U R 5 M URGH n U R n U GHn U R S:XdU R S:XaM&[XRU5(aURU 5 MTU RGHn U R S;aU S;aMU RU 5Hn SnU RU 5nUbUR [ ;aM0UR"[$R&:XaNUR(R+[,R.R0R25 URU 5 [RS U R S U S UR"S 35 M GM U R4HnU R S;aUS:waMU R7U5HnSnU RU5nUbUR [ ;aM0UR"[$R&:Xa=UR(R+[,R.R0R25 [RS U R SUSUR"S 35 M M SHtnU R9U5(dMU R;U5[$R&:XdM@U R=U[,R.R0R25 Mv GM UcGM[?UUUU UU5 GM [A[CUR55Vs/sHn[DRF"5PM nnURGH#n U R n SnU[CU 5:dM#U Un SnX;a_X;aX[IU U U[,R.R0RJ[,R.R0R25nUU- nGO{[IU U U[,R.R0R2[,R.R0RJ5nUU- nU RLGHnU RNRQU5nUbUR [ ;aM8UR"[$RR:XdMXUR(R+[,R.R0RJ5 [UXU5nUHknUU;aM [WU U UU-S-[,R.R0RJ[,R.R0R2UU5nUU- nMm GM UUS-- nU[CU 5:aGMGM& [YUUX5 U$![aQn[RSUS35 URU 5nUb[RSU S 35 SnAGN3SnAff=f![aQn[RSUS35 URU5nUb[RSUS 35 SnAGNSnAff=fs snf)a Traverse all ops in the whole model and set their inputs and outputs to the bf16 data type. This function will do some special processing for the batch normalization, which will keep the batchnorm's computations in FP32. Args: program (Program): The used program. amp_lists (AutoMixedPrecisionListsBF16): An AutoMixedPrecisionListsBF16 object. use_bf16_guard(bool): Determine whether to use `bf16_guard` when constructing the program. Default True. Ncreate_py_readerread>r/r1r0r2z-- z&, try to get it in the global block --z-- var z is got in the global block --z -- op type: z, in var name: z, in var dtype: z --rDz, out var name: z, out var dtype: )r=r>r,rr )-rrsetblocksrvrr@r}unsupported_listaddrErGrH ValueError_loggerdebugrIr,r*rFrRrSrrTrUrVrPrQrNryrOrrangelen collections OrderedDictrdrrprJrKr+r rkr )programrrqr{rrrto_bf16_pre_cast_ops origin_opsrWrrXr]r^r_erbrcra attr_name_rhrYr\in_var_cast_num pre_cast_numrr post_cast_nums rcast_model_to_bf16r's/1 '')LEM5J%))$ iiBww,,60Ar#=#=~NN!!"%>>77Z/#%88G#4K!F !&;!7~L)H ||v~~5 --dll.B.B.G.GH)--k:MM&rwwi{mK[\b\h\h[iilm+$5*@OOGGNO C$&IIh$7L"G "'))L"9',,l*J }}6 ..t||/C/C/H/HIMM&rwwi/? ~M^_f_l_l^mmpq)%8,<@ KK ** *fnn<LLDLL,@,@,E,EF @IV  # %!  ] r,1W^^1D+E+Ea !+EiiCHnSBL&-&5 ,,11 ,,11 'O!O3L.LL((--LL((--    , $&$7$7L#jjnn\:G',,l*J }}7 ..t||/C/C/H/HI#4Sl#K'/G&-7 (,@ % " #l 2Q 6 $ 4 4 9 9 $ 4 4 9 9 , 1-M)M9L(0%8( !-#MM")+6T U @& !!$JK#/"2"2<"@".#MM"),7U V Js8X -Y)'[ Y& AY! !Y& ) [ 3AZ? ?[ c/nURH"nURUR55 M$ U(aUO [5nU(aUO [ 5nUHnUR U;dM[ RSUR S35 URUR 5R5n [R"U 5n U R[U 5U5 M g)a Traverse all parameters in the whole model and set them to the BF16 data type. Whereas, this function will keep parameters of batchnorms in FP32. Args: place(base.CPUPlace|base.CUDAPlace): `place` is used to restore the BF16 weight tensors. program (Program): The used program. scope(base.Scope, optional): `scope` is used to get the FP32 weight tensor values. Default is None. to_bf16_var_names(set|list, optional): The data types of vars in `to_bf16_var_names` will be set to BF16. Usually, it is the returned value of `cast_model_to_bf16` API. z ---- cast z to bf16 dtype ----N) rrvall_parametersrrr7rrfind_var get_tensorrarrayr&) placerscoperrrWbf16_var_names var_scopeparamparam_tdatas rcast_parameters_to_bf16rsNe2245 +<&NLNI :: ' MMJuzzl2EF G((4??AG88G$D KK/5u =  rcUc [5nUR5nURn[5n[5nUGHnURS:XdURS:XaM&UR b#[ Xa5(aURU5 MVURUR;aURU5 MURUR;aURU5 MURUR;GaSnSnURHn U (dM URU 5Hn URU 5n U RcM#U RULa[X6U 5n U cMCO U Rn X;dU RUR;aSnMsX;dU RUR;dMSnM M U(aURU5 GMU(aURU5 GMGMURU5 GM Sn U [!U5:GaTX=nSnXe;aT[#UUU [$R&R(R*[$R&R(R,5nOXd;aUR/S5(akUR1S5[$R&R(R,:Xa4UR3S[$R&R(R*5 [#UUU [$R&R(R,[$R&R(R*5nOXS-- n U [!U5:aGMSgg) a Traverse all ops in current block and insert cast op according to which set current op belongs to. 1. When an op belongs to the fp32 list, add it to fp32 set 2. When an op belongs to the bf16 list, add it to bf16 set 3. When an op belongs to the gray list. If one of its inputs is the output of fp32 set op or fp32 list op, add it to fp32 set. If all of its previous ops are not fp32 op and one of its inputs is the output of bf16 set op or bf16 list op, add it to bf16 set. 4. When an op isn't in the lists, add it to fp32 op set. 5. Add necessary cast ops to make sure that fp32 set op will be computed in fp32 mode, while bf16 set op will be computed in bf16 mode. Args: main_prog (Program): The main program for training. NrrFTrr,r )rrrrr@rnrrr fp32_list bf16_list gray_listrErGrHrXr rrdrrTrUrVrrNryrO) main_progrqrWr bf16_op_set fp32_op_setrX is_fp32_op is_bf16_opr]r^r_prev_oprYr\s rrewrite_program_bf16rs(/1  " " $E ))C%K%K 77( (BGGv,=   " " .3G 4 4  OOB   77i)) ) OOB  WW ++ + OOB  WW ++ +JJ>>7')xx'8 !&;!7!99,$#YY"_&7&MG& ( /'-iiG$2&||y/B/BB)-J#2&||y/B/BB)-J+(9*2## OOB or C C. X * $$)) $$)) L  G$$GGG$ (<(<(A(AA Wdll&:&:&?&?@* $$)) $$)) L  a9 C..r)N)NNT)NN)+rloggingrnumpyrr* paddle.baserrrpaddle.base.log_helperrpaddle.base.wrapped_decoratorr fp16_utilsr r r r rqr__name__INFOrrTrU DENSE_TENSOR SELECTED_ROWSDENSE_TENSOR_ARRAYrIrxr&r-rdrkrrr}rrrrrrrrrrs   55-G 3  gll H   LL%%LL&&LL++ %* FRD 64 &ET@DcL>6p r