Αi SSKrSSKrSSKrSSKrSSKJr SSKJr SSKJrJ r SSK J r J r J r JrJr SSKJrJrJr SSKJrJr SSKJr SS KJrJr SS KJr S S KJr \R@"\!5r"\S 5r#Sr$Sr%Sr&Sr'S5Sjr(\)"5\)"5SS4Sjr*Sr+S\R,S\R,4Sjr,Sr-S6S\S\R,S\.S\/4Sjjr0Sr1Sr2S r3S7S!jr4S\S\R,S\R,S\.S\/4 S"jr5S\S\R,S\.S#\6S$\/S\/4 S%jr7S\S\R,S\.4S&jr8S'r9S(r:S)r;S*rS-r?S.r@S/rAS0rBS1rCS2rDS3rES6S4jrFg)8N)pir) ir_backward) ValueDictValueSet)call_decomp_rulecall_decomp_vjp decomp_ops_contain_unused_outputhas_decomp_rulehas_decomp_vjp)pir_chunk_id_guardpir_op_name_guardpir_op_role_guard)Block Operation)signature_safe_contextmanager) DebugPrintauto_recompute)core)registerc## [R"5nU(d[R"S5 Sv U(d[R"S5 gg!U(d[R"S5 ff=f7f)NTF)r_is_all_prim_enabled_set_prim_all_enabled) prim_states [/var/www/html/banglarbhumi/venv/lib/python3.13/site-packages/paddle/decomposition/decomp.py prim_guardr,sX**,J.  & &t ,   & &u -z  & &u -sA<!AA< A99A<c[U[R5(aU4$[U[R5(a [ U5$[ S[U5S35$)NzType z is not supported.) isinstancerValuetypingSequencetuple TypeErrortype)xss r_build_tensor_tupler&8sN"cii  u B ( (Ry uT"XJ&89 ::c[U5[U5:Xde/n[U5HupE[X[R5(aUR 5[ R"5;a$U[ UR 5;a USbeO3[U5S:Xa"[US[R5(deURUS5 MURU5 M U$)Nrr) len enumeraterrrnamer keysappend) orig_outs decomp_outsopresidxvalues r_analyse_decomp_resultsr4@s y>S- -- - C ,  incii 0 0 =BBDD;BGGIFFQx'''5zQ:eAh +J+JJJ JJuQx JJu - Jr'cTSn/nUR5HnUR5nUR5(ayUR5n[ U[ 5(aAUR 5U:Xa-UR5Vs/sHofR5PM nnURU5 MURS5 M UR 5U:XaU4$X R5Vs/sHo0R5UPM sn-n[U5$s snfs snf)z For standard api of operator, its inputs should keep consistent with organization of its inputs and attrs. Args: op (Operator): The target operator. builtin.combineN) operandssource initializedget_defining_oprrr+r-get_attr_namesattrsr")r0combine_op_nameinputsxinputprev_opitem api_argumentss r_prepare_python_api_argumentsrDRs(O F [[]      ++-G7I..LLNo53:3C3C3EF3E43EF MM% MM$ " wwyO#y5F5F5HI5Hhhjm5HIIM  GJs D 4D%c Sn/nUR5GH+nUR5nUR5(dM+UR5n[ U[ 5(aUR 5U:XaUR5HinUR5RnSU;dM%[R"SUR5RSUR 5S35 g MURnSU;dM[R"SURSUR 5S35 g g)Nr6z;Decomp op does not support dynamic shape -1, but got shape z in inputs of op  Tz in op ) r7r8r9r:rrr+shapewarningswarn)r0r=r>r?r@rArBrHs r_check_prim_dynamicrKss0'O F [[]      ++-G7I..LLNo5#,,.D KKM//EU{ YZ^ZeZeZgZmZmYnnACAHAHAJ@KKLM $ / ;MMUV[VaVaUbbijljqjqjsittuv +r'c 0[U5[U5:Xd"SUS[U5S[U535e[UU5HupVUbUc(U[R;a[ SUSUSU35eUcM8UbUbUb XS;aXdX5'UR nUR nUR n UR n Xx:XdSUSUS U35eS U ;d SUS 35eX:XdSUS U S U 35eUSLUSL- (aS5e g g)aZ Check whether the replaced outputs are consistent with origin outputs. Args: op_name (str): The name of operator. orig_outs (tuple): The outputs of original operator. new_outs (tuple): The outputs of replaced operator. orig_vars (dict): Origin variables of original block. dst_vars (list): Corresponding replaced variables of Origin variables. zwhen replace origin op z[ with composite rule, num of origin outs should be equal to new outs, but len(orig_outs) = z and len(new_outs) = Nzop z2 should not contain any None value. original outs=z and its composite rule outs=z\ with composite rule, origin out dtype should be equal to new out dtype, but orig_out dtype=z and new_out dtype=rFz1 with composite rule, composite out shape has -1.z\ with composite rule, origin out shape should be equal to new out shape, but orig_out shape=z and new_out shape=z"orig_out and new_out should match.)r)ziprops_contain_none ValueErrordtyperH) op_namer.new_outs orig_varsdst_varsorig_outnew_out orig_dtype new_dtype orig_shape new_shapes r_check_op_resultsr[s y>S] * !'+ #I//DS]O U * !   400 0gYPQZP[[xzByCD      $)=(4;Y01!J I!J I* )'3&&0\1DYKQ *Y& )'2cd &* )'3&&0\1DYKQ *!D(W_= 4 = Gr'rFc[RSU-n[U[5(de[U[5(de[R"XX#XE5$)a Search nonbasic ops which have be registered composite rules and replace them with primitive ops. The operators in blacklist will be excluded from program when decomposed into primitives, and only the operators in whitelist will be decomposed. The priority of blacklist is higher than whitelist, it means an operator both in blacklist and whitelist will not be decomposed. The finally set that will be decomposed is: (block.ops & ops have decomposite rule & whitelist) - blacklist Note: All variables must be contained inside the given program. Args: program (Program): The program to be processed. src_vars (list[Value]): In program, once some operator is decomposed, its vars will be replaced by new ones. This argument means some vars will be used later and corresponding vars will be returned for later usage. blacklist (frozenset): The Operators that will be exclude when decomposed into primitives. whitelist (frozenset): Only the operators in whitelist will be decomposed into primitives. start_index (int): The start index of decomposed operator in global block, default 0; end_index (int): The end index of decomposed operator in global block, default -1 means all ops will be composed. start_index and end_index follow the principle of left closed and right open, that is [start_index, end_index). Returns: dst_vars (list): A list contains all vars which replace origin ones in src_vars. forward_blacklist)r prim_configrintsinking_decomp)programsrc_vars blacklist whitelist start_index end_indexs r decomposergsZ>  !45 AI k3 ' '' ' i % %% %   9 r'c6UR5nUR5nUR5UR5:wag[UR55H9nURU5R URU55(aM9 g g)zAcheck whether the inputs of two builtins.combine ops are the sameFT)r: num_operandsrangeoperand_sourceis_same)input1input2builtin_combine_op1builtin_combine_op2is r_check_combine_inputsrrs 002 002'')-@-M-M-OO*779:A#2215=='66q9 ; r'fwd_opbwd_opczUb%UR5S-UR5:wagUR5nUR5Vs/sHo3R5PM nn[ U5[ U5:XdS5e/n[ U5H upgSU;dM UR XF5 M" UR5Vs/sHo3R5PM nnUR5n UR5Vs/sHjnUR5R5(dM(UR5R5R5S:XdMZUR5PMl n nSS/n UHn U R5(aLU R5R5S:Xa*Sn U Hn[X5(dMSn O U (d gMdU [U5;aMuU [U 5;aMU R5R5U ;aM g gs snfs snfs snf)z3check whether the bwd_op is corresponding to fwd_op_gradFz1backward op names do not match backward op inputsr6zpd_op.full_int_arrayz pd_op.fullT) r+get_input_namesr7r8r)r*r-resultsr9r:rrr)rsrtbwd_op_input_namesr? bwd_inputsfwd_op_related_inputs_outputsr2r+ fwd_inputs fwd_outputsfwd_vec_inputsinserted_op_name_listoperandin_fwd vec_inputs r _check_oprs ~0FKKMA//1&,oo&78&7((*&7J8 ! "c*o 5; 5%'!12  $  ) 0 0 A3'-oo&78&7((*&7J8.."K""A 88: ! ! #  HHJ & & ( - - /3D D  "4\B0    ! !'')..04EEF+ (<<!F,8J//h{33**,1137LL%1( M99s H. H3'H81.H8#H8cUR5n/SQn[U5HHupEXS;dM URU5R5nXa;dM2XnUR 5nUs $ g)N)out_gradOut_grad loss_grad)rwr*rr8r:) rtgrad_var_to_varry out_grad_namer2 input_nameroutrss r _get_fwd_opr0sk//19M$%78  &~~c*113H*%/,,. 9 r'blockrreturnc[RRUR5 UR 5nUR 5n[ R"U5n[U5nU=(d UnU(Ga[U5(a SSS5 gUb[R"U5 O[R"U5 [U5n U(a[U5n [XZU5n O[U"U 65n [XEU 5 [!X%U S9 UR 5["R$"5;aN['[)U55H5n U ["UR 5;dM X\R+X5 M7 OPUR 5["R$"5;aUSR+U S5 OUR+U 5 UR-U5 UbLSn UR 5HnUR/5(dMSn O U (aUR-U5 SnU S4sSSS5 $[1U5S4sSSS5 $!,(df  g=f)a Decompose the forward op into a list of primitive ops. Args: block (Block): the block to which the forward op belongs. fwd_op (pir.Operation): the forward op to be decomposed. grad_var_to_var (dict): a dict obtained from distributed processing, which maps the backward grad variable to its corresponding forward variable. prev_op (pir.Operation): the previous op of fwd_op in the block. If prev_op is builtin.combine, insertion point when decomposing fwd_op will be set to prev_op. Returns: new_outputs (tuple(Value)): the new outputs after decomposing. has_decomposed: whether the forward op has been successfully decomposed. NNF)r.rRrTF)rr program_guardrar+rxrget_decomp_ruler rKset_insertion_pointrDrr4r&r[_upgrade_grad_var_to_varr r,rjr)replace_all_uses_with remove_op has_one_user")rrsrrArQr. decom_rulehas_sink_decomp_rulelower input_argsr/rRr2rrBs r_decomp_fwd_opr=s    .++-NN$ --g6 .v622 "6**" / ."''0''/7v>J#.v6 2F/z:/FG g( ; %x  {{} @ E E GG Y0C? NO"<w / .|#U*} / . .s&A&ICI2B&I#I I I-c/nUR5HnUR5R5(aUR5R5R 5S:XajUR5R5n[ SUR 55Vs/sHnURU5PM nnURU5 MURUR5/5 M U$s snf)Nr6r) r7r8r9r:r+rjrirkr-)rs new_inputsr@builtin_combine_oprq new_inputs r_prepare_inputsrsJ" LLN & & ( ( ..0557;LL!&!?!?!A q"4"A"A"CDDA#11!4D    i (   u||~. /# s#C9c4UR5nUR5n[U5[U5:XdS5eUR5Vs/sHoDR 5PM nnUR 5n[U5[U5:XdS5eUR5Vs/sHoDR 5PM nnUR5Vs/sHjnUR 5R 5(dM(UR 5R5R5S:XdMZUR 5PMl nn/n /n [U5HupU R 5(aqU R5R5S:XaOSn UHn[X5(dMSn O U (d'U RU /5 U RXk5 MMU [U5;aMU [U5;aMU RU /5 U RXk5 M /nSnUHLnUS-U ;aURU U5 US- nM'UR[R"5/5 MN U$s snfs snfs snf) Nz7forward op output names do not match forward op outputsz7backward op input names do not match backward op inputsr6FTrrvr)rxget_output_namesr)r7r8rwr9r:r+r*rrr-rr fake_value)rsrtr}fwd_output_namesr?rzbwd_input_namesr|r~ grad_outputsgrad_output_namesrq bwd_inputrrnew_grad_outputsindexfwd_output_names r_prepare_grad_outputsrsc.."K..0  C $4 4A 4'-oo&78&7((*&7J8,,.O  3z? 2A 2 '-oo&78&7((*&7J8""A 88: ! ! #  HHJ & & ( - - /3D D  " L!*-   ! ! # #))+0026GGF+ (>>!F,##YK0!(();< Xj11 55##YK0!(();<'.. E+ g %*; ;  # #L$7 8 QJE  # #S^^%5$6 7 , _99sJ .J'J.J8Jc/n[U5HNup4UR5(aS/[X5-nOS/[X5-nURU5 MP U$)NFT)r*r9r)r-)r| bwd_outputsstop_gradientsr2 bwd_output stop_gradients r_prepare_stop_gradientsrscN$[1  ! ! # #"Gc*/&::M!FS%99Mm, 2 r'c,UcS5eUb4Ub1[U5H"upVX`;dM URU5XU'M$ UbPUbLUR5H7upx[U5H#upURU 5(dMXIX'M% M9 ggg)Nz"grad_var_to_var should not be None)r*popitemsrl) r orig_grads new_gradsr.rRr2 grad_inputgrad_varvarrqorin_vars rrrs  &L(LL &)"7(4OC,2A2E2E3#/ 5 !5,224MH(3 ;;x((08 O- 45"6r'c[U5nUR5Vs/sHoU/PM nn[X5n[XBR55nURR U5n [ UR5n [R"XXgU5n [ UR5n X- n U S:XaNURSR5UR5:XaURURS5 gURSR5S:Xa.URSRS5R5//n /nU H[nUSb.USR5(aURUS5 M7UR[R "55 M] [ U5[ UR55:XdS5e[#X2R5US9 U n[%X5H'nUR'URUU5 US- nM) UR)U5 URU5 [+U5S4$s snf) z Decompose the backward op into a list of primitive ops. If forward op has composite vjp rules (including custom vjp), call call_vjp() to get a list of primitive operators in backward graph, then replace backward op. rrFr builtin.splitrzNresults of original backward op do not match results of decomposed backward oprrT)rrxrropsrr)rcall_vjpr+rrr8r9r-rrrrjmove_oprr")rrsrtr fwd_inputs_ fwd_output fwd_outputs_ grad_outputs_stop_gradients_ bwd_op_idxbefore_num_opsnew_grad_inputs after_num_opsnum_appended_opsr1r insert_idxrqs r_decomp_bwd_with_vjprs "&)K39>>3CD3CZL3CLD)&9M-k>>;KLO(J^Nmm\/O NM$512!3!3!5!F  " & 99R=   ? 2 % " 5 5a 8 ? ? ABCO)J!}(Z]-F-F-H-H :a=) 3>>+, * 3x3v~~/00 \ 0 ! (8C  ~5A MM%))A, 3 !OJ6 $$S) Sz4YEs I#r|fwd_outputs_after_decomposec^^^Tc [S5eUR5Vs/sHoUR5PM nnUR5n[ UU4SjU55n[ U4SjU55n [ U4SjU55n UR R U5n [UR 5n [R"XU5n [UR 5n/nSn[U5HZunnUR5(aURU U5 US- nM6UR[R"55 M\ [TXS9 U n[!X5H'nUR#UR UU5 US- nM) UR%U5 UR'U5 Sn[ U5U4$s snf) aF Decompose the backward op into a list of primitive ops. If forward op has no composite vjp rules, and forward op has been decomposed to a list of primitive operators in forward graph previously, call grad() for the decomposed forward subgraph to get a list of primitive operators in backward graph, then replace backward op. z=To decompose backward op, please decompose forward op firstlyc3l># UH)nU[T5;aMU[T5;aM%Uv M+ g7fN)r).0rr|rs r *_decomp_bwd_without_vjp..Is<#I *- - H%@AA  #s 4 4 4c3.># UH nTUv M g7fr)r grad_outputrs rrrQs8D $ sc3\># UH!nUR5(dMTUv M# g7fr)r9)rrrs rrrTs+%J  ! ! # $ #%s, ,rrrT) RuntimeErrorr7r8rxr"rrr)rgradr*r9r-rrrrjrrr)rrtrr|rr?rz grad_inputsrrrrrrrr1input_grads_idxr2rrrqhas_decomposeds ``` r_decomp_bwd_without_vjpr4s#* K  '-oo&78&7((*&7J8.."K#L8DL%K(J^N!&&|,OO NM CO$[1Z  ! ! # # JJ7 8 q O JJs~~' ( 2K J > 1 eiilJ/a 2   % OOFN :~ %%c9sG c[X5n[X15(d'[RUR 5S35 g[ U5(d[ U5(ag[ UUUU5unnU(dVUR5Vs/sHofR5PM nn[UUU5unn U (a[UUUUU5unnXE4$s snf)a Decompose a backward op in pir program. Get the corresponding forward op according to grad_var_to_var firstly, then (1) try to decompose backward op by calling _decompose_bwd_with_vjp, if forward op has composite vjp rules (including custom vjp), _decompose_bwd_with_vjp will call call_vjp() to get a list of primitive operators in backward graph, then replace backward op successfully and return True; (2) when _decompose_bwd_with_vjp return False, means there is no composite vjp rules, try to decompose forward op firstly by calling _decomp_fwd_op firstly and get corresponding primitive operators in backward graph by calling _decompose_bwd_without_vjp secondly, then replace backward op successfully and return True; (3) if the backward op is still not decomposed by the above two steps, returns False. Args: block (Block): the block to which the backward op belongs. bwd_op (pir.Operation): the backward op to be decomposed. grad_var_to_var (dict): a dict obtained from distributed processing, which maps the backward grad variable to its corresponding forward variable. Return: new_input_grads (tuple(Value)): new results of backward op after decomposing. has_decomposed: whether the backward op has been successfully decomposed. zM can not be decomposed due to the mismatch between forward op and backward opr) rrloggerdebugr+rKrr7r8rr) rrtrrsrbwd_has_decomposedr?r|new_fwd_outputsfwd_has_decomposeds r_decomp_bwd_opr{s4 1F V $ $ {{}oj k 6""&9&&A&A     *0//*;<*;Qhhj*; <         (  "  ((+=sCcN/nUR5nURHnUR5RS5(d&UR5RS5(dMMUR5U;dMcUR UR55 M U$)Nrv_grad_) global_blockrr+endswithr-) pir_programbwd_opsrr0s r_get_all_bwd_opsrsyG++-L GGI  w ' '2779+=+=h+G+Gggiw& NN2779 %  Nr'c/n[R"5n[R"5nURU5 URU5 [R"S5 [R "S5 [ RRRS5Sn[ RRSS05 S[ RRRl URU5 U$)NTFLAGS_enable_pir_api) r_is_fwd_prim_enabled_is_bwd_prim_enabledr-_set_prim_forward_enabled_set_prim_backward_enabledpaddlebase framework get_flags set_flags global_var _use_pir_api_)stateprev_fwd_prim_stateprev_bwd_prim_stateprev_pir_api_flags r_set_prim_staters E335335 LL$% LL$%""4(##D) --778NO  &6:FKK$$2 LL"# Lr'c4[U5S:XdS5e[R"US5 [R"US5 [R R SUS05 US[RR Rl g)NzEstate should contain fwd_prim_state, bwd_prim_state and pir_api_staterrr) r)rrrrrrrrr)rs r_reset_prim_staters| u:?O? ""58,##E!H-  6aAB5:1XFKK$$2r'cr[5nUR5GHup4X0R5;dMX@R5;dM0[X5S:Xa%[X5S:XaXSnXSnXbU'Mf/n/n[X5S:XaUR XS5 O[X5S:Xa>XSR 5R 5S:XaUR XS5 O4[S[X55Hn UR XU 5 M [X5S:XaUR XS5 O[X5S:Xa>XSR 5R 5S:XaUR XS5 OOXSR 5n U R 5RS5(aUR XS5 [U5S:XdS5e[S[U55H n USX'U 'M GM U$)zbtranslate grad_var_to_var (mapping VarDesc->VarDesc) to pir_grad_var_to_var (mapping Value->Value)rrrz builtin.slicerF_z#translate pir_grad_var_to_var error) rrr,r)r-r:r+rjr) param_mappingrpir_grad_var_to_varrr new_grad_varnew_var new_grad_varsnew_varsrqlast_ops r_translate_gradvartovar_to_pirr s'#+(..0  ))+ +7I7I7K0KM+,1 *+q0,6q9 ',Q/4;L1 " }./14!(()@)CD /0A5%/2BBDIIK&'"(()@)CD"1c-*A&BC%,,]-DQ-GHD})*a/OOM$6q$9: *+q0%*1-==?DDF&'OOM$6q$9:+04DDFG||~..s33  (:1(=>8})P+PP)q#m"45A,33K@GGI)::bG% 48 LL 78L7MN9 4 3s CD$$ D2c [U/5 [RSnUR5nSn[R RR U5 UR5RnUGHznUR5nURS5SU;aM.Sn[U5(dMB[R"5(d[U5(aSnU(Gd[UR55 [UR5 [!UR"5 [ R$"U5 UR'5nSn [)U5n [+[-U55Hn XR/5(dMXR15R35n U R5S:XdMRSn [5U R5U R'5X5 U R7X5 UR9U 5 M SSS5 SSS5 SSS5 W (d8[;WW U5n [5UR5X5 UR7U 5 UR9U5 UR5S:XaUnUcGM.SnUR'5HnUR/5(dMSn O U(aUR9U5 SnGM} SSS5 [R R=U5 g!,(df  GN=f!,(df  GN=f!,(df  GN=f!,(df  Nd=f) zs Decompose all non-primitive ops into primitive ops in a pir program. It may contain forward ops and backward ops. backward_blacklistN.rFFTrr6)rgrr^rrrrrr+splitr _enable_prim_dynamic_shaperKr rop_roler chunk_idrrxrrjr)r first_useownerr[rrr4 set_insertion_point_to_block_end)rrcrpre_combine_oprr0r skip_decompr. is_next_splitr/rqnext_oprRrrBs rdecompose_dist_programr%Ss k2  !56I  $ $ &EN  & &{ 3&&(,,B'')K  %b)Y6Kb!!7799)"--"&K")"'')4)"**5*2;;7//3$&JJL (- &5b&9 !&s9~!6A(|7799*3,*@*@*B*H*H*J#*<<>_#D48M$5(/ (/(9(3%& %,$A$A(3%&%*OOG$<"78650)#:%{B$*"'')YI00:OOB'779 11%'N!- $I . 6 6 8++--(-I!!9!7%)Nq 4v JJ//6W876554 4 3swAM:A MML1 3A L 2L AL "L1 *M2A/M&(M%M L.)L1 1 M ;M M M M#cr[5n[X5n[X5 [X5 [ U5 U$)a^ Decompose all PHI ops into prim ops in a pir program. Args: pir_program (Program): the program to be decomposed param_mapping (dict): a map of program variables to pir program values grad_var_to_var (dict): a dict obtained from distributed processing, which maps the backward grad variable to its corresponding forward variable. )rr rrr)rrrrrs rdecompose_pir_programr's:  E8 9 9e r'c(/nUR5RHqnUR5S:Xa"URUR 5S5 UR5S:XdMOURUR 5S5 Ms U$)Nz pd_op.datarzbuiltin.parameter)rrr+r-rxrrxr0s r"get_inputs_from_data_and_parameterr*spG&&(,, 779 $ NN2::<? + 779+ + NN2::<? + - Nr'c/nUR5RHGnUR5S:XdMURUR S5R 55 MI U$)Nz pd_op.fetchr)rrr+r-rr8r)s rget_outputs_from_fetch_opr,sRG&&(,, 779 % NN2::a=//1 2- Nr'c/n[5nUR5H upEXCU'M UHnURX65 M U$r)rrr-)outputsrrx var2grad_varkvoutputs rget_grad_var_for_listr3sHG;L#))+Q,|+, Nr'ct^U4Sjn/nUH(nURU"UR555 M* U$)Nc>[TR5R5HupX:XdM Us $ [S5eNzop not found in programr*rrrr0r2op_iterras rgetIdx'get_defining_op_indices..getIdx<%g&:&:&<&@&@ALC} B455r')r-r:)ra output_valuesr:rxr2s` rget_defining_op_indicesr>s86 Gvf44678 Nr'c^U4Sjn/nTR5RH)nU"U5(dMURU"U55 M+ U$)Nc>[TR5R5HupX:XdM Us $ [S5er6r7r8s rr:#get_forward_op_idxs..getIdxr<r')rrr-)rais_forward_op_funcr:rxr0s` rget_forward_op_idxsrCsJ6 G""$(( b ! ! NN6": &) Nr'c[S5 [SU5 [U5n[U5nSn[U5(a[ [ X55n[ R"S5nUR[ R5 Ub[ [X55nUS:XaURS5 U$US-n[UUU/UU5upxU$! URS5 NG=f)NzStart Recompute Pir Program:zBefore Recompute: rFzauto-recomputezNo Forward Ops Found!zSkip Auto Recompute!r) rr*r,r)maxr>logging getLoggersetLevelINFOrCinfor) rrBr>r.fwd_op_end_idxrbackward_op_start_idxrars rauto_recompute_pir_programrMs-.#[1 / rCrMrr'rr[s  '> 7GE!   8 $..;$ B 8<@5vkk $N"/ MM/ MM/d IMN+ N+--N+:>N+ N+b$6r  <*8 8 MM8  MM8  8  8 vD& D& MMD&D& D& "' D&  D&NE) E) MME)E)P&>+\. FG7T.  #r'