Αi$SSKrSSKrSSKrSSKJr SSKJr SSKJr SSK J r SSK J r J r /rSr"SS 5r\"5rS rS S jrSS jrg)N) _legacy_C_ops)core)check_variable_and_dtype)Variable) LayerHelperin_dynamic_modemodel_parallel_rngc`\rSrSrSrSrSrSrSrSr \ R\ 4Sj5r S rg ) RNGStatesTracker"z Tracker the RNG states. c00Ul[5UlgNstates_setseeds_selfs j/var/www/html/banglarbhumi/venv/lib/python3.13/site-packages/paddle/distributed/fleet/layers/mpu/random.py__init__RNGStatesTracker.__init__'s e c00Ul[5Ulgrrrs rresetRNGStatesTracker.reset,s e rcX R;a[SUS35eURRU5 XR;a[SUS35e[R R SS9n[R R5URU'[R"U5 [R RUSS9 g)Nzseed z already existsstate T use_index) r ValueErroraddrpaddleincubate get_rng_stateregister_rng_state_as_indexseed set_rng_state)rnamer&orig_rng_state_indexs rr!RNGStatesTracker.add0s ;; uTF/:; ;  << vdV?;< <%<!>!>!NDLL  OO ) )*>$ ) O"(!>!>!>!NDLL  OO ) )*>$ ) Os A'D*B7.A D7A DD)rrN)__name__ __module__ __qualname____firstlineno____doc__rrr!r.r1 contextlibcontextmanagerMODEL_PARALLEL_RNGr4__static_attributes__rrr r "sA  L  L/ P Prr c[$r)RNG_STATE_TRACKERr?rrget_rng_state_trackerrBcs rcSSKJn UR5nUR5nUR 5nUR 5nUR 5nU(aUnUS-X6--U-nO-[RRSS5nUS-X6--U-n[R5 [R[U5 [R"U5 g)Nr)fleeti')paddle.distributedrDget_hybrid_communicate_groupget_model_parallel_rankget_model_parallel_world_size get_stage_idget_pipe_parallel_world_sizenprandomrandintrArr!r=r"r&) r&rDhcgmp_rankmp_sizepp_rankpp_size global_seed local_seeds rmodel_parallel_random_seedrVgs(  , , .C))+G//1G G..0G  AX 11G; ii''51  1_w'887B ,j9 KK rc Uc+[RRRXX$XV5$[ U[ [ [45(d [S5e[ U[ [ 45(aUS:XaU$SUs=::aS::dO [S55eUS;d[S55eUb[S55eUS:XaS OUn[5(a)[R"US US U(+S S SSSU5 upxU$[ U[5(a)URS/:wd[SUR35e[S0[5D6n [US/SQS5 U R![R"S9n U R%SSU 0S9 U R!UR&S9nU R![(R*R,R.SS9nU R%SU/U S.U/U/S.UU(+US.S9 U$)a Dropout is a regularization technique for reducing overfitting by preventing neuron co-adaption during training. The dropout operator randomly sets the outputs of some units to zero, while upscale others according to the given dropout probability. Args: x (Tensor): The input tensor. The data type is float32 or float64. p (float|int): Probability of setting units to zero. Default 0.5. axis (int|list|tuple): The axis along which the dropout is performed. Default None. rng_name (str): The random seed generator name, which used to obtain deterministic results. training (bool): A flag indicating whether it is in train phrase or not. Default True. mode(str): ['upscale_in_train'(default) | 'downscale_in_infer']. 1. upscale_in_train(default), upscale the output at training time - train: out = input * mask / ( 1.0 - dropout_prob ) - inference: out = input 2. downscale_in_infer, downscale the output at inference - train: out = input * mask - inference: out = input * (1.0 - dropout_prob) name (str, optional): Name for the operation (optional, default is None). For more information, please refer to :ref:`api_guide_Name`. Returns: A Tensor representing the dropout, has same shape and data type as `x` . Examples: We use ``p=0.5`` in the following description for simplicity. 1. When ``axis=None`` , this is commonly used dropout, which dropout each element of x randomly. .. code-block:: text Let's see a simple case when x is a 2d tensor with shape 2*3: [[1 2 3] [4 5 6]] we generate mask with the same shape as x, which is 2*3. The value of mask is sampled from a Bernoulli distribution randomly. For example, we may get such mask: [[0 1 0] [1 0 1]] So the output is obtained from elementwise multiply of x and mask: [[0 2 0] [4 0 6]] Using default setting, i.e. ``mode='upscale_in_train'`` , if in training phase, the final upscale output is: [[0 4 0 ] [8 0 12]] if in test phase, the output is the same as input: [[1 2 3] [4 5 6]] we can also set ``mode='downscale_in_infer'`` , then if in training phase, the final output is: [[0 2 0] [4 0 6]] if in test phase, the scale output is: [[0.5 1. 1.5] [2. 2.5 3. ]] z4p argument should be a number(int|float) or VariablerrEz!p argument should between 0 and 1)downscale_in_inferupscale_in_trainzBmode argument should be 'downscale_in_infer' or 'upscale_in_train'z1unsupported axis when using random seed generatorrXdowngrade_in_infer dropout_probis_testfix_seedFr&dropout_implementationzGRequired p.shape == [1] if type(p) is Variable, but received p.shape = dropoutx)float16float32float64)dtypeOut)typeoutputsT)rd stop_gradient)XSeed)reMask)r[r\r^)rfinputsrgattrs)r_)r"nn functionalr_ isinstancefloatintr TypeErrorr rrshaperlocalsr"create_variable_for_type_inferenceint32 append_oprdrVarDescVarTypeUINT8) r`paxisrng_nametrainingmoder(outmaskhelperr&s rr_r_s.Nyy##++A$$MM a%h/ 0 0NOO!c5\""qAv ;Q;G #FGG; = =zL@ = <;< !%(< <$ !))  L    $    a " "177qc>YZ[ZaZaYbc 3&(3 s5y 88v||8Lfudm<77agg7F88,,&&,,D9  d+ ED62 !'<*.   rr)g?NNTrYN)r;numpyrLr"r paddle.baserpaddle.base.data_feederrpaddle.common_ops_importrpaddle.frameworkrr__all__r=r rArBrVr_r?rrrsf <-9 ) ;P;P|%&4    Kr