Αi,$SSKJr SSKrSSKJrJr SSKrSSKJr SSKJ r \(a SSKJ r SSK J r "SS5r"S S 5rS rS rS rSrSrSSjrSSSjjrSSSjjrSrSSjrSSSjjrSSSjjrSSSjjrg) ) annotationsN) TYPE_CHECKINGLiteral) framework)Tensor) ProcessGroupc\rSrSrSrSSSjjr\SSj5r\SSj5r\SSj5r \SSj5r \SS j5r \SS j5r \SS j5r \SS j5rSS jrSSjrSSjrSSjrSrg)Groupz' The abstract representation of group. NcpXlUS:a [U5OSUlX lX0lX@lXPlg)Nr)_rank_in_grouplen _world_size_id_ranks_pg_name)self rank_in_groupidrankspgnames f/var/www/html/banglarbhumi/venv/lib/python3.13/site-packages/paddle/distributed/communication/group.py__init__Group.__init__"s3,)6!);3u:  cUR$N)rrs rrank Group.rank1s"""rcUR$r )rr!s rr Group.ranks5s {{rc,[UR5$r )rrr!s rnranks Group.nranks9s4;;rcUR$r )rr!s rr Group.name=s zzrcUR$r )rr!s r process_groupGroup.process_groupA xxrcUR$r )rr!s r world_sizeGroup.world_sizeEsrc6URR5$r )rrr!s rbackend Group.backendIsxx}}rcUR$r )rr!s rrGroup.idMr.rcHURS:agURS:agg)NrFT)r"r'r!s r is_memberGroup.is_memberQs! 99q= ;;?rcdUR5(aURRU5$g)Nr )r9rindexrr"s rget_group_rankGroup.get_group_rankXs' >>  ::##D) )rcLUR5(aURU$g)z Get the global rank of a process within a group. Args: rank (int): The local rank within the group. Returns: If the current process is a member of the group, returns the corresponding global rank; otherwise returns -1. r )r9rr=s rget_global_rankGroup.get_global_rank^s# >>  ::d# #rcSURSURSURS3nUSR[ [ UR 55- nUS- nXR(aUR- nU$S- nU$)Nzrank: z , nranks: z, id: z , ranks: z, z; name: None)r"r'rjoinmapstrrr)r debug_strs r__repr__Group.__repr__os}TYYKz$++fTWWIY O  TYYs3 344 Z ))TYY7 287 r)rrrrrr)NN) rintrrKr list[int]rzProcessGroup | Noner str | NonereturnrD)rNrK)rNrL)rNrM)rNr)rNrGrNbool)r"rKrNzint | Literal[-1])__name__ __module__ __qualname____firstlineno____doc__rpropertyr"rr'rr,r0r3rr9r>rArI__static_attributes__rrr r s#'          ##     "rr c\rSrSrSr0rSrg) _GroupManageryrrXN)rQrRrSrTglobal_group_idgroup_map_by_idrWrXrrrZrZys OOrrZc[R[R;a [S5e[R[R$)Nz$The global group is not initialized.)rZr\r] RuntimeErrorrXrr_get_global_groupr`~s9$$M,I,IIABB  ( ()F)F GGrcUR[R;a[SURS35eU[RUR'g)NzThe group with id z already exist.)rrZr]r_groups r_add_new_grouprdsA xx=000/zIJJ.3M!!%((+rc<UR[R:H$r )rrZr\rbs r_is_global_grouprfs 88}44 44rc[R"5nU(a<UR5(d'[R"SUSUR 35 gg)NzCurrent global rank z is not in group TF)distget_rankr9warningswarnr)rc global_ranks r_warn_cur_rank_not_in_grouprmsE--/K U__&& ";-/@ M  rcbURU5nUS:dSUSUR35eU$)NrzThe input rank z# can not be found inside the group )r>r)rlrc group_ranks r_get_or_throw_group_rankrpsB%%k2J ? +&I%**V? rcD[R[R;$)a Check whether the distributed environment has been initialized Returns: `True` if distributed environment has been initialized, otherwise `False`. Warning: This API only supports the dygraph mode. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env: DISTRIBUTED) >>> import paddle >>> print(paddle.distributed.is_initialized()) False >>> paddle.distributed.init_parallel_env() >>> print(paddle.distributed.is_initialized()) True )rZr\r]rXrris_initializedrrs2  ( (M,I,I IIrcUc [5OUnUR[R;dSURS35e[ U5(a[RR 5 g[RUR g)a- Destroy a given group for communication Args: group (Group, optional): The group to be destroyed. All of process groups, including the default group, will be destroyed and the distributed environment will be deinitialized. Returns : None Warning: This API only supports the dygraph mode. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env: DISTRIBUTED) >>> import paddle >>> import paddle.distributed as dist >>> dist.init_parallel_env() >>> group = dist.new_group([0, 1]) >>> dist.destroy_process_group(group) >>> print(dist.is_initialized()) True >>> dist.destroy_process_group() >>> print(dist.is_initialized()) False NzDestroy group with id z is invalid.)r`rrZr]rfclearrbs rdestroy_process_grouprust@$)=  eE 88}44 4  ,7 4%%++-  ) )%(( 3rcU[R;a[RU$[R"SUS35 g)a Get group instance by group id. Args: id (int): the group id. Default value is 0. Returns: Group: the group instance. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env: DISTRIBUTED) >>> import paddle >>> import paddle.distributed as dist >>> dist.init_parallel_env() >>> gid = paddle.distributed.new_group([2,4,6]) >>> paddle.distributed.get_group(gid.id) zGroup z is not initialized.N)rZr]rjrk)rs r get_grouprws<0 ] * **,,R00 MMF2$234 rc[R"5(a[RR U5$Sn[R "U40[ 5D6nURUSU/0SU/0S9 g)Nc_sync_calc_streamXOut)typeinputsoutputs)rin_dynamic_modepaddle_C_opssync_calc_stream LayerHelperlocals append_op)tensorop_typehelpers r_sync_calc_streamrsk  ""}}--f55&&&w;&(;&?VH%  rc[R"5(a![RR U/U5$Sn[R "U40[ 5D6nURUSU/0SU/0SU0S9 g)Nc_sync_comm_streamrzr{ring_idr|r}r~attrs)rrrrsync_comm_streamrrr)rrrrs r_sync_comm_streamrsw  ""}}--vh@@&&&w;&(;&?VH%g&  rcUbUR5(dgU(a [U5 gUcSO URn[X5 g)al wait to sync stream for group. Args: tensor (Tensor): The Tensor used before sync. group (Group): The Group instance to perform sync. use_calc_stream (bool): Whether to use calculation stream (True) or communication stream (False). Default to True. Returns: None. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env: DISTRIBUTED) >>> import paddle >>> paddle.distributed.init_parallel_env() >>> tindata = paddle.randn(shape=[2, 3]) >>> paddle.distributed.all_reduce(tindata, sync_op=True) >>> paddle.distributed.wait(tindata) Nr)r9rrr)rrcuse_calc_streamrs rwaitrs;8 !2!2&!}!%((&*rcvUbUR5(dg[R"5(aUc [5OUn[R"5n[ U[R 5(aURR5nO+UR5nURRU5nUR5 gUcSO URn[R"S/SSS9n[R"5(aO[RRR5(d![R RXUSU5$gSn[ U["5(d [%S5e[R&"U40[)5D6nUR+US U/0S U/0SU0S 9 g) a Barrier among all participators in the group. Args: group (Group): The group instance return by new_group or None for global default group. Returns: None. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env: DISTRIBUTED) >>> import paddle >>> from paddle.distributed import init_parallel_env >>> paddle.set_device(f'gpu:{paddle.distributed.ParallelEnv().dev_id}') >>> init_parallel_env() >>> paddle.distributed.barrier() Nrint32)dtyperbarrierz,The type of 'group' for barrier must be int.rzr{r)r9rrr`_current_expected_place isinstanceCPUPlacer,r get_device_idrrrfullcoreis_compiled_with_xpu _legacy_C_opsrK ValueErrorrrr)rcplacetask device_idrbarrier_tensorrrs rrrEsy, !2!2  ""',}!#%113 eY// 0 0&&..0D++-I&&..y9D =aehhG[[!aw7N  ""$$99;;''// 7 < '3''KL L&&w;&(;.)*^,-g&  rc[U5(a [S5eUc[5nUR$UnUR$)a Get the backend of given group. Args: group (Group): The group to work on. Use the global group as default. Returns: Returns the name of the given group backend. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env: DISTRIBUTED) >>> import paddle >>> paddle.distributed.init_parallel_env() >>> paddle.distributed.get_backend() NCCL zInvalid group specified)rmr_r`r3rbs r get_backendrsC(#5))455#(=  E ==7rs#) !-YYx H 4 5J8'4T<    IM#+ #+'#+AE#+ #+L7 tr