Αi SSKJr SSKJr "SS5r"SS\5r"SS\5r"S S \5r"S S \5r g) N)RNGStatesTrackercV\rSrSrSrSrSrSr\S5r \S5r Sr S r S r g ) StrategyGroupBasea The base class of communication group with distributed strategy. Args: list_of_ranks: A 2D-array, such as `[[0, 1, 2, 3], [4, 5, 6, 7]]`. Ranks in sublist represents they are in the same communication group. Returns: The instance of strategy group. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env: DISTRIBUTED) >>> import paddle.distributed as dist >>> from paddle.distributed.fleet.base.strategy_group import StrategyGroupBase >>> dist.init_parallel_env() >>> strategy_group = dist.fleet.base.strategy_group.StrategyGroupBase([[0, 1], [2, 3]]) >>> print(strategy_group.world_size) 2 c[R"5(dS5e[U5(dS5e[R"5UlXlUR 5Ul[5Ul g)z% Initialize the communication group. z6The global communication group need to be initialized.z#The list_of_ranks can not be empty.N) distis_initializedlenget_rank_rank_list_of_ranks _create_group_grouprrandom_states_tracker)self list_of_rankss l/var/www/html/banglarbhumi/venv/lib/python3.13/site-packages/paddle/distributed/fleet/base/strategy_group.py__init__StrategyGroupBase.__init__.sj""$$ D $=!!H#HH!]]_ +((* %5%7"c:URRX5 g)z# Add random seed for current rank. N)radd)rnameseeds radd_random_seed!StrategyGroupBase.add_random_seed;s ""&&t2rcUR$)z Get the random states tracker. )rrs rget_random_states_tracker+StrategyGroupBase.get_random_states_trackerAs)))rc^/mURHnTR[U55 M [U4SjT55nU(aTS$T$)z The world size of communication group. Returns: Integer if the world_size of each group are equal, or a list of world_size if they are not equal. c32># UH oTS:Hv M g7f)rN).0 world_sizeworld_size_lists r /StrategyGroupBase.world_size..Rs ?N/!, ,sr)r appendr all)rranksis_valuer&s @rr%StrategyGroupBase.world_sizeGsW((E  " "3u: .) ?N  &.q!B?BrcUR$)z The communication group which current rank belongs to. Returns: Group if current rank only belong to single communication group, or a list of Group if it belongs many. )rrs rgroupStrategyGroupBase.groupWs{{rcD/UlURHDn[R"US9nURU;dM)URR U5 MF UR(dg[ UR5S:XaURS$UR$)Nr+r) list_of_groupr r new_groupr r)r )rr+r/s rrStrategyGroupBase._create_groupas((ENN/EzzU"""))%0) !!t))*a/""1% '' rcSURS3nUR(dUS-$[[UR55HnUSUSURUS3- nM U$)Nzseed: z; z No group.zGroup[z]: )_seedr4ranger )r debug_stris r__repr__StrategyGroupBase.__repr__qsmTZZL+ !!{* *s4--./A 6!C(:(:1(='>bA AI0r)rr r r4rN)__name__ __module__ __qualname____firstlineno____doc__rrrpropertyr%r/rr<__static_attributes__r#rrrrsJ2 83 *  C C rrc,^\rSrSrSrU4SjrSrU=r$)DPGroupza The communication group strategy for data parallel. Args: list_of_ranks: A 2D-array, such as `[[0, 1, 2, 3], [4, 5, 6, 7]]`. Ranks in sublist represents they are in the same communication group. Returns: The instance of data parallel strategy group. c>[TU]U5 [UR[5(aSUR S35eg)NRank z belongs to multi dp groupssuperr isinstancer/listr rr __class__s rrDPGroup.__init__B 'djj$// DJJ<: ; //rr#r>r?r@rArBrrD __classcell__rOs@rrFrFz   rrFc,^\rSrSrSrU4SjrSrU=r$)MPGroupa The communication group strategy for model parallel. Args: list_of_ranks: A 2D-array, such as `[[0, 1, 2, 3], [4, 5, 6, 7]]`. Ranks in sublist represents they are in the same communication group. Returns: The instance of model parallel strategy group. c>[TU]U5 [UR[5(aSUR S35eg)NrIz belongs to multi mp groupsrJrNs rrMPGroup.__init__rQrr#rRrTs@rrWrWrUrrWc,^\rSrSrSrU4SjrSrU=r$) ShardingGroupa  The communication group strategy for sharding parallel. Args: list_of_ranks: A 2D-array, such as `[[0, 1, 2, 3], [4, 5, 6, 7]]`. Ranks in sublist represents they are in the same communication group. Returns: The instance of sharding parallel strategy group. c>[TU]U5 [UR[5(aSUR S35eg)NrIz! belongs to multi sharding groupsrJrNs rrShardingGroup.__init__sB 'djj$// DJJ<@ A //rr#rRrTs@rr\r\rUrr\cb^\rSrSrSrU4Sjr\S5r\S5r\S5r Sr Sr U=r $) PPGroupa  The communication group strategy for pipeline parallel. Args: list_of_ranks: A 2D-array, such as `[[0, 1, 2, 3], [4, 5, 6, 7]]`. Ranks in sublist represents they are in the same communication group. Returns: The instance of pipeline parallel strategy group. c">[TU]U5 [UR[5(aSUR S35eSUlSUlSUlSUl SUl SUl URS:aUR5 gg)NrIz belongs to multi pp groupsr3)rKrrLr/rMr _send_next_group_send_prev_group_recv_next_group_recv_prev_group_rank_of_next_stage_rank_of_prev_stager%_create_p2p_grouprNs rrPPGroup.__init__s 'djj$// DJJ<: ; /!% $ $ $#' #' ??Q   " " $ rcUR$)zp Rank of the previous pp stage. Returns: The global rank of previous pp stage. `None` if without previous. )rirs rrank_of_prev_stagePPGroup.rank_of_prev_stage'''rcUR$)zd Rank of the next pp stage. Returns: The global rank of next pp stage. `None` if without next. )rhrs rrank_of_next_stagePPGroup.rank_of_next_stagerorc^URURURUR4$)z Communication subgroup in order to switch data with previous and next stage. Returns: Four subgroups including send/recv to/from prev/next. )rdrerfrgrs r p2p_groupsPPGroup.p2p_groupss2  ! !  ! !  ! !  ! !   rcURnURHn[U5Hup4X#S-U-nX#S- U-nURU:Xa XPlX`l[ R"XE/S9nURU:XaXplOURU:XaXpl [ R"Xd/S9nURU:XaXl MURU:XdMXl M M UR(a3UR(a"UR(aUR(dSURS35eg)Nr3r2z/Error occurs while creating p2p group for rank .) r%r enumerater rhrirr5rdrgrerf) rdegreer+idxrank next_rank prev_rank next_group prev_groups rrjPPGroup._create_p2p_groups((E&u- !7f"45 !7f"45 ::%/8,/8,!^^42CD ::%,6)ZZ9,,6)!^^92CD ::%,6)ZZ9,,6)'.).  ! !%%%%%%  K =TZZL J  K &&r)rhrirfrgrdre) r>r?r@rArBrrCrmrqrtrjrDrSrTs@rrarasU % ((((    KKrra) paddle.distributed distributedr#paddle.distributed.fleet.layers.mpurrrFrWr\rar#rrrsR "@ccL  &  & % &\K\Kr