Αi,SSKJr SSKJr SSKrSSKJr SSKJr SSK J r SSK J r \(aSSKJ r SSKJr SS K Jr SS KJr SS jrSS jrSS jrSSSjjrg)) annotations) TYPE_CHECKINGN) framework) data_feeder)_get_global_group)Tensor)task)Group)ExecutionStreamTypecUc [5OUnU(aURRUU5$URRXU5nU(aUR"5 U$)N)r process_group%all_gather_into_tensor_on_calc_streamall_gather_into_tensorwait) out_tensor in_tensorgroupsync_opuse_calc_streamr s r/var/www/html/banglarbhumi/venv/lib/python3.13/site-packages/paddle/distributed/communication/stream/all_gather.py"_all_gather_into_tensor_in_dygraphr!se$)=  eE""HH       5 5w D KcrUc [5OUn[U5S:Xa=U[UR5Vs/sHn[R "U5PM sn- nU(aUR RX5$UR RXU5nU(aUR"5 U$s snf)Nr) rlenrangenrankspaddle empty_liker all_gather_on_calc_stream all_gatherr) tensor_listtensorrrr_r s r_all_gather_in_dygraphr$9s$)=  eE ;15;NO;Na))&1;NOO ""<<       ) )+w GD KPs B4cSn[R"U40[5D6nURURS9nUHn[ R "US/SQS5 M [ R "US/SQS5 UcSO URn[R"5n URUSU/0SU/0UU S .S 9n U(a)[RRU RlUR!5 [#UR$5S:Xa'UR'[(R*"US55 gUR'[(R,"XiS55 g) Nr )dtyper!) float16float32float64int32int64boolint8uint8 complex64 complex128r"rxout)ring_idr)typeinputsoutputsattrs)r LayerHelperlocals"create_variable_for_type_inferencer&rcheck_variable_and_dtypeiddistget_world_size append_opr DefaultStreamvalue dist_attrexecution_streamclearrshapeextendrunstacksplit) r!r"rrop_typehelperr2elemr3rops r_all_gather_in_static_moderMQs?G  " "7 7fh 7F  3 3&,, 3 GC,,     $((  $=aehhG  "F    fX  B(;(I(I(O(O % 6<<A6>>#q126<<Q78rcUb UR5(d [S5eU(dU(a [S5e[R"5(a5[R "U5(a [ XX#U5$[XX#U5$UbS5e[R "U5(a [S5e[XX#5$)a Gather tensors across devices to a correctly-sized tensor or a tensor list. Args: tensor_or_tensor_list (Union[Tensor, List[Tensor]]): The output. If it is a tensor, it should be correctly-sized. If it is a list, it should be empty or contain correctly-sized tensors. tensor (Tensor): The input tensor on each rank. The result will overwrite this tenor after communication. Support float16, float32, float64, int32, int64, int8, uint, bool, complex64 or complex128 as the input data type. group (Group, optional): Communicate in which group. If none is given, use the global group as default. sync_op (bool, optional): Indicate whether the communication is sync or not. If none is given, use true as default. use_calc_stream (bool, optional): Indicate whether the communication is done on calculation stream. If none is given, use false as default. This option is designed for high performance demand, be careful to turn it on except you are clearly know its meaning. Returns: Return a task object. Warning: This API only supports the dygraph mode now. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env: DISTRIBUTED) >>> import paddle >>> import paddle.distributed as dist >>> dist.init_parallel_env() >>> local_rank = dist.get_rank() >>> tensor_list = [] # type: ignore >>> if local_rank == 0: ... data = paddle.to_tensor([[4, 5, 6], [4, 5, 6]]) >>> else: ... data = paddle.to_tensor([[1, 2, 3], [1, 2, 3]]) >>> task = dist.stream.all_gather(tensor_list, data, sync_op=False) >>> task.wait() # type: ignore[union-attr] >>> print(tensor_list) [[[4, 5, 6], [4, 5, 6]], [[1, 2, 3], [1, 2, 3]]] (2 GPUs) zjThe group should not be None and all ranks which invoke this operation should be the member of this group.z5use_calc_stream can only be true in sync op behavior.z3Group can not be used in static graph mode for now.zLOnly support passing a tensor list to `all_gather` in static graph mode now.) is_member RuntimeErrorrin_dynamic_moder is_tensorrr$rM)tensor_or_tensor_listr"rrrs rr r s\ !2!2 x    C    ""   1 2 25%u *%u } A }   1 2 2^ .%u r) rrrrrr rr,rr,returnr ) r! list[Tensor]r"rrr rr,rr,rTr ) r!rUr"rrr rr,rTNone)NTF) rSzTensor | list[Tensor]r"rrz Group | Nonerr,rr,rTz task | None) __future__rtypingrrpaddle.distributed distributedr=r paddle.baser&paddle.distributed.communication.grouprrpaddle.base.corer r %paddle.distributed.utils.stream_utilsr rr$rMr rrr`s"# !#D%<E      0       0<9<9'-<96;<9FJ<9 <9D! L0L L L L  L  Lr