ΑiZ#4SSKJr SSKJr SSKrSSKJr SSKJr SSK J r SSK J r J r SSKJrJr \(aSSKJr SS KJr SS KJr SS K Jr SS KJr SS jrSrSr\R6SSS4SSjjr\R6SSS4Sjrg)) annotations) TYPE_CHECKINGN) framework) data_feeder)_get_global_group_warn_cur_rank_not_in_group)ReduceOp_get_reduce_op)Sequence)Tensor)task)Group) _ReduceOpc[U5nU(aURRXU5$URRXXt5nU(aUR"5 U$N)r process_group$reduce_scatter_tensor_on_calc_streamreduce_scatter_tensorwait) out_tensor in_tensoropgroupsync_opuse_calc_streamcallerop_typer s v/var/www/html/banglarbhumi/venv/lib/python3.13/site-packages/paddle/distributed/communication/stream/reduce_scatter.py!_reduce_scatter_tensor_in_dygraphr&s_R G""GG 7      4 4w D Kc[U5nU(aURRXU5$URRXXd5nU(aUR"5 U$r)r rreduce_scatter_on_calc_streamreduce_scatterr)tensor tensor_listrrrrrr s r_reduce_scatter_in_dygraphr&?s_R G""@@       - -W D Kr cSn[R"US/SQU5 [R"U40[ 5D6nUcSO UR n[ R"5nURUSU/0SU/0UUS.S9 g) Nr#r$) float16float32float64int32int64int8uint8booluint16rxout)ring_idnranks)typeinputsoutputsattrs) rcheck_variable_and_dtyper LayerHelperlocalsiddistget_world_size append_op)r$tensor_or_tensor_listrrhelperr3r4s r_reduce_scatter_in_static_moderBRsG((  " " "7 7fh 7F=aehhG  "F  +,-! r TFcJ[U5(agU(dU(a [S5e[R"5(aJUc [ 5OUn[ R "U5(a[UUUUUU5$[UUUUUU5$UbS5e[XU5$)a[ Reduce, then scatter a tensor (or a tensor list) across devices. Args: tensor (Tensor): The output tensor on each rank. The result will overwrite this tenor after communication. Support float16, float32, float64, int32, int64, int8, uint8 or bool as the input data type. tensor_or_tensor_list (Union[Tensor, List[Tensor]]): The input to scatter. If it is a tensor, it should be correctly-sized. If it is a list, it should contain correctly-sized tensors. op (ReduceOp.SUM|ReduceOp.MAX|ReduceOp.MIN|ReduceOp.PROD, optional): The reduction used. If none is given, use ReduceOp.SUM as default. group (Group|None, optional): Communicate in which group. If none is given, use the global group as default. sync_op (bool, optional): Indicate whether the communication is sync or not. If none is given, use true as default. use_calc_stream (bool, optional): Indicate whether the communication is done on calculation stream. If none is given, use false as default. This option is designed for high performance demand, be careful to turn it on except you are clearly know its meaning. Returns: Return a task object. Warning: This API only supports the dygraph mode now. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env: DISTRIBUTED) >>> import paddle >>> import paddle.distributed as dist >>> dist.init_parallel_env() >>> if dist.get_rank() == 0: ... data1 = paddle.to_tensor([0, 1]) ... data2 = paddle.to_tensor([2, 3]) >>> else: ... data1 = paddle.to_tensor([4, 5]) ... data2 = paddle.to_tensor([6, 7]) >>> dist.stream.reduce_scatter(data1, [data1, data2]) >>> out = data1.numpy() >>> print(out) >>> # [4, 6] (2 GPUs, out for rank 0) >>> # [8, 10] (2 GPUs, out for rank 1) N5use_calc_stream can only be true in sync op behavior.z3Group can not be used in static graph mode for now.) r RuntimeErrorrin_dynamic_moderpaddle is_tensorrr&rB)r$r@rrrrs rr#r#tsb#5))  C    ""',}!#%   1 2 24%  .%  } A }. 5  r c [U5(agU(dU(a [S5e[R"5(a Uc [ 5OUn[ UUUUUUS5$[S5e)a) Reduce, then scatter a flattened tensor across devices. Args: out_tensor (Tensor): The output tensor on each rank. The result will overwrite this tenor after communication. Support float16, float32, float64, int32 or int64 as the input data type. in_tensor (Tensor): The input tensor to reduce and scatter. op (ReduceOp.SUM|ReduceOp.MAX|ReduceOp.MIN|ReduceOp.PROD, optional): The reduction used. If none is given, use ReduceOp.SUM as default. group (Group, optional): Communicate in which group. If none is given, use the global group as default. sync_op (bool, optional): Indicate whether the communication is sync or not. If none is given, use true as default. use_calc_stream (bool, optional): Indicate whether the communication is done on calculation stream. If none is given, use false as default. This option is designed for high performance demand, be careful to turn it on except you are clearly know its meaning. Returns: Return a task object. Warning: This API will be deprecated in the future, and only supports the dygraph mode now. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env: DISTRIBUTED) >>> import paddle >>> import paddle.distributed as dist >>> dist.init_parallel_env() >>> if dist.get_rank() == 0: ... data1 = paddle.to_tensor([7, 8, 9]) ... data2 = paddle.to_tensor([10, 11, 12]) ... dist.stream.scatter(data1, src=1) >>> else: ... data1 = paddle.to_tensor([1, 2, 3]) ... data2 = paddle.to_tensor([4, 5, 6]) ... dist.stream.scatter(data1, [data1, data2], src=1) >>> out = data1.numpy() >>> print(out) >>> # [1, 2, 3] (2 GPUs, out for rank 0) >>> # [4, 5, 6] (2 GPUs, out for rank 1) NrD_reduce_scatter_basezUpaddle.distributed.stream._reduce_scatter_base is only supported in dygraph mode now.)rrErrFrr)rrrrrrs rrJrJs|b#5))  C    ""',}!#%0       "   _ r )r#)r$r r@zTensor | Sequence[Tensor]rrrz Group | Nonerr/rr/returnz task | None) __future__rtypingrrGpaddle.distributed distributedr=r paddle.baser&paddle.distributed.communication.grouprr'paddle.distributed.communication.reducer r collections.abcr r paddle.base.corer rrrr&rBSUMr#rJr rrWs# !#M(%<A 2&JLL! S S 4S  S   S  S  S S r||   Gr