ΑiYSSKJr SSKJrJr SSKrSSKrSSKJr SSK J r SSK J r J r \(aSSKJr SS KJr SS KJr \"S 5rSSS jjrSSS jjrg)) annotations) TYPE_CHECKINGTypeVarN) framework)stream)convert_object_to_tensorconvert_tensor_to_object)Tensor)task)Group_Tc0[R"XX#5$)aJ Gather tensors from all participators and all get the result. As shown below, one process is started with a GPU and the data of this process is represented by its group rank. Through the all_gather operator, each GPU will have data from all GPUs. .. image:: https://githubraw.cdn.bcebos.com/PaddlePaddle/docs/develop/docs/api/paddle/distributed/img/allgather.png :width: 800 :alt: all_gather :align: center Args: tensor_list (list): A list of output Tensors. Every element in the list must be a Tensor whose data type should be float16, float32, float64, int32, int64, int8, uint8, bool, bfloat16, complex64 or complex128. tensor (Tensor): The Tensor to send. Its data type should be float16, float32, float64, int32, int64, int8, uint8, bool, bfloat16, complex64 or complex128. group (Group|None, optional): The group instance return by new_group or None for global default group. sync_op (bool, optional): Whether this op is a sync op. The default value is True. Returns: None. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env: DISTRIBUTED) >>> import paddle >>> import paddle.distributed as dist >>> dist.init_parallel_env() >>> tensor_list = [] # type: ignore >>> if dist.get_rank() == 0: ... data = paddle.to_tensor([[4, 5, 6], [4, 5, 6]]) >>> else: ... data = paddle.to_tensor([[1, 2, 3], [1, 2, 3]]) >>> dist.all_gather(tensor_list, data) >>> print(tensor_list) >>> # [[[4, 5, 6], [4, 5, 6]], [[1, 2, 3], [1, 2, 3]]] (2 GPUs) )r all_gather) tensor_listtensorgroupsync_ops k/var/www/html/banglarbhumi/venv/lib/python3.13/site-packages/paddle/distributed/communication/all_gather.pyrr&s\   [% AAc[R"5(dS5e[U5up4/n[XTU5 [ [ U5R 55nUR5n[R"Xv/5n[R"U5n/n [XU5 [U 5H"upUR[X5U 55 M$ g)a Gather picklable objects from all participators and all get the result. Similar to all_gather(), but python object can be passed in. Args: object_list (list): A list of output object. The datatype of every element in the list is same as the input obj. obj (Any): The picklable object to send. group (Group): The group instance return by new_group or None for global default group. Returns: None. Warning: This API only supports the dygraph mode. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env: DISTRIBUTED) >>> import paddle >>> import paddle.distributed as dist >>> dist.init_parallel_env() >>> object_list = [] # type: ignore >>> if dist.get_rank() == 0: ... obj = {"foo": [1, 2, 3]} >>> else: ... obj = {"bar": [4, 5, 6]} >>> dist.all_gather_object(object_list, obj) >>> print(object_list) >>> # [{'foo': [1, 2, 3]}, {'bar': [4, 5, 6]}] (2 GPUs) z4all_gather_object doesn't support static graph mode.N)rin_dynamic_moder rintmaxitemnumpynpresizepaddle to_tensor enumerateappendr ) object_listobjrr len_of_tensorlist_len_of_tensormax_len_of_tensor numpy_data input_tensorris rall_gather_objectr+WsF  $ $ & &> &5S9F!%8C 2388:;J:':;J##J/LK{%0{+  $V-B C ,r)NT) rz list[Tensor]rr rz Group | Nonerboolreturnz task | None)N)r#zlist[_T]r$rrr r-None) __future__rtypingrrrrrr paddle.distributed.communicationrserialization_utilsr r r paddle.base.corer &paddle.distributed.communication.groupr rrr+rrr6s#) 3 %< B  .B.B .B .B .B  .Bd48: : ": +0:  : r