pj"SSKJr SSKJrJr \(aSSKJr SSKrSSKJ r SSKrSSK J r SSK Jr SSjr"SS 5r"S S 5rS rS rSSSSSSSS.SSjjrg)) annotations) TYPE_CHECKINGCallable)IterableNFullyShardAuto FullyShardc`[[RRRSS5$)N_in_auto_parallel_F)getattrpaddlebase framework global_var_/var/www/html/pdf-tiff/venv/lib/python3.13/site-packages/paddle/distributed/fsdp/fully_shard.pyin_auto_parallel_moders'  ((*> rcN\rSrSr%SrS\S'SrS\S'SrS\S'SrS\S 'S r g) MixedPrecisionPolicy%Nzpaddle.dtype | None param_dtype reduce_dtype output_dtypeTboolcast_forward_inputsr) __name__ __module__ __qualname____firstlineno__r__annotations__rrr__static_attributes__rrrrr%s.'+K$+(,L%,(,L%, $$rrc$\rSrSr%SrS\S'Srg) OffloadPolicy-Tr pin_memoryrN)rrr r!r'r"r#rrrr%r%-s Jrr%c[U5$)Nr modulemeshreshard_after_forwardshard_placement_fn mp_policyoffload_policyignored_params enable_tensor_fusion_and_overlaps r_fully_shard_manual_parallelr21s f rc[UUUS9 U$)N)r1rr)s r_fully_shard_auto_parallelr4>s )I MrT)r+r,r-r.r/r0r1cUc [5nUc [5nU(a [U5O [5nUUUUUUUU4n [5(a[ U 6$[ U 6$)aV Apply fully sharded data parallel (FSDP) to the given module. This function wraps the input module with fully sharded data parallelism, which shards model parameters, gradients, and optimizer states across multiple devices. It supports both auto_parallel mode and manual_parallel mode. Args: module (Layer): The neural network module to be wrapped with fully sharded data parallelism. mesh (dist.ProcessMesh, optional): The process mesh defining the device arrangement for sharding. Defaults to None, which uses the default mesh. reshard_after_forward (bool | int | None, optional): Controls when to reshard the parameters after forward pass. If True or 1, reshard after each forward pass. If False or 0, keep sharded. If None, use default strategy. Defaults to None. shard_placement_fn (Callable[[paddle.Tensor], dist.Shard | None] | None, optional): A function that determines how each tensor should be sharded. Takes a tensor as input and returns a Shard placement or None. If None, uses default sharding strategy. Defaults to None. mp_policy (MixedPrecisionPolicy | None, optional): Mixed precision policy configuration. If None, creates a default MixedPrecisionPolicy. Defaults to None. offload_policy (OffloadPolicy | None, optional): Offload policy configuration for CPU offloading. If None, creates a default OffloadPolicy. Defaults to None. ignored_params (Iterable[paddle.Tensor] | None, optional): Parameters that should not be sharded. These parameters will be kept in full precision and not distributed. Defaults to None. enable_tensor_fusion_and_overlap (bool, optional): Whether to enable tensor fusion and communication/computation overlap optimization. When True, parameters are fused into contiguous buffers for more efficient communication and memory access, and enables prefetching and asynchronous communication to overlap with computation for better performance. Defaults to True. Returns: module: A wrapper module that applies FSDP to the input module. Examples: .. code-block:: python >>> # type: ignore >>> # doctest: +REQUIRES(env:DISTRIBUTED) >>> # python -m paddle.distributed.launch --device=0,1 train.py >>> import paddle >>> import paddle.distributed as dist >>> from paddle.distributed.fsdp import fully_shard >>> mesh = dist.ProcessMesh([0, 1], dim_names=["x"]) >>> model = paddle.nn.Linear(10, 10) >>> inputs = paddle.rand(shape=[10, 10]) >>> inputs = dist.shard_tensor(inputs, mesh, [dist.Shard(0)]) >>> opt = paddle.optimizer.AdamW(parameters=model.parameters()) >>> model = fully_shard(model, mesh) >>> tr_loss = model(inputs) >>> tr_loss.backward() >>> opt.step() >>> opt.clear_grad() )rr%setrr4r2) r*r+r,r-r.r/r0r1ignored_params_setargss r fully_shardr9PsuD(* &-N35  ( D)400+T22r)returnr)r*paddle.nn.Layerr+zdist.ProcessMeshr,zbool | int | Noner-z3Callable[[paddle.Tensor], dist.Shard | None] | Noner.zMixedPrecisionPolicy | Noner/zOffloadPolicy | Noner0zIterable[paddle.Tensor] | Noner1rr:r;) __future__rtypingrrcollections.abcrrpaddle.distributed distributeddist,paddle.distributed.auto_parallel.fully_shardrIpaddle.distributed.fleet.meta_parallel.sharding.group_sharded_fully_shardr rrr%r2r4r9rrrrDs#*(% G %% *"/3 -1+/59-1X3 X3 X3- X3  X3+X3)X33X3'+X3X3r