ΑioSSKJr SSKJrJr \(aSSKJr SSKrSSKJ r SSKrSSK J r SSK Jr SSjr"SS 5r"S S 5rS rS rSSSSSSS.SSjjrg)) annotations) TYPE_CHECKINGCallable)IterableNFullyShardAuto FullyShardc`[[RRRSS5$)N_in_auto_parallel_F)getattrpaddlebase framework global_varc/var/www/html/banglarbhumi/venv/lib/python3.13/site-packages/paddle/distributed/fsdp/fully_shard.pyin_auto_parallel_moders'  ((*> rcN\rSrSr%SrS\S'SrS\S'SrS\S'SrS\S 'S r g) MixedPrecisionPolicy%Nzpaddle.dtype | None param_dtype reduce_dtype output_dtypeTboolcast_forward_inputsr) __name__ __module__ __qualname____firstlineno__r__annotations__rrr__static_attributes__rrrrr%s.'+K$+(,L%,(,L%, $$rrc$\rSrSr%SrS\S'Srg) OffloadPolicy-Tr pin_memoryrN)rrr r!r'r"r#rrrr%r%-s Jrr%c[U5$Nr modulemeshreshard_after_forwardshard_placement_fn mp_policyoffload_policyignored_paramss r_fully_shard_manual_parallelr21s f rc[X5 U$r)rr*s r_fully_shard_auto_parallelr4=s6 Mr)r,r-r.r/r0r1cUc [5nUc [5nU(a [U5O [5nUUUUUUU4n[5(a[ U6$[ U6$)a Apply fully sharded data parallel (FSDP) to the given module. This function wraps the input module with fully sharded data parallelism, which shards model parameters, gradients, and optimizer states across multiple devices. It supports both auto_parallel mode and manual_parallel mode. Args: module (Layer): The neural network module to be wrapped with fully sharded data parallelism. mesh (dist.ProcessMesh, optional): The process mesh defining the device arrangement for sharding. Defaults to None, which uses the default mesh. reshard_after_forward (bool | int | None, optional): Controls when to reshard the parameters after forward pass. If True or 1, reshard after each forward pass. If False or 0, keep sharded. If None, use default strategy. Defaults to None. shard_placement_fn (Callable[[paddle.Tensor], dist.Shard | None] | None, optional): A function that determines how each tensor should be sharded. Takes a tensor as input and returns a Shard placement or None. If None, uses default sharding strategy. Defaults to None. mp_policy (MixedPrecisionPolicy | None, optional): Mixed precision policy configuration. If None, creates a default MixedPrecisionPolicy. Defaults to None. offload_policy (OffloadPolicy | None, optional): Offload policy configuration for CPU offloading. If None, creates a default OffloadPolicy. Defaults to None. ignored_params (Iterable[paddle.Tensor] | None, optional): Parameters that should not be sharded. These parameters will be kept in full precision and not distributed. Defaults to None. Returns: module: A wrapper module that applies FSDP to the input module. Examples: .. code-block:: python >>> # type: ignore >>> # doctest: +REQUIRES(env:DISTRIBUTED) >>> # python -m paddle.distributed.launch --device=0,1 train.py >>> import paddle >>> import paddle.distributed as dist >>> from paddle.distributed.fsdp import fully_shard >>> mesh = dist.ProcessMesh([0, 1], dim_names=["x"]) >>> model = paddle.nn.Linear(10, 10) >>> inputs = paddle.rand(shape=[10, 10]) >>> inputs = dist.shard_tensor(inputs, mesh, [dist.Shard(0)]) >>> opt = paddle.optimizer.AdamW(parameters=model.parameters()) >>> model = fully_shard(model, mesh) >>> tr_loss = model(inputs) >>> tr_loss.backward() >>> opt.step() >>> opt.clear_grad() )rr%setrr4r2) r+r,r-r.r/r0r1ignored_params_setargss r fully_shardr9Jsrx(* &-N35   D)400+T22r)returnr)r+paddle.nn.Layerr,zdist.ProcessMeshr-zbool | int | Noner.z3Callable[[paddle.Tensor], dist.Shard | None] | Noner/zMixedPrecisionPolicy | Noner0zOffloadPolicy | Noner1zIterable[paddle.Tensor] | Noner:r;) __future__rtypingrrcollections.abcrrpaddle.distributed distributeddist,paddle.distributed.auto_parallel.fully_shardrIpaddle.distributed.fleet.meta_parallel.sharding.group_sharded_fully_shardr rrr%r2r4r9rrrrDs#*(% G %%   "/3 -1+/59Q3 Q3 Q3- Q3  Q3+Q3)Q33Q3Q3r