ϑi*+SSKrSSKJr SSKJr SSKJr SSKJr SSK J r /r "SS5r "S S \RS 9r"S S \RS 9rg)N)base)Executor RoleMakerBase)SGD)OptimizerWithMixedPrecisionc$\rSrSrSrSrSrSrSrg)ModezQ There are various mode for fleet, each of them is designed for different model. N) __name__ __module__ __qualname____firstlineno____doc__ TRANSPILERPSLIB COLLECTIVE__static_attributes__rf/var/www/html/banglarbhumi/venv/lib/python3.13/site-packages/paddle/incubate/distributed/fleet/base.pyr r sJ EJrr c\rSrSrSrSrSrSrSrSr SSjr S r S r SS jr S rS rSrSSjrSrSr\R*S5r\R*SSj5r\R*S5r\R*S5r\R*SSj5r\R*SSj5r\R*SSj5rSrg)Fleet$z Fleet is the base class, transpiler and pslib are implementation of Fleet. Args: mode(Mode): the implementation of Fleet's mode. Returns: None cHSUlXlSUlSUlSUlg)NF)_is_initialized_mode _optimizer _role_maker _executor)selfmodes r__init__Fleet.__init__/s%$ rc6URR5$)z Check whether the node is the first instance of worker. Returns: bool: True if this is the first node of worker, False if not. )r"is_first_workerr$s rr)Fleet.is_first_worker6s//11rc6URR5$)z6 Get current worker index. Returns: int: node id )r" worker_indexr*s rr-Fleet.worker_index@,,..rc6URR5$)zD Get current total worker number. Returns: int: worker numbers )r" worker_numr*s rr1Fleet.worker_numIs**,,rc6URR5$)z Check whether the node is an instance of worker. Returns: bool: True if this is a node of worker, False if not. )r" is_workerr*s rr4Fleet.is_workerR))++rcU(a)SRURR55$URR5$zy Get current server endpoints, such as ["127.0.0.1:1001", "127.0.0.1:1002"]. Returns: list/string: server endpoints ,)joinr"get_trainer_endpointsr$ to_strings rworker_endpointsFleet.worker_endpoints\: 88D,,BBDE E##99; ;rcH[URR55$)zC Get current total worker number. Returns: int: server number )lenr"get_pserver_endpointsr*s r server_numFleet.server_numis4##99;<?M  AY E!((rNc [[R"55UlU(a=[ U[ 5(d(SSKJn U(a[ X5(d [S5eXlURR5 SUl g)z should be called only once in user's python scripts, init() will initialize RoleMaker which is used for identifying current node's role, e.g. worker, server, etc. Args: role_maker(RoleMakerBase): subclass of RoleMakerBase. Returns: None rrz/role_maker must be an instance of RoleMakerBaseTN) rrCPUPlacer#rSr,paddle.incubate.distributed.fleet.role_makerrUr" generate_roler)r$ role_makerRoleMakerBaseIncubates rinit Fleet.initsk"$--/2 j]CC *Z"O"OE& &&(#rc:URRX5 g)z all reduce between workers, only support array of one dim. Args: input(list|numpy.array): array of one dim output(list|numpy.array): array of one dim N)r"all_reduce_worker)r$inputoutputs rrkFleet.all_reduce_workers **59rc8URR5 g)z barrier between workers N)r"barrier_workerr*s rrpFleet.barrier_workers '')rcgNrr*s r init_workerFleet.init_worker rc grsr)r$ model_dirkwargss r init_serverFleet.init_serverrvrcgrsrr*s r run_serverFleet.run_serverrvrcgrsrr*s r stop_workerFleet.stop_workerrvrcgrsrr$ optimizerstrategys rdistributed_optimizerFleet.distributed_optimizerrvrcgrsr)r$executordirnamefeeded_var_names target_vars main_programexport_for_deployment legacy_formats rsave_inference_modelFleet.save_inference_models rcgrsr)r$rrrs rsave_persistablesFleet.save_persistables rvr)r#rr r!r")Frs)NTF)rrrrrr&r)r-r1r4r>rDrGrJrMrPr`rhrkrpabcabstractmethodrtrzr}rrrrrrrrrr$s2/-, <=/ <,)!)F$8:*                 "       rr) metaclassc\rSrSrSrS Sjr\RS Sj5r\RS5r \RS Sj5r Sr g) DistributedOptimizeria DistributedOptimizer is a wrapper for paddle.base.optimizer A user should pass a paddle.base.optimizer to DistributedOptimizer minimize() function is implemented. DistributedOptimizer is the starting point for a user who wants to run distributed training. The optimized information will be stored in Fleet() instance who holds the global information about current distributed training. Args: optimizer(Optimizer): subclass of Optimizer. strategy(any): the user define config for Optimizer. Returns: None Nc[U[R5(d [U[5(d [ S5eXlX lg)Nz*optimizer must be an instance of Optimizer)rSr __bases__rrUr! _strategyrs rr&DistributedOptimizer.__init__$s=)S]]33J 2= = HI I#!rcg)ai First part of `minimize`, do auto-diff to append backward ops for the current program. Args: loss (Variable): loss variable to run optimizations. startup_program (Program): startup_program for initializing parameters in `parameter_list`. parameter_list (list): list of Variables to update. no_grad_set (set|None): set of Variables should be ignored. callbacks (list|None): list of callables to run when appending backward operator for one parameter. Return: list: list of (param, grad) pair, grad is the output of backward. Examples: See examples in `apply_gradients`. Nr)r$lossstartup_programparameter_list no_grad_set callbackss rbackwardDistributedOptimizer.backward-s8 rcg)ak Second part of `minimize`, appending optimization operators for given `params_grads` pairs. Args: params_grads (list): list of (param, grad) pair to do optimization. Returns: list: A list of operators appended to the current program. Examples: .. code-block:: python >>> # doctest: +SKIP('The network is not defined.') >>> loss = network() >>> optimizer = base.optimizer.SGD(learning_rate=0.1) >>> params_grads = optimizer.backward(loss) >>> # you may append operations for params_grads here >>> # ... >>> optimizer.apply_gradients(params_grads) Nr)r$ params_gradss rapply_gradients$DistributedOptimizer.apply_gradientsKs. rcg)a Add operations to minimize `loss` by updating `parameter_list`. This method combines interface `backward()` and `apply_gradients()` into one. Args: losses (Variable|Variable List): loss variable to run optimizations. scopes (Scope| Scope List): scope instance. startup_programs (Program|Program List): startup_program for initializing parameters in `parameter_list`. parameter_list (list): list of Variables to update. no_grad_set (set|None): set of Variables should be ignored. Returns: tuple: (optimize_ops, params_grads) which are, list of operators appended; and list of (param, grad) Variables pair for optimization. Nr)r$lossesscopesstartup_programsrrs rminimizeDistributedOptimizer.minimizeds6 r)r!rrs)NNNN) rrrrrr&rrrrrrrrrrrsv$"    :   0    rr)rpaddlerpaddle.base.executorr(paddle.distributed.fleet.base.role_makerrpaddle.optimizerrpaddle.static.amp.decoratorr__all__r ABCMetarrrrrrsM )B C j ckkj Zn S[[n r