ΑiD"SSKrSSKrSSKJr SSKJr SSKrSSKJr SSK J r \ "\ "\R"SS555r\ "\ "\R"S S555r\R "\\R$S S 9rS rS#S jrS$SjrSrSrSrSrSrSr"SS5r"SS\5r"SS5r"SS5r Sr!Sr""SS \RFRH5r%"S!S"\RLRN5r(g)%N)deque)Enum) log_helper) CUDAGraph+PADDLE_DEBUG_ENABLE_CUDAGRAPH_LAYER_LOGGING01PADDLE_DEBUG_CUDAGRAPHEDLAYER_FALLBACK_TO_DEFAULTz[%(levelname)s] %(message)s)fmtcF[(dg[RU5 gN)enable_debug_printloggerinfoxs e/var/www/html/banglarbhumi/venv/lib/python3.13/site-packages/paddle/device/cuda/cuda_graphed_layer.py debug_printr&s   KKNc/nU(aURU5 UcSnUc[US35 g[U[R5(aU(a<URSUR 35 URSUR 35 U(a+URS[UR5535 U(aURSU"U535 [SRU55 gg)NcX[URS5S-R55$)Nfloat32i)floatastypesum)ts rprint_tensor..8s  3d :??ABrz is Nonezshape = zplace = zptr = zhash = z | ) appendr isinstancepaddleTensorshapeplacehexdata_ptrjoin)rname print_meta print_ptr print_hashhashoutputs r print_tensorr.,sF  d |BytfH%& Av}} % %  MMHQWWI. / MMHQWWI. /  MMF3qzz|#4"56 7  MMGDG9- .EJJv&' &rcr[(dg[URSS55 [[U5 g)Nd-)rrcenterrecursive_applyr.)rbanners rprinterr5Gs(   c3'(L!$rc l^[U[5(aUVs/sHn[TU5PM sn$[U[5(a[U4SjU55$[U[5(a0UR 5VVs0sHup4U[TU5_M snn$T"U5$s snfs snnf)Nc3<># UHn[TU5v M g7fr r3).0itemfunctions r "recursive_apply..TsK_Xt44s)r listr3tupledictitems)r; input_varr:keyvalues` rr3r3Ps)T""[U[R5(aTRU5 ggr )r r!r"r)argrets rr!recursive_flatten..appendls& c6== ) ) JJsO *rr8)targetrrNs @rrecursive_flattenrQis CF# Jrc`/[U5Q[[UR555Q$r )rQr?values)argskwargss rrecursive_flatten_args_kwargsrV{s1  4   51 2 rc"[[U5$r )r3rJrs rrrs ?=!4rc~[U[R5(aUR(agUR$g)zWReturns the gradient of a Paddle Tensor if it's a tensor; otherwise, returns the input.N)r r!r"rGgradrs rget_grad_tensorrZs)!V]]## ??66M rc8\rSrSrSrSrSrSrSrSr Sr g ) CUDAGraphWithStaticInputOutputcXl[5UlSUlSUlSUlSUlSUlSUlg)NF) num_warmup_stepsrgraph has_recordedhas_preserved_inputs args_static kwargs_static inputs_staticoutputs_staticselfr_s r__init__'CUDAGraphWithStaticInputOutput.__init__sC 0[ !$)!!""rcUR(d9SUlXlX l[URUR5Ulg[X5n[ URU5HupEUR US5 M g)a For the CUDA Graph, it is crucial that the buffer remains address-stable, meaning that the buffer addresses for any inputs to the CUDA Graph should not change. One solution to achieve this is to preserve all input tensors. This function attempts to recursively flatten the input arguments and keyword arguments to identify all tensors passed to the layer (though it may still miss some due to other implicit ways inputs can be passed to a layer). It then preserves references to these input tensors as `self.inputs_static` so that the buffer pointers can be reused later. When this method is called subsequently, it copies the values back to the preserved input tensors to ensure the buffers are reused. TN)rbrcrdrVrezipcopy_)rhrTrUinputsx_staticrs rpreserve_or_copy/CUDAGraphWithStaticInputOutput.preserve_or_copysr(((,D %# !' !>  $"4"4"D 34@F"4#5#5v> q$' ?rc>URX#5 URR5 U"UR0URD6UlURR 5 [S5 URR5 SUl UR $)NzF[CUDAGraph] Record-Replay Start (Graph is replayed for the first time)T) rpr` capture_beginrcrdrf capture_endrreplayra)rhfrTrUs rrecord%CUDAGraphWithStaticInputOutput.records d+   "!1!1HT5G5GH   T   """rcXlgr )rf)rhrfs rset_output_static0CUDAGraphWithStaticInputOutput.set_output_statics,rcUR(d [S5eURX5 [S5 URR 5 UR $)NzGraph should be recorded firstz[CUDAGraph] Replay Start)ra RuntimeErrorrprr`rurf)rhrTrUs rru%CUDAGraphWithStaticInputOutput.replaysJ  ?@ @ d+./ """rcl[R"SU35 URRU5 g)Nzsave graph to )loggingrr`print_to_dot_files)rhr(s rsave#CUDAGraphWithStaticInputOutput.saves' ~dV,- %%d+r)rcr`rbrarerdr_rfN) __name__ __module__ __qualname____firstlineno__rirprwrzrur__static_attributes__rrr\r\s  #(4 #-#,rr\c$\rSrSrSrSrSrSrSrg)CUDAGraphLayerStatusz3Enum to represent the status of a CUDA Graph Layer.rrN) rrrr__doc__WARMUPRECORD CUDAGRAPHrrrrrrs= F FIrrc,\rSrSrSrSrSrSrSrg)CUDAGraphForwardBackwardcn[U5Ul[U5Ul[RUlgr )r\ forward_graphbackward_graphrrstatusrgs rri!CUDAGraphForwardBackward.__init__s+;r?rlrG)ysdysydyvdvs rselect_y_with_gradr,s EA"fmm$$F2 Bu & &\EA!V]]++Q___2" a ' 'Jr6==,I,II I 5LrcFUup/n[X5HupEUR(d[URc1UR[R "UR 55 MTURUR5 MqURS5 M [U5$r )rlrGrYrr!zerosr#r?)rn grad_inputsdetached_grad_inputs args_gradr detached_xs r get_args_gradr<s(.%KI[? &  j.>.>!?@  1   T "@  rc8\rSrSrSr\S5r\S5rSrg)_CUDAGraphedLayeriNz A custom layer that integrates CUDA Graph recording and execution into PaddlePaddle's autograd system. It handles forward and backward operations differently based on the CUDA graph layer status. c^UupE[U5n[U5n[XE5nX64n[US5 TR5(d [(a_[ S5 [ R"5 TR"U0UD6nSSS5 TR[RSUW45 OTR5n U R5(a`[ S[U 535 U4Sjn U RR "U /UQ70UD6nTR[R"XU45 OV[ S[U 535 U RR$"U0UD6nTR[R&U SU45 [ S5 UR)T5 [US5 [U5$!,(df  GNA=f) z Handles the forward pass of the layer. It operates differently based on the context's status: warmup, recording, or CUDA graph step. z Forward inputz"[CUDAGraph] Forward Step (Default)Nz%[CUDAGraph] Forward Step (Record) id c>[R"5 TR"U0UD6sSSS5 $!,(df  g=fr )r! enable_gradr)rTrUcontexts rforward*_CUDAGraphedLayer.forward..forwardss,++-&}}d=f=.--s 3 Az$[CUDAGraph] Forward Step (Graph) id z[CUDAGraph] Forward Step EndzForward output)rFrVr5r*debug_cudagraphedlayer_fallback_to_defaultrr!rrrrrrridrrwrrursave_for_backward) ctxr arg_tuplerrTrUrrnrr`rs ` rr_CUDAGraphedLayer.forwardTs ! d|''..wHHH!!)00%CB2e9+NO''..??!!)33UD!D 23 g&#$ay?&%s 6G Gc4UR5unUR5up4pV[Xa5upx[Xx4S5 U[R :Xa8[ S5 URU5 [U5n UR5 OU[R:Xa~[ S[U535 Sn URRXU5 [U5n URRU 5 UR5 UR!U5 OcU[R":XaD[ S[U535 URR%Xx5n UR!U5 O ['S5e[ S5 [U S5 U $) z Handles the backward pass of the layer. Similar to forward, it handles backward based on the context's status: warmup, record, or CUDAGraph. zBackward inputz#[CUDAGraph] Backward Step (Default)z&[CUDAGraph] Backward Step (Record) id c&URU5 gr )backward)rrs rr,_CUDAGraphedLayer.backward..backwards  2rz%[CUDAGraph] Backward Step (Graph) id zUnknown cuda graph statusz[CUDAGraph] Backward Step EndzBackward output) saved_tensorrrr5rrrrrrrrrrwrzrtrrrur}) rrrrr`rnrrrrrs rr_CUDAGraphedLayer.backwardsd %%' &-&6&6&8#"2+)* )00 0 = > JJrN%f-I    ! +22 2 @E L M   ' 'R 8&f-I  2 29 =        & +55 5 ?5 {K L,,33A:I    &:; ;34 ,-rrN) rrrrr staticmethodrrrrrrrrNs0 33j00rrcr^\rSrSrSrS S\R R4U4SjjjrSr Sr Sr Sr U=r $) CUDAGraphedLayeria CUDAGraphedLayer: A PaddlePaddle Layer to convert an eager mode model to utilize CUDA Graphs. CUDA Graphs provide a way to capture kernel-level operations of a model and play them back efficiently, allowing for potential speedups in repetitive computations, such as those during training iterations. This layer is a wrapper that enables the usage of CUDA Graphs with PaddlePaddle models. Overview: - The layer encapsulates another layer (the model to be converted). - During the first few (num_warmup_steps) iterations, the layer operates in eager mode without any CUDA Graphs. - After the warmup steps, the layer captures the forward and backward computations and replays them using CUDA Graphs in subsequent iterations. Usage: model = Model() graphed_model = CUDAGraphedLayer(model) Parameters: - layer (paddle.nn.Layer): The PaddlePaddle model/layer to be converted. - num_warmup_steps (int): The number of iterations before the CUDA Graph capture begins. Default is 3. Notes: - Restrictions: * CPU-GPU Synchronization: Operations that synchronize the CPU with the GPU, like device to host transfers, are not allowed. * CPU Work: Any operations on the CPU within the captured graph are not recorded. * Memory Address (Pointer) Consistency: Replays consistently read from and write to identical virtual memory addresses. * Dynamic Operations: - Control Flow: Dynamic control flows, especially those based on CPU data like if/else statements, are prohibited. - Tensor Shapes: Dynamic tensor shapes are not supported. - Allowed Operations: * CUDA RNG Operations: CUDA-based Random Number Generation operations are allowed. rc>[TU]5 [X5UlUR S[ U5R 3U5 g)NzGraphed )superrirr add_sublayertyper)rhrr_ __class__s rriCUDAGraphedLayer.__init__s< '@  HT%[%9%9$:;UCrc^[X5n[R"URX4/UQ76$r )rVrapplyr)rhrTrUrs rrCUDAGraphedLayer.forwards23DA  && LL4. +6  rc6URR5$r )rrrs rrCUDAGraphedLayer.is_warmup_steps||**,,rc6URR5$r )rrrs rr#CUDAGraphedLayer.is_cuda_graph_steps||..00r)r)r)rrrrrr!nnLayerrirrrr __classcell__)rs@rrrs9#JDfiiooDD  -11rr)UnnamedTFTN)r5))ros collectionsrenumrr! paddle.basergraphsrboolintgetenvrr get_loggerrINFOrrr.r5r3rJrQrVrFrZr\rrrrrautogradPyLayerrrrrrrrr s!  " ?EF.2 EsKL.*    gll =    (6% #$ 5G,G,V4 = = 6=6=r  $m//m`61vyy61r