ΑiGSSKrSSKrSSKrSSKrSSKJr \"SS5rSrSrSr Sr Sr S r S r S rS rS rSrSrSrg)N) get_loggerINFOrootcSnSnS/n[R"U5RS5R[R 5nSU3nUHDnUR U5S:dMUR5R5SnU"U5n U$ U$![a"n[RSU35 SnAgSnAff=f) z get cpu info from lscpu c /nURS5nUHbn[URS5S5[URS5S5pTUR[[ XES-555 Md U$)N,-rr)splitintextendlistrange)iprocessed_cpu_info cpu_ranges cpu_rangestartends f/var/www/html/banglarbhumi/venv/lib/python3.13/site-packages/paddle/distributed/utils/process_utils.py_process_raw_cpu_info,_get_cpu_info.._process_raw_cpu_info!svWWS\ #IIOOC(+,IOOC(+,  % %d5a+@&A B $ "!Nlscpuutf-8noder_get_cpu_info failed, reason:) subprocess check_outputdecoder oslinesepfindstrip Exceptionloggerwarning) numa_idrcpuscmdoutputnuma_keyline raw_cpu_infoes r _get_cpu_infor1s " i((-44W=CCBJJO'#Dyy"a'#zz|113A6 ,\:   6qc:;s$A$B-+BB C 'CC c.[R"S5$)z" check if nvidia-smi is available nvidia-smishutilwhichrr_has_nvidia_smir8<s << %%rc.[R"S5$)z check if xpu-smi is available xpu-smir4r7rr _has_xpu_smir;Cs << ""rc[UR55S:XagURS5n[U5U::agX!$)Nrr )lenr%r )str_device_list local_rankvisible_devicess r_get_xpu_device_from_envrAJsD ? "#q(%++C0O ?z)  &&rc[R"S5nUb [X5$[R"S5nUb [X 5$[R"S5nUb [X05$[U5$)z+ get currently used xpu physical device id XPULINK_VISIBLE_DEVICESXPU_VISIBLE_DEVICESCUDA_VISIBLE_DEVICES)r"getenvrAstr)r?xpulink_visible_devicesxpu_visible_devicescuda_visible_devicess r_get_xpu_devicerKSst !ii(AB*'(?LL))$9:&'(;HH99%;<''(<II z?rc[R"S5nUbUS:Xa [U5$URS5n[ U5U::agX$)z+ get currently used gpu physical device id rENr )r"rFrGr r=)r?rJs r_get_gpu_devicerNhsW99%;<#';r'A:/55c:  J.  ++rcSSSSU/n[R"USS9RS5nUR5R 5SnU$![ a"n[ RS U35 S nAg S nAff=f) z# get gpu numa info from nvidia-smi r3topoz-Cz-irtimeoutrrN)rr r!r%r r&r'r()gpu_idr+r,r)r0s r_get_gpu_numa_inforUusyVT48((a8??H,,.&&(, 6qc:;sA A A<A77A<c [U5n/SQn[R"S5S:Xa/SQn[R"USS9R S5nUR 5US-R5S n/nURS 5HFnURS 5upgUR[[U5[U5S-55 MH U$) N)r:rPz-mCUDA_DEVICE_ORDEROAM_ID)r:rPz-mo<rQrrr r ) r r"rFrr r! splitlinesr r r)xpu_idr+r, cpu_affinity affinity_maskaffinity_rangerrs r_get_xpu_affinity_maskr`s [F #C yy$%1)  $ $S" 5 < LM&,,S1#))#. U3u:s3x!|<=2 rc[5(d[RS5 [$[ [ [ R"SS55S5n[U5nUc[RS5 [$[U5nUc[RS5 [$US:Xa[RS5 [$[U5nUc[RS 5 [$[ R"S5n[RS U35 [ R"SU5 [ R"S5n[RS U35 [$) z set affinity for gpu zQnvidia-smi is not available, set_affinity is aborted, plz check your environment.PADDLE_LOCAL_RANK0rzgFailed to get device id from cuda_visible_devices, set_affinity is aborted, plz check your environment.zMFailed to get numa info, set_affinity is aborted, plz check your environment.zN/Aznvidia-smi topo return numa id as N/A, set_affinity is aborted, plz check your environment. (Notice: This is expected behavior when executed on single numa node environment)zLFailed to get cpu info, set_affinity is aborted, plz check your environment.Check affinity before setting: zcheck affinity after setting: )r8r'r( FAIL_CODEmaxr r"rFrNrUr1sched_getaffinityinfosched_setaffinity SUCCESS_CODE)r? device_idr)r^affinitys rset_affinity_gpurms5    _ S#6<=qAJ +I u  +G [ % | !'*M Z ##A&H KK1(<=M*##A&H KK0 ;< rc[5(d[RS5 [$[ [ [ R"SS55S5n[U5nUc[RS5 [$[U5n[ R"S5n[RSU35 [ R"SU5 [ R"S5n[RSU35 [$)z set affinity for xpu zNxpu-smi is not available, set_affinity is aborted, plz check your environment.rbrcrzMFailed to get device id, set_affinity is aborted, plz check your environment.rdzCheck affinity after setting: )r;r'r(rerfr r"rFrKr`rgrhrirj)r?rkr^rls rset_affinity_xpuros >> \ S#6<=qAJ +I [ *95M##A&H KK1(<=M*##A&H KK0 ;< rc[RR5(a [5$[RR 5(a [ 5$[ RS5 [$)Nz-Currently set_affinity only supports gpu env.) paddledeviceis_compiled_with_cudarmis_compiled_with_xpuror'r(rer7rr set_affinityrusO }}**,,!!  + + - -!! FGr)r"r5rrq"paddle.distributed.utils.log_utilsrr'rjrer1r8r;rArKrNrUr`rmrorur7rrrwse  9 FF # @&#'* ,   &R2r