IЦi l%SSKrSSKrSSKrSSKJr SSKJr SSKJrJ r J r J r J r J r SSKrSSKJr SSKJr SS/r\"5(d:SSKr"S S 5rS r\\R.S l\\R.S lgSS KJr SSKJrJrJ r J!r!J"r"J#r#J$r$J%r%J&r&J'r'J(r(J)r) \RT"\+5r,\ (aSSK-J.r. "SS\Rb5r2\2"5r3\2\4S'SS\54Sjjr6"SS5rSS.S\5S\ \7S4S\ \ \5S4S\4Sjjrg!\/a \,RaS5 Nlf=f)N)reduce)chain)DictListOptionalTuple TYPE_CHECKINGUnion) is_available)not_noneinit_device_mesh DeviceMeshc\rSrSrSrg)_DeviceMeshStubN)__name__ __module__ __qualname____firstlineno____static_attributes__r\/var/www/html/ai-image-ml/venv/lib/python3.13/site-packages/torch/distributed/device_mesh.pyrrs rrcgNrrrr_init_device_mesh_stubrs rztorch.distributed.device_mesh)Backend) _find_pg_by_ranks_and_tag_get_default_group_get_group_tag get_backendget_process_group_ranksget_rankget_world_sizeinit_process_groupis_initialized new_group ProcessGroup split_group) ArrayLikezCDeviceMesh requires numpy >= 1.21 to be installed for type checkingc b\rSrSrSSjrSSjrSSS\\S 4S \\\ S 4SS4S jr SSSS \ \SS4S jjr SSjr SSS\ \ 4Sjr\S\S\ 4Sj5r\S\S\ 4Sj5rSSS \S\ 4SjrSS\ S\S\ \R(SS4SjjrS\\\ S 44SjrSSS \S\S4SjrSrg)_MeshEnvBreturnNcJ/Ul0Ul0Ul0Ul0Ulgr) mesh_stackchild_to_root_mappingmesh_dim_group_optionsroot_to_flatten_mappingflatten_name_to_root_dimsselfs r__init___MeshEnv.__init__Cs702DOGID &  'UWD (  *rrch[UR5S:Xa [S5eURS$)Nrz#No device mesh is currently active!)lenr0 RuntimeErrorr5s rget_current_mesh_MeshEnv.get_current_meshOs.4??#q("#HII??2& &r device_meshsubmesh_dim_names. submesh_dimscn^UVs/sHn[U4SjUS5PM nnTRn/n/nSn [X25Hup[U 5S:atUR U SU - U SU - S9nUR U SU - 5 U [U 5S- - n UR UR TU RS5 MUR U SU - 5 UR TRU S5 M [[UR55n UHn U RU 5 M UR"/U QUQ76R"S/UQ76nTR5nUH#n[TR UUSS9nUU;dM!UnM% UWlTUR"U'U$s snf)Nc@>UTRRU5-$r)meshsize)xyr?s r*_MeshEnv.create_sub_mesh..`s[%5%5%:%:1%=!=rrr:) start_dimend_dimFmesh_dim_names _init_backend)rrDzipr;flattenappendr3_dim_group_infoslistrangendimremovepermutereshaper#r device_typer1)r6r?r@rAmesh_dimslice_dim_size mesh_tensor slice_dim_idxslice_dim_group_infonum_dims_flattenmesh_dim_indices mesh_dim_namemesh_dims_remained_idxidxpg_ranks_by_dimcur_rankmesh_ndsubmesh res_submeshs ` rcreate_sub_mesh_MeshEnv.create_sub_meshTs% !-  !-H = !- &**KM#%  ! 36|3W/ '(1,"-"5"5"21"58H"H 0 47G G#6#K"(()9!)}MM&*ii 0@0@0E0E0G&H #$(y~~/B/B)C$D !';#%,,-@A(<(nn44&)=gb12 !))+H*!+))$1#3" &)7&+>G & &'9 :Tf ( ( 3 3Ir B= QGLMaGbD * *9 5m D% %w$s .I)3"I.cPURRUS5nU(dU$U$r)r1get)r6r?rys rrq_MeshEnv.get_root_meshs)2266{DII&/; >Y >rcURU5nURnU(a3U(a,[U5S:XdS5eUSnURX$5$g)z Returns the index of the mesh dim in the root mesh. The device_mesh passed in needs to be sliced out from the root mesh or submesh of the root mesh. rJz"The submesh can only be a 1D mesh.rN)rqrNr;get_mesh_dim_by_name)r6r?rychild_mesh_dim_nameschild_mesh_dim_names rget_root_mesh_dim_MeshEnv.get_root_mesh_dimsc **;7I#.#=#= 1,-28782&:1&=#00PPrrZc4[U5R5$r)_get_device_handle device_countrZs rnum_devices_per_host_MeshEnv.num_devices_per_hosts%k2??A ArcB[5[RU5-$r)r$r,rrs r num_hosts_MeshEnv.num_hostss"#x'D'D['QQ QrcURb[UR5S:Xa [S5eX!R;a[SUS3SUR35e[URR U55$)NrzNo `mesh_dim_names` found.zMesh dimension 'z' does not exist.z.Available mesh dimensions are: mesh_dim_names=)rNr;KeyErrorr rr)r6r?rbs rr_MeshEnv.get_mesh_dim_by_names**2{112a70$>$>>&}o5FGD[E_E_D`aK66<<]KL Lrr|backend pg_optionsc$X#4URU'gr)r2)r6r|rrs r_set_mesh_dim_group_options$_MeshEnv._set_mesh_dim_group_optionss 18/DD ' ' ,rc^ XRU5:wa [S5eURRU05 URUn/URQUQm [ U 4SjU55(d[ SUST S35eSn/nUHlnXc;aX6nUSnURU5 O-URRU5nURU45 X::a[ SUS3SUS 3S 5eUnMn U$) z Validate whether the mesh_dim_names is valid for slicing the given device_mesh. If valid, return dim indexes of the slice mesh in the device mesh. z'Cannot create a submesh from a submesh.c3.># UH nUT;v M g7frr).0rbvalid_mesh_dim_namess r 0_MeshEnv._get_slice_mesh_dims..2s%3M!55%3szInvalid mesh_dim_names z% specified. Valid mesh_dim_names are .r:z specified. z!Found mesh dim indices to slice: rnz.Mesh dim indices should be in ascending order.) rqr<r4rtrNallrrRrr) r6r?rNr4curr_idxslice_mesh_dimsrb mesh_indicesnext_idxrs @r_get_slice_mesh_dims_MeshEnv._get_slice_mesh_dimssR00=="#LMM  * * 5 5k2 F(,(F(F{(S %$++$*$ %3-n-=>00D/EQHH O!/  =#<#KL ,B/H#**<8*99?? NH#**H;7'"1.1AN;O;LBOH $!"0$# "rcpURX5nURRSU5RSURR U55nUR 5n/nUHGn[ URUU4SS9nXW;aURU/O/UlURU5 MI U$)zH Return all the submeshes of a given mesh dimension of the device mesh. r:FrM) rrDswapdimsrYrEr#rrZrSrR) r6r?rbr[rerf res_submeshesmesh_1drhs r_get_all_submeshes_MeshEnv._get_all_submeshesSs 00LH)..77HEMMK$$))(3O#++-HM*$++$1#3"'  *!11(;<( $$W-+! r)r1r4r2r0r3r.Nr.rr)r?rr.r)rrrrr7r=rstrrintrjrrrqr staticmethodrrr C10dBackendOptionsrrrrrrrr,r,Bs  ' M %M  %S#XM uS#X/ M   M `MQ@ &+@ &>> # xdoctest: +SKIP("no rank") >>> from torch.distributed.device_mesh import DeviceMesh >>> >>> # Initialize device mesh as (2, 4) to represent the topology >>> # of cross-host(dim 0), and within-host (dim 1). >>> mesh = DeviceMesh(device_type="cuda", mesh=[[0, 1, 2, 3],[4, 5, 6, 7]]) rZrD.rNNTrMr*rOr.cXl[U[R5(a(URR S:wa[ SU35e[U[R5(a+UR5R[RS9O#[R"US[RS9Ul U(a [U5OSUl [URR5R55UlSUlUS:waU(a UR%5 UR'5 [)5(a([+5S:Xa[,R."5UlUR[15:HR35nUR5S5S;deUR5S5S:aUSR5OSUlgg) Ncpuz!`mesh` must be a CPU tensor, got dtypedevicerxlathreadedr)rrJ)rZ isinstancerTensorrtype ValueErrordetachtortensorrDrxrNrQtolist_flatten_mesh_list _thread_id_get_or_create_default_group_init_process_groupsr&r! threading get_identr#nonzerorE_coordinate_on_dim)r6rZrDrNrO rank_coordss rr7DeviceMesh.__init__sx + $ --$++2B2Be2K #DTF!KLLdELL11   uyy 1\\$uEIIF I &R%,k2F1GqIYIYHZZce((6J)JK%' 'rc /n[5nURRS:XaURR5[ 5:Xa[ [ [ 555n[RR5(a[U5S:Xa [SUSS9OUnUR[U5UUR45 GO[ URR5GHjnURRSU5R!SURR#U55nU[$R&;a[$R&UunnOSupxUR((aSUR(U3OS U3n Sn[+US S5=n b[-UUUR/5U S 9nUHn U R/5n U c [U UUU S 9nUR15U ;dM8[3U5U:a [5S UR15SU S35eUR[55U UR45 M GMm Xlg)NrJgloozcpu:gloo,cuda:nccl mesh_default)rranks group_descr:)NNmesh_ mesh_dim_bound_device_id) parent_pgr split_ranksr)rrrrzFEach device mesh dimension should get only one process group, but got z in !)rrDrVrr$rTrUrcudar r!r'rRr group_namerrYrErpr2rNrr)rr#r;r<r rS) r6dim_group_infos default_groupr dim_groupr|rerrrrdim_meshsubgroup_rankss rrDeviceMesh._init_process_groupssoACO.0Myy~~"tyy'8N#345zz..00#M2f<  4##1' &&&y1!,,!0C'+ii&8&8S&A&I&IDIINN3/'O oDDD,BB3G#&/9+ ..  3 3C 89:(.!%I+2)+(>(@'1 % %4)1): +2(1&4(/+5+5 )I ==?n<"?3c9&2&lmqmzmzm|l}~**8)9%<'"!",22$28I3F$G$2$-$8$8!"+%4k1b%4 !rcD[RRU5 U$r)rpr0rRr5s r __enter__DeviceMesh.__enter__bs  & & - -d 3Krc@[RR5 gr)rpr0pop)r6exc_type exc_value exc_tracebacks r__exit__DeviceMesh.__exit__hs  & & * * ,rcUR(d-SURSURR5S3nU$SURSURR5SURS3nU$)Nz DeviceMesh('z', )z, mesh_dim_names=)rNrZrDr)r6device_mesh_reprs r__repr__DeviceMesh.__repr__ls**t//0DII4D4D4F3GqI  $ #$D$4$4#5S9I9I9K8LL]^b^q^q]rrst  $ #rc[USS5UlUR(dQ[URURR UR URUR45UlUR$)N_hash) rrhashrrDshaperZrNrr5s r__hash__DeviceMesh.__hash__tsc w5DJ::!// ((++  :: rotherc[U[5(dg[U5[U5:XagURUR:H=(a URR URR :H=(aY UR UR :H=(a9 URUR:H=(a URUR:H$)NFT) rridrrDrrZrNr)r6rs r__eq__DeviceMesh.__eq__seZ00$x2e9$++u/G/GG< 5::+;+;;<((E,=,==<++u/C/CC<5+;+;; rcpUR(d [S5e[U[5(aU4OUnXR:XaU$[R X5n[ RRR5 [RXU5nSSS5 U$!,(df  W$=f)a Slice the current DeviceMesh based on the mesh_dim_names given to create a submesh. The submesh created consists of the dimensions and the communicators indicated by ``mesh_dim_names`` Args: mesh_dim_names (Union[str, Tuple[str]]): the name or the tuple of names of the mesh dimension of the DeviceMesh to create the submesh for. Returns: A :class:`DeviceMesh` object The following program runs on each process/rank in an SPMD manner in a world size of 8. In the first example: Calling mesh_2d["tp"] on rank 0, 1, 2, 3 returns a 1D submesh of DeviceMesh:([0, 1, 2, 3]). Calling mesh_2d["tp"] on rank 4, 5, 6, 7 returns a 1D submesh of DeviceMesh:([4, 5, 6, 7]). Calling mesh_2d["dp"] on rank 0, 4 returns a 1D submesh of DeviceMesh:([0, 4]). Calling mesh_2d["dp"] on rank 1, 5 returns a 1D submesh of DeviceMesh:([1, 5]). Calling mesh_2d["dp"] on rank 2, 6 returns a 1D submesh of DeviceMesh:([2, 6]). Calling mesh_2d["dp"] on rank 3, 7 returns a 1D submesh of DeviceMesh:([3, 7]). In the second example: Calling mesh_3d["dp", "cp"] on rank 0, 1, 4, 5 returns a 2D submesh of DeviceMesh:([[0, 1], [4, 5]]). Calling mesh_3d["dp", "cp"] on rank 2, 3, 6, 7 returns a 2D submesh of DeviceMesh:([[2, 3], [6, 7]]). Calling mesh_3d["cp", "dp"] on rank 0, 1, 4, 5 returns a 2D submesh of DeviceMesh:([[0, 4], [1, 5]]). Calling mesh_3d["cp", "dp"] on rank 2, 3, 6, 7 returns a 2D submesh of DeviceMesh:([[2, 6], [3, 7]]). Example:: >>> # xdoctest: +SKIP("no rank") >>> from torch.distributed.device_mesh import DeviceMesh >>> >>> # Initialize a 2D device mesh as (2, 4) to represent the topology >>> # of cross-host(dim 0), and within-host (dim 1). >>> mesh_2d = init_device_mesh(device_type="cuda", (2,4), mesh_dim_names=("dp", "tp")) >>> tp_mesh = mesh_2d["tp"] >>> dp_mesh = mesh_2d["dp"] >>> >>> # Initialize a 3D mesh. >>> mesh_3d = init_device_mesh(device_type="cuda", (2,2,2), mesh_dim_names=("dp", "pp", "cp")) >>> # The order of the mesh_dim_names provided deteremines the order of dimensions in the submesh. >>> dp_cp_mesh = mesh_3d["dp", "cp"] >>> cp_dp_mesh = mesh_3d["cp", "dp"] z1Cannot slice a DeviceMesh without mesh_dim_names!N) rNr<rrrprr _subclasses fake_tensorunset_fake_temporarilyrj)r6rNrrhs r __getitem__DeviceMesh.__getitem__sZ&&"#VWW&0%D%D!. !4!44 "1"F"F#&&22IIK-==oGL LKs B&& B5r[c[US5(d [S5eURRS:a(Uc%[SURRS3SS5eURRS:Xa$Uc![ [ UR S SS 65$[RU5n[RRUS5nU(a8XR5;a%X1R S SS n[ [ U65$[U[5(a[RX5OUn[ [ UR USS 65$) ab Returns the single ProcessGroup specified by mesh_dim, or, if mesh_dim is not specified and the DeviceMesh is 1-dimensional, returns the only ProcessGroup in the mesh. Args: mesh_dim (str/int, optional): it can be the name of the mesh dimension or the index of the mesh dimension. Default is None. Returns: A :class:`ProcessGroup` object. rSz*DeviceMesh process groups not initialized!rJNFound the DeviceMesh have dimensionsJOptional kwarg `mesh_dim` needs to be specified when device_mesh.ndim > 1.zmIf you want to get the list of all the ProcessGroups in the DeviceMesh,please use `get_all_groups()` instead.r)hasattrr<rDrVr rrSrprqr3rrurrr)r6r[ryr3rs r get_groupDeviceMesh.get_groups[4!344"#OPPyy~~!h&6"00@ L`=yy~~"x'7-t/D/DQ/G/KL(55d;I&5&M&M&Q&Q4' #'87S7S7U+U"9"C"T"TUV"WXZYZ"[ 9? KLL"(C00$88H!  -t/D/DX/NrPQ/RSrc[URR5Vs/sHoRU5PM sn$s snf)zp Returns a list of ProcessGroups for all mesh dimensions. Returns: A list of :class:`ProcessGroup` object. )rUrDrVr)r6is rget_all_groupsDeviceMesh.get_all_groupss205TYY^^/DE/D!NN1%/DE EEs?rogroupc"[U[5(a[U5n[U[R5(aUR 5U:wd'Ub>[U[R5(dX$:wa[ S[U5SU35e[R"US[RS9n[UUUSS9n[U5X@R4/Ul U$[U5n[U5S:Xa [ S5eUc [ S 5e[U[R5(a,UR!5R#[RSS 9O#[R"US[RS9nUR$[U5:wa)[ S UR 5S [U5S 35e[XUSS9nUVs/sH$n[U5[U5UR4PM& snUl U$s snf)a Constructs a :class:`DeviceMesh` with ``device_type`` from an existing :class:`ProcessGroup`. The constructed device mesh has number of dimensions equal to the number of groups passed. If more than one group is passed, then the ``mesh`` argument is required. z Invalid mesh z for ProcessGroup with ranks rrFrMrz.Expects at least one ProcessGroup to be passedz0Must pass mesh if passing multiple ProcessGroups)rrzEExpects mesh with ndim equal to number of ProcessGroups but got mesh z and z ProcessGroups)rr(r"rrrrrrrrr rrSrTr;rrrV)rrZrDrN group_ranksr?groupss r from_groupDeviceMesh.from_groups %..5e< tU\\22t{{} 7S$&tU\\::+$'D {2OP[}]||KUYYO(#1"'  $E*K9I9IJ0 ,#"%[F6{a !QRR| !STTdELL11   uyy ?\\$uEIIF  yyCK'  KKM?%F }NL%.PUK$ , $E #5)+E2$$ $ ,K ( ,s+H crUcURR5$URRU5$r)rDrrE)r6r[s rrEDeviceMesh.sizeTs*(0(8499??$ VdiinnX>V Vrc.URR$r)rDrVr5s rrVDeviceMesh.ndimWs99>> !rc@[URR5$r)rxrDrr5s rrDeviceMesh.shape[s) )rc[5$)z" Returns the current global rank. )r#r5s rr#DeviceMesh.get_rank_s : rcURS:a'Uc$[SURRS3S5eUcSn[UR U55n[ U[ 5(dS5e[[U55$)as Returns the local rank of the given mesh_dim of the DeviceMesh. Args: mesh_dim (str/int, optional): it can be the name of the mesh dimension or the index of the mesh dimension. Default is None. Returns: An integer denotes the local rank. The following program runs on each process/rank in an SPMD manner. In this example, we have 2 hosts with 4 GPUs each. Calling mesh_2d.get_local_rank(mesh_dim=0) on rank 0, 1, 2, 3 would return 0. Calling mesh_2d.get_local_rank(mesh_dim=0) on rank 4, 5, 6, 7 would return 1. Calling mesh_2d.get_local_rank(mesh_dim=1) on rank 0, 4 would return 0. Calling mesh_2d.get_local_rank(mesh_dim=1) on rank 1, 5 would return 1. Calling mesh_2d.get_local_rank(mesh_dim=1) on rank 2, 6 would return 2. Calling mesh_2d.get_local_rank(mesh_dim=1) on rank 3, 7 would return 3. Example:: >>> # xdoctest: +SKIP("no rank") >>> from torch.distributed.device_mesh import DeviceMesh >>> >>> # Initialize device mesh as (2, 4) to represent the topology >>> # of cross-host(dim 0), and within-host (dim 1). >>> mesh = DeviceMesh(device_type="cuda", mesh=[[0, 1, 2, 3],[4, 5, 6, 7]]) rJrrrrz1We expect ProcessGroup before calling `get_rank`!)rVr<rDr rrr(r#)r6r[mesh_dim_groups rget_local_rankDeviceMesh.get_local_rankes8yy1}!1"00@ L`!%dnnX&>?N  CB CH^45 5rc@UR(a UR$S$)z Return the relative indices of this rank relative to all dimensions of the mesh. If this rank is not part of the mesh, return None. N)rr5s rget_coordinateDeviceMesh.get_coordinates /3.E.E4** O4 OrrbcdUR(d [S5e[RX5$)a Returns a 1D DeviceMesh by flattening the current DeviceMesh. If no mesh_dim_name is provided, the default is a string concatentaing the mesh_dim_names of the given submesh with each mesh_dim_name separated by "_". For example, if we have a 3D mesh DeviceMesh([[[0, 1], [2, 3]], [[4, 5], [6, 7]]], mesh_dim_names=("dp", "cp", "tp")), calling mesh_3d["dp", "cp"]._flatten() will create a 1D submesh DeviceMesh([0, 1, 2, 3], mesh_dim_names=("dp_cp",)) on rank 0, 1, 2, 3 and a 1D submesh DeviceMesh([4, 5, 6, 7], mesh_dim_names=("dp_cp",)) on rank 4, 5, 6, 7. After the flattened dimension is created, to access the flattened dimesnion in mesh_3d, one can use the existing slicing method to obtain the flattened mesh through calling mesh_3d["dp_cp"]. z3Cannot flatten a DeviceMesh without mesh_dim_names!)rNr<rpr)r6rbs r_flattenDeviceMesh._flattens0&&"I#66tK Kr)rrSrrrrZrDrNrrr)'rrrr__doc__r__annotations__rrrrr boolr7rrrrrrobjectr rrr(rrrrr#rEpropertyrVrr#r/r2r5rrrrrr{sg% Nll sCx119="& ( (  k12(  %U38_5 (   ( ( T (:r 4h   - $c $   4 G "'U38_(<"=G  G R, huS#X&?, <, \ FD$6 F @D@  9= @ tL'99:@ @ 5{!:;<@  %U38_5 @   @  @ D W# W# W  "# "  "  *5c? *  * c  ( 68E#s(O+D( 6PS( 6T PHT#Y$7 P L(3- L< L Lrro mesh_shape.rNr.c @Ubm[[U55[U5:wa[SSU35e[U5[U5:wa%[SS[U5S[U5S35eU(a%UR5(d[SUS 3S 5e[R "S 5 [R "[R"U5[RS 9RU5nSSS5 [UWUS 9nU$!,(df  N=f)a Initializes a `DeviceMesh` based on `device_type`, `mesh_shape`, and `mesh_dim_names` parameters. This creates a DeviceMesh with an n-dimensional array layout, where `n` is the length of `mesh_shape`. If `mesh_dim_names` is provided, each dimension is labeled as `mesh_dim_names[i]`. .. note:: `init_device_mesh` follows SPMD programming model, meaning the same PyTorch Python program runs on all processes/ranks in the cluster. Ensure `mesh_shape` (the dimensions of the nD array describing device layout) is identical across all ranks. Inconsistent `mesh_shape` may lead to hanging. .. note:: If no process group is found, init_device_mesh will initialize distributed process group/groups required for distributed communications behind the scene. Args: device_type (str): The device type of the mesh. Currently supports: "cpu", "cuda/cuda-like". Passing in a device type with a GPU index, such as "cuda:0", is not allowed. mesh_shape (Tuple[int]): A tuple defining the dimensions of the multi-dimensional array describing the layout of devices. mesh_dim_names (Tuple[str], optional): A tuple of mesh dimension names to assign to each dimension of the multi-dimensional array describing the layout of devices. Its length must match the length of `mesh_shape`. Each string in `mesh_dim_names` must be unique. Returns: DeviceMesh: A :class:`DeviceMesh` object representing the device layout. Example:: >>> # xdoctest: +SKIP("no rank") >>> from torch.distributed.device_mesh import init_device_mesh >>> >>> mesh_1d = init_device_mesh("cuda", mesh_shape=(8,)) >>> mesh_2d = init_device_mesh("cuda", mesh_shape=(2, 8), mesh_dim_names=("dp", "tp")) Nz"Each mesh_dim_name must be unique.z/Found repeated mesh_dim_name in mesh_dim_names z6mesh_shape and mesh_dim_names should have same length!zFound len(mesh_dim_names): z and len(mesh_shape):rz0Device type with index is not supported but got rnzUIf you maintained a 'torch.device' object, it's recommended to pass in 'device.type'.rr)rZrDrN) r;setr<isalpharrarangervrwrviewr)rZr<rNrDr?s rr r sR  %3~&'3~+>>"8EnEUV :#n"55"L1#n2E1FF[\_`j\k[llmn {2244B;-rRg \\% << * 5UYYGLLZXD! #) ! s 3AD D)r)8loggingrvr functoolsr itertoolsrtypingrrrrr r rtorch.distributedr torch.utils._typing_utilsr __all__sysrrmodulesrr torch._C._distributed_c10drr"torch.distributed.distributed_c10drrr r!r"r#r$r%r&r'r(r) getLoggerrlogger numpy.typingr* ImportErrorwarninglocalr,rpr8rrrrrrrSsX DD *. | ,~~   ?NCKK/0;0KK'B      x (F  . l!9??l!\ !) OX*11mLmLf59 GG#s(OG!sCx1 G  G[  NNU  sDD32D3