IЦiP %SSKrSSKrSSKrSSKrSSKJr SSKJr SSKJ r J r J r J r J r JrJr SSKrSSKJr SSKJr SSKJrJr SSKJr SS KJr SS KJrJrJrJ r J!r!J"r"J#r#J$r$ SS K%J&r&J'r'J(r(J)r)J*r*J+r+J,r, SS K-J.r.J/r/J0r0J1r1 SS K2J3r3 SSK4J5r5 SSK6J7r7 \Rp"\95r:\Rv\"SS\'5r?"SS\?5r@S-S\ \A\ 4S\S\BS\&4SjjrCS\ \&S\ \&4SjrDS\ \A\ 4S \BS\)4S!jrES-S\ \)S"\BS\\ \)\44S#jjrFS\!S\4S$jrGS%\S&\S\B4S'jrHS(\RS)\S\B4S*jrJS+\ \)S\S\B4S,jrKg).N)ChainMap)reduce)AnycastDictListOptionalTupleUnion)narrow_tensor_by_index)dedup_save_plans)FLATTEN_MAPPINGflatten_state_dict)_flatten_sharded_tensors) set_element)BytesStorageMetadataChunkStorageMetadataMetadata MetadataIndexSTATE_DICT_TYPE STORAGE_TYPES StorageMetaTensorStorageMetadata)LoadPlan LoadPlannerReadItemSavePlan SavePlanner WriteItem WriteItemType)"_create_default_metadata_only_plan_create_read_items_create_write_items_init_state_dict)find_state_dict_object)DTensor)_versionlogger)DefaultSavePlannerDefaultLoadPlannercreate_default_local_load_plancreate_default_global_load_plancreate_default_local_save_plancreate_default_global_save_planc (\rSrSr%\\S'SS\S\S\\S\SS4 S jjrSS \ S \\ S \SS4S jjr S\ 4Sjr S\\ S\\\ \44SjrS\ S\ 4SjrS\S\\R,\R044SjrS\S\4SjrS\S\4SjrSrg)r*BmappingsNrflatten_sharded_tensorsdedup_replicated_tensorsdedup_save_to_lowest_rankreturnchXlX l0UlX@lUb[R S5 gg)NzDefaultSavePlanner's `dedup_replicated_tensors` argument is being deprecated, and no longer has any effect. Please remove this argument from your call.)rr3r2r5r)warning)selfrr3r4r5s k/var/www/html/ai-image-ml/venv/lib/python3.13/site-packages/torch/distributed/checkpoint/default_planner.py__init__DefaultSavePlanner.__init__Es9#5'>$ )B& # / NN"  0 state_dict storage_metais_coordinatorcUR(a[U5uolUR(a [U5nXlX0lgN)rr2r3rr>r@)r9r>r?r@s r:set_up_planner!DefaultSavePlanner.set_up_plannerWs:  " "(::(F %J  ' '1*=J$,r=c[URUR5nUR(a[R "XR S9nXlUR$)N planner_data)r.r>r@r dataclassesreplacer2plan)r9rJs r:create_local_plan$DefaultSavePlanner.create_local_plandsD-doot?R?RS  " "&&t--HD yyr= all_planscp[XR5n[U5up#UR(a@UVs/sHoDRPM nn[ [ U65n[R"X6S9n[X#5(d [S5eX l X0l URUR4$s snf)NrFzFailed to validate global plan) r r5r/rrGdictrrHrI_validate_global_plan ValueError global_planmetadata)r9rMrRrSpplanner_data_dictmerged_mappingss r:create_global_plan%DefaultSavePlanner.create_global_planls%Y0N0NO ? J  " " :E EA  E"8->#?@O"**8RH$[;;=> >& ..!FsB3new_plancXlU$rB)rJr9rYs r: finish_planDefaultSavePlanner.finish_plans  r= write_itemcZURUR5nURX5$rB) lookup_objectindextransform_object)r9r^objects r: resolve_dataDefaultSavePlanner.resolve_datas)##J$4$45$$Z88r=rac.[URU5$zSExtension from the planner interface to make it easy to extend the default planner.r%r>r9ras r:r` DefaultSavePlanner.lookup_object%doou==r=rccUR[R:Xa-[R"5n[ R "X#5 UnU$rg)typer BYTE_IOioBytesIOtorchsave)r9r^rcbytess r:rb#DefaultSavePlanner.transform_objects5 ??m33 3JJLE JJv %F r=) r5r3rrRr@r2rSrJr>)TTNFNF)__name__ __module__ __qualname____firstlineno__r__annotations__boolr r;rrrCrrKrr rrWr\rr rqTensorrorprdrrr`rb__static_attributes__r=r:r*r*Bs$((,37*/  "&#+4.  $(   */3$ -# -{+ - -  -8/h/ tH~x' (/0H9y9U5<<;S5T9>=>S>9cr=r*c f\rSrSr%Sr\\S'\\S'SS\S\S\SS 4S jjr SS \S \ \ S \SS 4Sjjr S\ 4SjrS\\ S\\ 4SjrS\ S\ 4SjrS\S\R(SS 4SjrS\4SjrS\S\R0SS 4SjrS\S\R04SjrS\S\R04SjrSrg ) r+aS DefaultLoadPlanner that adds multiple features on top of LoadPlanner. In particular it adds the following: flatten_state_dict: Handle state_dict with nested dicts flatten_sharded_tensors: For FSDP in 2D parallel mode allow_partial_load: If False, will raise a runtime error if a key is present in state_dict, but not in the checkpoint. original_state_dictr2rr3allow_partial_loadr6NcDXlX l0Ul0UlX0lgrB)rr3rr2r)r9rr3rs r:r;DefaultLoadPlanner.__init__s% #5'>$#%  "4r=r>rSr@c[U5 XlUR(a [U5nUR(a[ U5uolXlX lX0lgrB) r$rr3rrr2r>rSr@)r9r>rSr@s r:rC!DefaultLoadPlanner.set_up_plannersM $#-  ' '1*=J  " "(::(F %J $ ,r=cURceUR(a[URR 55n[URR R 55nX!- nU(a]S[ l[UR5upE[UR 55nXc-(aXEsUlUl S[ l[URURUR(+5$)N2_3) rSrsetr>keysstate_dict_metadatar(_derived_versionrr2r,r)r9 current_keys load_keys missing_keysold_state_dict old_mappingsold_keyss r:rK$DefaultLoadPlanner.create_local_plans}}(((  " "$t3356LDMM==BBDEI$3L,1)/A,,0,~2245*5C2DOT]-1)- OOT]]0G0G,G  r=rRc[U5$rB)r-)r9rRs r:rW%DefaultLoadPlanner.create_global_plans .{;;r=rYcU$rBr~r[s r:r\DefaultLoadPlanner.finish_plansr= read_itemvaluec *UR(aL[URURURR [ R"USS95 g[ R"USS9URURR 'g)NF) weights_only) rrrr2 dest_indexfqnrqloadr>)r9rrs r: load_bytesDefaultLoadPlanner.load_bytessl  " " (( i22667 5u5  9> E9DOOI0044 5r=cZURUR5nURX5$rB) lookup_tensorrtransform_tensorr9rtensors r:resolve_tensor!DefaultLoadPlanner.resolve_tensors)##I$8$89$$Y77r=rcgrBr~rs r: commit_tensor DefaultLoadPlanner.commit_tensors r=rac.[URU5$rgrhris r:r DefaultLoadPlanner.lookup_tensorrkr=cB[X!RUR5$rg)r dest_offsetslengthsrs r:r#DefaultLoadPlanner.transform_tensor s%f.D.DiFWFWXXr=)rr3rr@r2rSrr>)TTFru)rvrwrxry__doc__rrzrr{r;r rrCrrKrrWr\rrorprrrqr|rrrrr}r~r=r:r+r+s7)($((,#( 5  5"& 5! 5  5(,$ -#-8$- -  -&% 8% N=>U\\>Y(YELLYr=r+c t^\rSrSrSrS U4SjjrS\S\S\4Sjr SS \ S\ \S \SS4U4S jjjr S r U=r$)_EmptyStateDictLoadPlanneria Extension of DefaultLoadPlanner, which rebuilds state_dict from the saved metadata. Useful for loading in state_dict without first initializing a model, such as when converting a DCP checkpoint into a Torch save file. . N.B. `state_dict` must be an empty dictionary when used with this LoadPlanner .. warning:: Because the entire state dict is initialized, It's recommended to only utilize this LoadPlanner on a single rank or process to avoid OOM. Nc2>Xl[TU]"U0UD6 grB)rsuperr;)r9rargskwargs __class__s r:r;#_EmptyStateDictLoadPlanner.__init__s  $)&)r=keyrSr6c V^TRcgUTR;a/nURRU5nUHKnU(a0URSR US[ U5/55 M:URU5 MM [ U4SjU55(agg)NT.c3@># UHoTR;v M g7frBr).0unflattened_keyr9s r: A_EmptyStateDictLoadPlanner._should_include_key..3sTCS$))+CSsF)rrGgetappendjoinstrany)r9rrSunflattened_keysrGrs` r:_should_include_key._EmptyStateDictLoadPlanner._should_include_key!s 99  $))  &(,,005 +O ''HH.r2C4HIJ !''8 , TCST T Tr=r>r@c>U(aeUceURR5HupEURXB5(dM[U[5(a3[ R "URURRS9nXBR;a[XRUU5 MXQU'M [TU]5XU5 g)N)dtype)ritemsr isinstancerrqemptysize propertiesrrGrrrC)r9r>rSr@kvrs r:rC)_EmptyStateDictLoadPlanner.set_up_planner8s ~###00668DA++A88!233KKall.@.@A)))J(=(=a(@!D !1 9 z^Dr=rrBru)rvrwrxryrr;rrr{rrr rCr} __classcell__)rs@r:rrsj *sh44(,$ E#E8$E E  EEr=rr>rSstrictr6c :/nUR5HupEXAR;aU(a[SUS35eM,URUn[U[5(aX[ USS5bJUR UR 5:wa,[SUR SUR 5SU35e[U[5(a.URR5bU[XFU5- nMMU[XFU5- nM [U5$)Nz&Missing key in checkpoint state_dict: rrzSize mismatch between saved z and current: z for ) rr RuntimeErrorrrgetattrrrQr& device_meshget_coordinater"r)r>rSrrequestsrobjmds r:r,r,PsH$$& 22 2"%KC5PQ#RSS  ) )# . r0 1 1VT*6388:%.rwwi~chhj\QVWZV[\  c7 # #--/;.s<<< *3C8 8H/'2 H r=rMcU$)z Create global load plan used by DefaultLoadPlanner. The default load behavior involved no global coordination and this function currently doesn't change the local plans. r~)rMs r:r-r-zs  r=r@c/nUR5HUup4[U[5(a-URR 5bU[ X45- nMEMGU[ X45- nMW [ U5$)z Create the ``SavePlan`` used by DefaultSavePlanner. On non-coordinator ranks, this function ignores tensors and non-tensor objects, only producing writes for ShardedTensor objects. On the coordinator rank, produce writes for all values. )rrr&rrr#r)r>r@rrrs r:r.r.stH$$& c7 # #--/;/99< +C5 5H' H r=rewrite_index_hintsc60n/nUGHn/nURGHnUR[R:XdURR U;deUR[R :Xa3[5X&RR 'URU5 MURce[[URURR [URRURR/S955nUnU(aF[R "UR[#UR$5S9n [R "XiS9nURU5 URR&cSURR S35eUR$RURR&5 GM UR[R "XES95 GM U[)U54$)a" Create the global plan and metadata used by DefaultSavePlanner. Metadata is produced by concatenating the metadata of all ``WriteItem`` from the supplied plans. The only global planning change is to update index hints in all ``MetadataIndex`` objects if ``rewrite_index_hints`` is True. )rrchunks)razZ Cannot create MD for tensor without bounds. FQN: z )r)rrmr SHARDrarrnrr tensor_datarr setdefaultrrrHrIlenrchunkr) rMrr new_plansrJ new_itemsitem tensor_mdnew_item new_indexs r:r/r/s$&BI JJD99 3 33zz~~R///yyM111%9%;::>>"  &''333 )MM -'+'7'7'B'B!%!1!1!6!6#%   & + 3 3 #i.>.>*?!I +224IH  *$$**6**..)*6   ''(8(8(>(>?EF ,,TCDKL x| $$r=c8[U5n[U/5up#U$)zTReturn the ``Metadata`` if DefaultSavePlanner was used to checkpoint ``state_dict``.)r!r/)r>rJ_rs r:_create_default_local_metadatars -j 9D +TF 3EA Ir=box0box1c[UR5n[U5HgnURUURUURU-:a gURUURUURU-:dMg g g)z9Check if two boxes overlap. Tuples are (offset, lengths).FT)roffsetsrangesizes)rrndimsis r:_check_box_overlaprsu  E 5\ <<?dll1o 1 = = <<?dll1o 1 = =  r=outer_box_size inner_boxc[[U55HTnURUS:a gURUS:a gURUURU-X:dMT g g)NrFT)rrrr)rrrs r:_check_box_boundsrsn3~& '   Q ! # ??1  !   Q )//!"4 4~7H H ( r=rRc SnURR5GHJup4[U[5(aM[ UR 5S:XaM8Sn[ UR5Hupg[UR U5(d$[RSUUR U5 SnU[[RURS5- nURUS-SH.n[Xx5(dM[RSX7U5 SnM0 M [[RUR S5n XY:wdGM0[RSUU U5 SnGMM U$)NTrz~ key:%s has out of bounds chunk: tensor-size:%s chunk: %s Fr'z$key:%s has overlapping chunks: %s %szq key:%s invalid fill tensor-volume: %s chunks-volume: %s )rrrrrr enumeraterrr)r8roperatormulrr) rRrSall_goodrr chunks_volume chunk_idxchunk0chunk1 tensor_volumes r:rPrPs>H2288:  e1 2 2  uzz?a   !*5<V %H 8!"90x||UZZ;  ) NN HS;V Or=)T)LrHrologgingr collectionsr functoolsrtypingrrrrr r r rqtorch.distributed._shard._utilsr .torch.distributed.checkpoint._dedup_save_plansr )torch.distributed.checkpoint._nested_dictrr2torch.distributed.checkpoint._sharded_tensor_utilsr&torch.distributed.checkpoint._traverser%torch.distributed.checkpoint.metadatarrrrrrrr$torch.distributed.checkpoint.plannerrrrrrrr ,torch.distributed.checkpoint.planner_helpersr!r"r#r$"torch.distributed.checkpoint.utilsr%torch.distributed.tensorr&r( getLoggerrvr)Loggerrz__all__r*r+rrr{r,r-r.r/rrSizerrPr~r=r:rs  @@@ BKX>    F,!**844 TTnsYsYl>E!3>EDDH'S#X'*2'<@' 'T H~  (^ S#X04 :!%4%H~4%4% 4>8 #$4%n8 1 9M RV  JJ +?   -tH~--d-r=