}-j1' ddlZddlZddlZddlZddlmZmZddlmZddl m Z ej dZ de de fdZd&d eede fd Zd ee e fdee e ffd Zej dZde de fdZdZdZdddddddddZdZd'dZeGdd Z d(d!e d"ed#ed$ejdzfd%ZdS))N) OrderedDict defaultdict) dataclass)Anyz(?<=\.)(\d+)(?=\.|$)keyreturnc8td|S)zBReplace every dot-delimited integer with '*' to get the structure.*) _DIGIT_RXsub)rs a/var/www/html/banglarbhumi/venv/lib/python3.11/site-packages/transformers/utils/loading_report.py _pattern_ofrs ==c " "" valuesc t|dkrt|dSt|}t||kr|dd|dSdt t|S)zEFormat a list of ints as single number, {a, ..., b}, or first...last.rz...z, )lenstrsortedjoinmap)rcutoffs r _fmt_indicesrsw 6{{a6!9~~ F^^F 6{{V),,r ,,, 99Sf%% & &&rmappingc ld}t|tsd|D}d}tt}|D]\}}t |}t|}t|D]s\}}t|||kr'|| t||| t|t|| |i} |D]\}} | dd| d}} |d} | d} tdt| D]}|dz t| kr^| |dz rSt!t#| |dz }t| |dz dkr | d |zd zz } n | |z } n| dz } | | |z } || | <t%| }|r|S|S) z Merge keys like 'layers.0.x', 'layers.1.x' into 'layers.{0, 1}.x' BUT only merge together keys that have the exact same value. Returns a new dict {merged_key: value}. Fci|]}||Sr.0ks r z#update_key_name..2s)))A1a)))rTNrr rr{}) isinstancedictrlistitemsr findallr enumeraterappendsetaddintsplitrangerrrkeys)r not_mappingbucketrvaldigspattid out_itemsrsetspartsfinalinsertouts r update_key_namer@)s;K gt $ $))))) .9$.?.?FMMOO!!S  %%3dOO ( (DAq6$<  A%%t ##CEE*** 4LO  A ' ' ' 't C    I  f3B3Kc 3aq#e**%%  A1us4yy  T!a%[ %fT!a%[&9&9::tAE{##a''S6\C//EEVOEE  U1X EE % i Cxxzz Jrz\x1b\[[0-9;]*mscRtdt|S)N)_ansi_rer r)rAs r _strip_ansirEYs <<CFF # ##rc t|}td|tt|z }|d|zzS)Nr )rmaxrrE)textwidthtpads r _padrM]s= D A a[^^,,, - -C sSy=rcB|rtt|g|zn|g}d|Dddt|D}ddD}fd|D}d||g|zS)Nc@g|]}td|DS)c3NK|] }tt|V!dSN)rrE)r!xs r z)_make_table...fs033!#k!nn%%333333r)rH)r!cols r z_make_table..fs/ D D Dc33s33333 D D Dr | c3<K|]\}}t||VdSrQrM)r!hws r rSz_make_table..gs.IIDAqT!QZZIIIIIIrz-+-c3 K|] }d|zV dS)-Nr)r!rZs r rSz_make_table..hs&22a#'222222rc jg|]/}ddt|D0S)rVc3<K|]\}}t||VdSrQrX)r!crZs r rSz)_make_table...is.==datAqzz======r)rzip)r!rwidthss r rUz_make_table..is< L L L!EJJ==c!Vnn=== = = L L Lr )r(r`r)rowsheaderscols header_linesep_linebodyrbs @r _make_tablerjcs-1 @4wi$&( ) ) )yD D Dt D D DF**IIC4H4HIIIIIKzz22622222H L L L Lt L L LD 99k8,t3 4 44rzzzz zzzz)resetredyelloworangepurplebolditalicdimc~tjrt||tdS|S)zdReturn color/style-formatted input `s` if `sys.stdout` is interactive, e.g. connected to a terminal.rk)sysstdoutisattyPALETTE)rAcolors r _styleryys< z%.7!7WW%5777rPcX tjjS#t$r|cYSwxYwrQ)shutilget_terminal_sizecolumns Exception)defaults r _get_terminal_widthrs='))11 s  ))ceZdZUdZeeed<eeed<eeeeeeefed<e eed<e eefed<dZ dZ d ed zfd Z d S) LoadStateDictInfoaj Mutable container for state-dict loading results and diagnostics. Each entry in this structure is mutable, and will usually be mutated in-place during the loading pipeline. Attributes: missing_keys (`set[str]`): Keys that are missing from the loaded checkpoints but expected in the model's architecture. unexpected_keys (`set[str]`): Keys that are found in the checkpoints, but not expected in the model's architecture. mismatched_keys (`set[tuple[str, tuple[int], tuple[int]]]`): Keys that are found in the checkpoints and are expected in the model's architecture, but with a different shape. error_msgs ( `list[str]`): Some potential error messages. conversion_errors (`dict[str, str]`): Errors happening during the on-the-fly weight conversion process. missing_keysunexpected_keysmismatched_keys error_msgsconversion_errorsc4|jd|jDzS)zMReturn all effective missing keys, including `missing` and `mismatched` keys.ch|] }|d S)rrr s r z;LoadStateDictInfo.missing_and_mismatched..s#G#G#GQAaD#G#G#Gr)rrselfs r missing_and_mismatchedz(LoadStateDictInfo.missing_and_mismatcheds" #G#G$2F#G#G#GGGrc8|j|j|j|jdS)Nrrrrrrs r to_dictzLoadStateDictInfo.to_dicts*!-#3#3/    rrNc t}g}d}|jrh|dtdddtddzz }t|jD]+}tdd}|||d d g,|jrg|dtd d dtd dzz }t|jD]*}td d }|||d g+|jr|dtdddtddzz }d|jD}t|D]R\}\}} tdd}||dt|dt| g} || S|j r|dtdddtddzz }t|j D]3\}} tdd}d| d} |||| g4t|dkrdSddg} |dkr| dgz } n| d d gz } t|| }||z}|S)!z/Generate the minimal table of a loading report.z Notes:z - z UNEXPECTED:rn zbcan be ignored when loading from different task/architecture; not ok if you expect identical arch.rq UNEXPECTEDrCzMISSING:rlzsthose params were newly initialized because missing from the checkpoint. Consider training on your downstream task.MISSINGz MISMATCH:rmzRckpt weights were loaded, but they did not match the original empty weight shapes.c i|] \}}}|||f Srr)r!abr_s r r#z;LoadStateDictInfo.create_loading_report..s$FFFgaAAq6FFFrMISMATCHz$Reinit due to size mismatch - ckpt: z vs model:z CONVERSION:roz$originate from the conversion scheme CONVERSIONz rNKeyStatusDetails)re) rrryr@r,rrr)rrrrj)rterm_wrdtipsr"statusiteratorr shape_ckpt shape_modeldatav_detailsretablereports r create_loading_reportz'LoadStateDictInfo.create_loading_reports#$&&   1 >6-::>>>tBB D%T%9:: 1 1 h77 QB/0000   - 86*e448886F<< D%T%677 - - 511 QO,,,,   " <6+x88<<6-::>>>6BB D((>??EEGG 3 31 h77)!>>> Q12222 t99>>4(# C<<  { "GG Bx GD'222 r)__name__ __module__ __qualname____doc__r-r__annotations__tupler/r(r'rrrrrr rrs"c(XsE#Jc :;<<<<S CH~%%%HHH   ;sTz;;;;;;rrpretrained_model_name_or_pathignore_mismatched_sizes loading_infologgerc|tjt}|jrBd|j}d|vr|dz }t d|jjd||}|dStd|jjdtd d |d }| ||z|j rt d |s|j rt d dSdS)z Log a readable report about state_dict loading issues. This version is terminal-size aware: for very small terminals it falls back to a compact Key | Status view so output doesn't wrap badly. Nz z size mismatchzb You may consider adding `ignore_mismatched_sizes=True` to `from_pretrained(...)` if appropriate.z#Error(s) in loading state_dict for z: rpz LOAD REPORTrkz from: rczWe encountered some issues during automatic conversion of the weights. For details look at the `CONVERSION` entries of the above report!zjYou set `ignore_mismatched_sizes` to `False`, thus raising an error. For details look at the above report!) logging getLoggerrrr RuntimeError __class__rrwwarningrr)modelrrrr error_msgrpreludes r log_state_dict_reportrs\~"8,,mKK 788 i ' ' v IkAYkk`ikklll / / 1 1F ~C%/":CCPWHXCCa~CCCG NN7V#$$$%      # |'C  x       r)r)rzrQ)rrer|rt collectionsrr dataclassesrtypingrcompiler rrr(r/rr'r@rDrErMrjrwryrrboolLoggerrrrr rs* 00000000!!!!!! BJ. / / #S#S#### ''c'#''''*T#s(^*S#X****Z 2:' ( ($3$3$$$$ 555      ``````` `P%) , , #&, ", $ , NT ! , , , , , , r