}-j1NdZddlmZddlZddlmZddlmZddlm Z ddl m Z dd l m Z mZe r dd lmZd d lmZiZeed GddZGddZedZ d-d.dZd/d$Zd0d1d)ZejZd2d*Zd0dd+d,ZdS)3z Contains the logic for automatic additional output capture with our forward decorators. This mostly describe the hooks used and the logic to make capture thread/context safe. ) annotationsN) ContextVar) dataclasswraps) TYPE_CHECKING)is_torchdynamo_compilingrequires)nnPreTrainedModel)torch)backendscVeZdZUdZded<dZded<dZded <dZded <d Zd ed <dS)OutputRecordera Configuration for recording outputs from a model via hooks. Attributes: target_class (Type): The class (e.g., nn.Module) to which the hook will be attached. index (Optional[int]): If the output is a tuple/list, optionally record only at a specific index. layer_name (Optional[str]): Name of the submodule to target (if needed), e.g., "transformer.layer.3.attn". class_name (Optional[str]): Name of the class to which the hook will be attached. Could be the suffix of class name in some cases. capture_initial_hidden_state (bool): Whether to prepend the first module's input as the initial hidden state. ztype[nn.Module] target_classrintindexN str | None layer_name class_nameTboolcapture_initial_hidden_state) __name__ __module__ __qualname____doc____annotations__rrrrc/var/www/html/banglarbhumi/venv/lib/python3.11/site-packages/transformers/utils/output_capturing.pyrr'sk  "!!!ENNNN!J!!!!!J!!!!)- ------r"rc*eZdZdZdZdZdZdZdS)CompileableContextVara Convenience wrapper around a ContextVar for usage with `torch.compile`. This behaves exactly as a `ContextVar`, except when compilation is triggered in which case it behaves as a simple global variable. This is useful as `torch.compile` cannot trace the `get` method of `ContextVar`. This however means that the access to the underlying variable is not thread-safe when compilation is triggered. cNt|d|_d|_d|_dS)N)defaultF)r context_var global_var compiling)selfnames r#__init__zCompileableContextVar.__init__Ds)%dD999r"cP|jr|jS|jSN)r*r)r(get)r+s r#r0zCompileableContextVar.getIs) > *? "#'')) )r"crtr||_d|_dS|j|S)NT)r r)r*r(set)r+values r#r2zCompileableContextVar.setPs: # % % /#DO!DN4#''.. .r"cl|js|d|_d|_dS|j|dS)NF)r*r)r(reset)r+tokens r#r5zCompileableContextVar.resetXs? > *U]"DO"DNNN   " "5 ) ) ) ) )r"N)rrrrr-r0r2r5r!r"r#r%r%<sZ ***///*****r"r%output_collectorTmodule nn.ModulekeystrrrrrreturnNonecDfd}||dS)zaInstall the forward hook needed to capture the output described by `key` and `index` in `module`.ct}||vrdSr@dkr:t|dkr!||dt |t s||dS|#||dSdS)N hidden_statesr)_active_collectorr0keyslenappend isinstancetuple)r8argsoutputcollected_outputsrrr:s r#output_capturing_hookz;install_output_capuring_hook..output_capturing_hookis-1133  $3D3I3I3K3K(K(K F ' 3C?,B,BsK\]`KaGbGbfgGgGg c " ) )$q' 2 2 2&%(( 9 c " ) )& 1 1 1 1 1 E] & c " ) )&- 8 8 8 8 8' &r"N)register_forward_hook)r8r:rrrJs ``` r#install_output_capuring_hookrLdsE 9 9 9 9 9 9 9   !677777r" parent_module module_name capture_tasks list[tuple[str, OutputRecorder]]cddlm}|D]B\}}t||st ||d||,t ||d|C|D]o\}}|jt||js!|jG||jr-|j |j |vrSt|||j |j pdS)a Recursively install all output capturing hooks on all submodules of `parent_module`. Note that we need to use this recursive approach instead of simply iterating over all modules, because we want to respect the `capture_tasks` of all individual submodels (`PreTrainedModel` instances) in the graph. That is, once we reach a submodel in the graph, its children should use this submodel's `capture_tasks`, but other parts of the graph should not. r r.)prefixN) modeling_utilsrnamed_childrenrErecursively_install_hooks"install_all_output_capturing_hooksrrendswithrrLrr)rMrNrOrr,r8r:specss r#rVrVzs%100000&4466WW f&/22 W %f.E.Et.E.E} U U U U /v>U>Ut>U>U V V V V V$nn U   *z-I[/\/\ *   ([-A-A%BR-S-S (+0@ 0S0S (U[%Jl m m mnnr"modelrrSrctt|jpi}g}|D]\}}t |t s|g}|D]z}t |tsLd|vrdnd}t |tsdn|}t |ts|nd} t| ||}|||f{||nd}t|||t|dddS) z Install the output recording hooks on all the modules in `model`. This will take care of correctly dispatching the `_can_record_outputs` property of each individual submodels in case of composite models. r@rr N)rrr!_output_capturing_hooks_installedT) _CAN_RECORD_REGISTRYr0r; __class__itemsrElistrrDrVsetattr) rZrS capture_flagsrOr: layer_specsrYrrrs r#rWrWs9 ),,S-A-ABBHbMM)//11 / /[+t,, (&-K  / /Ee^44 f,33)3E3)?)?JTTU ,6uc,B,BLuu &LZdeee  #u . . . .  /)VVrFeV];;; E6=====r"ct|ddrdSt5t|ddr ddddSt|ddddS#1swxYwYdS)z Check if the model already has output capturing hooks installed, and install them if it is not already the case. Note that this is thread-safe, in case 2 (or more) threads want to install them concurrently. r]FN)getattr_hook_installation_lockrW)rZs r#maybe_install_capturing_hooksrhsu95AA 22 5=u E E   22222222 +5111 222222222222222222sAAAA)tie_last_hidden_statesc,fd}| ||S|S)a Decorator to intercept specific layer outputs through hooks. The hooks are installed only once and lazily, the first time output capture is requested with the `output_xxx` kwargs/config. The implementation is fully context/thread safe, except when using `torch.compile`, as dynamo is unable to trace through `ContextVar` methods. Args: tie_last_hidden_states (`bool`, *optional*, defaults to `True`): Whether to overwrite `out.hidden_states[-1]` with the `out.last_hidden_state`. This is true for all language models and should be toggled off only if `out.hidden_states[-1]` has to be the hidden state before last layer norm, which is needed for some vision models (e.g. CLIP, SigLIP) c@tfd}|S)Ncdtjdd}tt jpi}fd|D}d|vr-dtjdd|d<d|vr-dtjdd|d <d |D}t|d krtt |}  g|Ri}t |n#t |wxYw|D]_} | d kr snt|d r4|| dd|| <|| |jnCt|dr3|| dd|| <|| |jt#|| || <| dkrt%|| t&r`t|| dkrGt#|| d dd|| <t#|| ddd|d| z<-t#|| || <Gt#|| || <a|dur|}|S)N return_dictTc xi|]6}d|d|tjd|d7S)output_F)r0rfconfig).0kkwargsr+s r# zHcapture_outputs..wrapped_fn..wrapper..s[! vzz-A--mXYmm]b9c9cddr"cross_attentionsoutput_attentionsFoutput_cross_attentionsmask_decoder_attentionsoutput_mask_decoder_attentionscFi|]\}}||ddgS)ror\)replace)rqrrvs r#rtzHcapture_outputs..wrapped_fn..wrapper..s2 g g g$!Qef g9b!9!92 g g gr"rr@vision_hidden_stateslast_hidden_state attentionsr r cross_)poprfrpr^r0r;r_r`rCrhrAr2r5hasattrrDr}rrFrErato_tuple) r+rGrsrmcapturable_flagsrecordable_keysrI output_tokenoutputsr:funcris ` ` r#wrapperz4capture_outputs..wrapped_fn..wrappers]!**]GDKX\4]4]^^K 477DN8K8KLLRPR )O "%555=CZZ'>QSX)Y)Y>> 9:),<<QSX)Y)YEE @A!h g/J_J_JaJa g g g $%%))-d333,001BCCL 6$t5d555f55"'' 5555!'' 5555) A A/))1Q *@AAQ1B31G1L)#.)#.55g6RSSSS *=>>Q1B31G1L)#.)#.55g6OPPP#():3)?#@#@GCLLL((!"23"7>>E3GWX[G\C]C]abCbCb',->s-CADqD-I'J'J 278I#8NqtRSt8T2U2U3//',->s-C'D'D #():3)?#@#@GCLLe##!**,,Ns ( EE+r)rrris` r# wrapped_fnz#capture_outputs..wrapped_fns: t= = = = =  = ~r"r!)rrirs ` r#capture_outputsrs@AAAAAF z$ r")T) r8r9r:r;rrrrr<r=)rMr9rNr;rOrPr<r=r/)rZrrSrr<r=)rZrr<r=)r __future__r threading contextvarsr dataclassesr functoolsrtypingr import_utilsr r rr rTrr^rr%rArLrVrWLockrgrhrr!r"r#rs #"""""""""""!!!!!! <<<<<<<<1000000  :....... .&!*!*!*!*!*!*!*!*J*)*<==SW88888,nnnn@>>>>>:)).**2222&TTTTTTTTr"