RЦiQ)2SrSSKrSSKJr SSKrSSKJr SSKJrJ r J r J r J r SSK Jr Sr\4S jrS rS rS r"S S5r/SQ4Sjr/SS4Sjr/S4Sjr/S4SjrSS\R2R4S\R2R44Sjjrg)zLModel / state_dict utils Hacked together by / Copyright 2020 Ross Wightman Ndeepcopy)FrozenBatchNorm2d)BatchNormAct2dSyncBatchNormActFrozenBatchNormAct2dfreeze_batch_norm_2dunfreeze_batch_norm_2d)ModelEmac[U[5(a[UR5$[ US5(a[UR 5$[ US5(a[UR 5$U$)Nmodule _orig_mod) isinstancer unwrap_modelemahasattrrr)models O/var/www/html/ai-image-ml/venv/lib/python3.13/site-packages/timm/utils/model.pyrrs[%""EII&& 5( # # - - UK ( (0 0Lc.U"U5R5$N) state_dict)r unwrap_fns rget_state_dictrs U  & & ((rcn[R"UR/SQS9S-5R5$)z@calculate average channel square mean of output activations raxisr)torchmeanitemrinputoutputs ravg_sq_ch_meanr( s, ::fkkyk1Q6 7 < < >>rch[R"UR/SQS95R5$z=calculate average channel variance of output activations rr r"r#varr$r%s r avg_ch_varr-&' ::fjjij0 1 6 6 88rch[R"UR/SQS95R5$r*r+r%s ravg_ch_var_residualr0,r.rc*\rSrSrSrSrSrSrSrg)ActivationStatsHook2aIterates through each of `model`'s modules and matches modules using unix pattern matching based on `hook_fn_locs` and registers `hook_fn` to the module if there is a match. Arguments: model (nn.Module): model from which we will extract the activation stats hook_fn_locs (List[str]): List of `hook_fn` locations based on Unix type string matching with the name of model's modules. hook_fns (List[Callable]): List of hook functions to be registered at every module in `layer_names`. Inspiration from https://docs.fast.ai/callback.hook.html. Refer to https://gist.github.com/amaarora/6e56942fcb46e67ba203f3009b30d950 for an example on how to plot Signal Propagation Plots using `ActivationStatsHook`. cXlX lX0l[U5[U5:wa [ S5e[ SU55Ul[X#5HupEURXE5 M g)Nz_Please provide `hook_fns` for each `hook_fn_locs`, their lengths are different.c3<# UHoR/4v M g7fr)__name__).0hook_fns r /ActivationStatsHook.__init__..KsIW++R0s) r hook_fn_locshook_fnslen ValueErrordictstatszip register_hook)selfrr;r< hook_fn_locr8s r__init__ActivationStatsHook.__init__Dsh ( | H -./ /III $' $? K   { 4%@rc^^UU4SjnU$)Nch>T"XU5nTRTRRU5 gr)r@r6append)rr&r'outr8rCs rappend_activation_statsAActivationStatsHook._create_hook..append_activation_statsPs,&0C JJw'' ( / / 4r)rCr8rKs`` r _create_hook ActivationStatsHook._create_hookOs 5'&rcURR5HBup4[R"X15(dM"URUR U55 MD gr)r named_modulesfnmatchregister_forward_hookrN)rCrDr8namers rrB!ActivationStatsHook.register_hookVsF JJ446LD??455  ( ():):7)C D7r)r;r<rr@N) r6 __module__ __qualname____firstlineno____doc__rErNrB__static_attributes__rMrrr2r22s" 5'Err2)rr\cn[R"SSU5n[XUS9nU"U5nUR$)zExtract average square channel mean and variance of activations during forward pass to plot Signal Propagation Plots (SPP). Paper: https://arxiv.org/abs/2101.08692 Example Usage: https://gist.github.com/amaarora/6e56942fcb46e67ba203f3009b30d950 gg?)r;r<)r"normalr2r@)rr;r< input_shapexhook_s rextract_spp_statsrc]s5  R[)A u( SD aA ::rTfreezecUS;dS5e[U[RRRR [RRRR [[45(a [S5e[U[5(aU/nUnUVs/sHoPRU5PM nn[U5(d![[UR565upA[XA5HupeUR!5HnUS:XaSOSUlM U(dM4SnUS:Xa[%U5n [U[RRRR [RRRR [[45(a U"XU 5 MM['U5n [U[([*45(dMU"XU 5 M gs snf) a  Freeze or unfreeze parameters of the specified modules and those of all their hierarchical descendants. This is done in place. Args: root_module (nn.Module, optional): Root module relative to which the `submodules` are referenced. submodules (list[str]): List of modules for which the parameters will be (un)frozen. They are to be provided as named modules relative to the root module (accessible via `root_module.named_modules()`). An empty list means that the whole root module will be (un)frozen. Defaults to [] include_bn_running_stats (bool): Whether to also (un)freeze the running statistics of batch norm 2d layers. Defaults to `True`. mode (bool): Whether to freeze ("freeze") or unfreeze ("unfreeze"). Defaults to `"freeze"`. )rdunfreezez,`mode` must be one of "freeze" or "unfreeze"zYou have provided a batch norm layer as the `root module`. Please use `timm.utils.model.freeze_batch_norm_2d` or `timm.utils.model.unfreeze_batch_norm_2d` instead.rdFTcURSS5n[U5S:a(URUS5RUSU5 gURX5 g)N.r r)rsplitr= get_submodule add_module)rrT submodulesplits r_add_submodule(_freeze_unfreeze.._add_submodulesN C+u:>((q2==eAh R%%d6rN)rr"nnmodules batchnorm BatchNorm2d SyncBatchNormrrAssertionErrorstrrjr=listrAnamed_children parameters requires_gradr r rr) root_module submodulesinclude_bn_running_statsmoderQmnprnress r_freeze_unfreezeros ) )Y+YY )+ HH   & & 2 2 HH   & & 4 4     lm m*c"" \ M8BC 1++A. JC z??$(k.H.H.J)K$L! M.A'+x'7eTAO # # 7x*1-a((22>>((22@@&( " #;37 -Q/a"35I!JKK";37?/ Ds,Hc[XUSS9 g)a: Freeze parameters of the specified modules and those of all their hierarchical descendants. This is done in place. Args: root_module (nn.Module): Root module relative to which `submodules` are referenced. submodules (list[str]): List of modules for which the parameters will be frozen. They are to be provided as named modules relative to the root module (accessible via `root_module.named_modules()`). An empty list means that the whole root module will be frozen. Defaults to `[]`. include_bn_running_stats (bool): Whether to also freeze the running statistics of `BatchNorm2d` and `SyncBatchNorm` layers. These will be converted to `FrozenBatchNorm2d` in place. Hint: During fine tuning, it's good practice to freeze batch norm stats. And note that these are different to the affine parameters which are just normal PyTorch parameters. Defaults to `True`. Hint: If you want to freeze batch norm ONLY, use `timm.utils.model.freeze_batch_norm_2d`. Examples:: >>> model = timm.create_model('resnet18') >>> # Freeze up to and including layer2 >>> submodules = [n for n, _ in model.named_children()] >>> print(submodules) ['conv1', 'bn1', 'act1', 'maxpool', 'layer1', 'layer2', 'layer3', 'layer4', 'global_pool', 'fc'] >>> freeze(model, submodules[:submodules.index('layer2') + 1]) >>> # Check for yourself that it works as expected >>> print(model.layer2[0].conv1.weight.requires_grad) False >>> print(model.layer3[0].conv1.weight.requires_grad) True >>> # Unfreeze >>> unfreeze(model) rdr}r~Nrr{r|r}s rrdrds@[G_fnorc[XUSS9 g)a Unfreeze parameters of the specified modules and those of all their hierarchical descendants. This is done in place. Args: root_module (nn.Module): Root module relative to which `submodules` are referenced. submodules (list[str]): List of submodules for which the parameters will be (un)frozen. They are to be provided as named modules relative to the root module (accessible via `root_module.named_modules()`). An empty list means that the whole root module will be unfrozen. Defaults to `[]`. include_bn_running_stats (bool): Whether to also unfreeze the running statistics of `FrozenBatchNorm2d` layers. These will be converted to `BatchNorm2d` in place. Defaults to `True`. See example in docstring for `freeze`. rfrNrrs rrfrfs[G_fpqrrreturncH^U(d [U5nU4SjmT"U5 U$)Nc&>UR5H|up[US5(a[XUR55 OC[US5(aUR 5 O![US5(aUR 5 T"U5 M~ g)Nfusereparameterizeswitch_to_deploy)rxrsetattrrrr)r child_namechild_fuses rr#reparameterize_model.._fusesq!"!1!1!3 Juf%%uzz|4 011$$& 233&&( %L"4rr)rinplacers @rreparameterize_modelrs"  %L Lr)F)rYrRcopyrr"torchvision.ops.miscr timm.layersrrrr r model_emar rrr(r-r0r2rcrrdrfrpModulerrMrrrs 211 %1)? 9 9 (E(E^% $.0$U]C8L$& pF&($r"588??r