j?UddlmZddlmZddlZddlmZddlmZddl m Z ddl m Z dd l mZmZmZmZmZmZmZd d gZd d deedddd d deeddddZeeeeeffed<GddejZGddejZGddejZd0de dedefdZ!d0de dedefd Z"d!ede d"eededef d#Z# d1d$e$d%e$d&e$d'e%d(ee%d)e$d*e$d+e d,e$dzdejfd-Z& d1d$e$d%e$d&e$d'e%d(ee%d)e$d*e$d+e d,e$dzdejfd.Z' d1d$e$d%e$d&e$d'e%d(ee%d)e$d*e$d+e d,e$dzdejfd/Z(dS)2)deepcopy)AnyN)VOCABS)AdaptiveAvgPool2dload_pretrained_params)CrossShapedWindowAttentionMultiHeadSelfAttention OSRABlock PatchEmbed PatchMerging PermuteLayer SqueezeLayervip_tinyvip_base)gh|?5?g= ףp=?gV-?)gA`"?gl?g$C?r rfrenchzLhttps://doctr-static.mindee.com/models?id=v0.11.0/vip_tiny-033ed51c.pt&src=0)meanstd input_shapeclassesurlzLhttps://doctr-static.mindee.com/models?id=v0.11.0/vip_base-f6ea2ff5.pt&src=0)rr default_cfgscPeZdZdZdedeffd ZdejdejfdZxZ S)ClassifierHeadzKClassification head which averages the features and applies a linear layer. in_features out_featuresc|ttj|||_dS)N)super__init__nnLinearfc)selfrr __class__s j/var/www/html/Carbon-Document/venv/lib/python3.11/site-packages/doctr/models/classification/vip/pytorch.pyr#zClassifierHead.__init__1s0 )K66xreturncT||dS)Nr dim)r&r)r'r+s r)forwardzClassifierHead.forward5s wwqvv!v}}%%%r*) __name__ __module__ __qualname____doc__intr#torchTensorr0 __classcell__r(s@r)rr.swUU7C7s777777&&%,&&&&&&&&r*rceZdZdZ ddedejdejdzdejdzded edzf fd Z d e j d e j fd Z xZ S)VIPBlockzNUnified block for Local, Global, and Mixed feature mixing in VIP architecture.NF embed_dim local_unit global_unitproj downsampleout_dimct|r|td||_||_||_|rt ||nd|_dS)aH Args: embed_dim: dimension of embeddings local_unit: local mixing block(s) global_unit: global mixing block(s) proj: projection layer used for mixed mixing downsample: whether to downsample at the end out_dim: out channels if downsampling Nz0`out_dim` must be specified if `downsample=True`)r/rA)r"r# ValueErrorr=r>r?rr@)r'r<r=r>r?r@rAr(s r)r#zVIPBlock.__init__<sp$   Q'/OPP P$& JT^,9gFFFFZ^r*r+r,c0|j\}}}}|jI|jD]?}||d|}||||f}||||d}@nt |j|jD]\}}||d|}t j|dd\} } || ||f} || ||f} t j| | gd}|dd |d||}| ||z}| dddd }t|j tjr| |}|S) z Forward pass for VIPBlock. Args: x: input tensor (B, H, W, C) Returns: Transformed tensor N)chunksr/r.r rr)shaper>r=reshapezipr6chunkcat transpose contiguousr?permute isinstancer@r$Module) r'r+bhwCblklblkgblkx1x2s r)r0zVIPBlock.forwardWsW 1a   # + +IIaQ''CAq6NNIIaAr**  +"$/43CDD 7 7 dIIaQ''Qqa888BT"q!f%%T"q!f%%Ir2hA...KK1%%0022::1b!QGGIIaLL1$IIaAq))4466 dory 1 1 #""Ar*)NNFN)r1r2r3r4r5r$ ModuleListrQboolr#r6r7r0r8r9s@r)r;r;9sXX -1!% "___M_]T) _ i$ _  _t______6$$%,$$$$$$$$r*r;ceZdZdZ ddededeed eed eed eed eed eedeeeefdededee e fdzddffd Z dZ de de ddfdZ xZS)VIPNetzU VIP (Vision Permutable) encoder architecture, adapted for text recognition. rTN in_channelsrA embed_dimsdepths num_heads mlp_ratios split_sizes sr_ratiosr num_classes include_topcfgr,c | |_dtjddtDfdt t D} t ||dg}tttg}t|D]\}}||}|}||}||}||}||}| |}|t |dz kr ||dznd}|||||||||du| }| || tj tj|d d td t!|d dft#d tj tj|d |dtjtjd}| || r#| t+|| t-j|||jdS)a| Args: in_channels: number of input channels out_dim: final embedding dimension embed_dims: list of embedding dims per stage depths: number of blocks per stage num_heads: number of heads for attention blocks mlp_ratios: ratio for MLP expansion split_sizes: local window split sizes sr_ratios: used for some global block adjustments input_shape: (C, H, W) num_classes: number of output classes include_top: if True, append a classification head cfg: optional config dictionary c6g|]}|S)item).0r+s r) z#VIPNet.__init__..s EEEAqvvxxEEEr*rg?c ~g|]9}td|td|dz:S)Nr )sum)rnirbdprs r)roz#VIPNet.__init__..sE^^^ac#fRaRj//CwQw,@,@@A^^^r*)r`r<r N) r<depthrc mlp_ratio split_sizesr_ratio drop_pathr@rArEgư>)eps)rrFrr rr.F)bias)p)rir6linspacerqrangelenr_vip_local_mixer_vip_mixed_mixer_vip_global_mha_mixer enumerateappendr$ Sequential LayerNormrrrr% HardswishDropoutrr"r#apply _init_weights)r'r`rArarbrcrdrerfrrgrhri drop_pathslayersmixer_functionsrrmixer_fnr<depth_inum_headrusp_sizerwrxnext_dimblockmlp_headrsr(s ` @r)r#zVIPNet.__init__sv<EE3F !D!DEEE^^^^^5QTU[Q\Q\K]K]^^^ 'K:VW=YYYZ   !  %_55 ! !KAx"1 IQiG |H"1 I!!nG |H"1 I,-J!0C,C,Cz!a%((HH#"#"!#$D0    E MM%   M Z^666\**!:b>1"566###      = IjngE : : : LNN J       h  @ MM.+>> ? ? ?&!! 4%&&&&&r*cPt|tjrVtj|jd|j'tj|jddSdSt|tjr)tj |jdddSt|tj tj frLtj|jdtj|jddSdS)Ng{Gz?)rrfan_outrelu)mode nonlinearityg?) rPr$r%init trunc_normal_weightrz constant_Conv2dkaiming_normal_r BatchNorm2d)r'ms r)rzVIPNet._init_weightss a # # - G ! !!( ! 5 5 5v!!!!&!,,,,,"! 29 % % - G # #AH96 # R R R R R BL".9 : : - G  afa ( ( ( G  ah , , , , , - -r* path_or_urlkwargsc "t||fi|dS)zLoad pretrained parameters onto the model Args: path_or_url: the path or URL to the model parameters (checkpoint) **kwargs: additional arguments to be passed to `doctr.models.utils.load_pretrained_params` Nr)r'rrs r)from_pretrainedzVIPNet.from_pretraineds" t[;;F;;;;;r*)rr_TN)r1r2r3r4r5listtupler\dictstrrr#rrr8r9s@r)r^r^~sW-8 %)Y'Y'Y'Y'I Y' S Y' 9 Y'IY'#YY'9Y'3S=)Y'Y'Y'#s(^d "Y' Y'Y'Y'Y'Y'Y'v - - -<3<#<$<<<<<<<>yy))99II))"K0      r*c Ntd|fddgdgdgdgdgdgd d d gd |S) aZ VIP-Base encoder architecture. Corresponds to SVIPTRv2-B variant in the paper (VIPTRv2B function in the official implementation: https://github.com/cxfyxl/VIPTR/blob/main/modules/VIPTRv2.py) Args: pretrained: whether to load pretrained weights **kwargs: optional arguments Returns: VIPNet model rrr)rri)r )rr )rrrrrrrrrrs r)rrsj  "??yy**99II))"K0      r*archrc |dtt|d|d<|dt|d|d<|dt|d|d<tt|}|d|d<|d|d<|d|d<|dt dd|i|}|rV|dtt|dkr|nd}|t|d||S) a  Internal constructor for the VIPNet models. Args: arch: architecture key pretrained: load pretrained weights? ignore_keys: layer keys to ignore **kwargs: arguments passed to VIPNet Returns: VIPNet instance rgrrriNr)rrl)getr~rrpopr^r)rrrr_cfgmodel _ignore_keyss r)rr-s:$#JJ}c,t:LY:W6X6XYYF="JJ}l46H6WXXF= 9l4.@.KLLF9 L& ' 'D /D /DY'DO JJy  & &t &v & &ES'-]&;s