j UddlmZddlmZddlZddlmZddlmZddlm Z ddl m Z dd l m Z d d gZd d deedddd d deeddddZeeeeeffed<GddejZGddejZ d"dededeedzdedef dZd#dededefd Zd#dededefd!ZdS)$)deepcopy)AnyN)nn)VOCABS) EncoderBlock)PatchEmbeddingload_pretrained_paramsvit_svit_b)gh|?5?g= ףp=?gV-?)gA`"?gl?g$C?r rfrenchzHhttps://doctr-static.mindee.com/models?id=v0.6.0/vit_s-5d05442d.pt&src=0)meanstd input_shapeclassesurlzHhttps://doctr-static.mindee.com/models?id=v0.6.0/vit_b-0fbef167.pt&src=0)r r default_cfgscTeZdZdZdededdffd ZdejdejfdZxZ S) ClassifierHeadzClassifier head for Vision Transformer Args: in_channels: number of input channels num_classes: number of output classes in_channels num_classesreturnNc|ttj|||_dSN)super__init__rLinearhead)selfrr __class__s j/var/www/html/Carbon-Document/venv/lib/python3.11/site-packages/doctr/models/classification/vit/pytorch.pyrzClassifierHead.__init__/s2 Ik;77 xc@||dddfS)Nr)r!)r"r&s r$forwardzClassifierHead.forward8s yy111a4!!!r%) __name__ __module__ __qualname____doc__intrtorchTensorr( __classcell__r#s@r$rr's888  888888""%,""""""""r%rceZdZdZ dded ed ed ed eeefd eeeefdedededee e fdzddffd Z de de ddfdZ xZ S)VisionTransformeraVisionTransformer architecture as described in `"An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale", `_. Args: d_model: dimension of the transformer layers num_layers: number of transformer layers num_heads: number of attention heads ffd_ratio: multiplier for the hidden dimension of the feedforward layer patch_size: size of the patches input_shape: size of the input image dropout: dropout rate num_classes: number of output classes include_top: whether the classifier head should be instantiated r5rTNd_model num_layers num_heads ffd_ratio patch_sizerdropoutr include_topcfgrc t|||t|||||z|tjg} | r#| t ||t j| | |_dSr) rrrGELUappendrrrr?) r"r8r9r:r;r<rr=rr>r?_layersr#s r$rzVisionTransformer.__init__Ns ; < < Y99LgWYW^W`W` a a$   A NN>';?? @ @ @'""r% path_or_urlkwargsc "t||fi|dS)zLoad pretrained parameters onto the model Args: path_or_url: the path or URL to the model parameters (checkpoint) **kwargs: additional arguments to be passed to `doctr.models.utils.load_pretrained_params` Nr )r"rDrEs r$from_pretrainedz!VisionTransformer.from_pretrainedes" t[;;F;;;;;r%)r4rr6r7TN)r)r*r+r,r-tuplefloatbooldictstrrrrGr0r1s@r$r3r3=s,'-,7 %)    #s(O 3S=)#s(^d " .<3<#<$<<<<<<<`_. Patches: (H, W) -> (H/8, W/8) NOTE: unofficial config used in ViTSTR and ParSeq >>> import torch >>> from doctr.models import vit_s >>> model = vit_s(pretrained=False) >>> input_tensor = torch.rand((1, 3, 32, 32), dtype=tf.float32) >>> out = model(input_tensor) Args: pretrained: boolean, True if model is pretrained **kwargs: keyword arguments of the VisionTransformer architecture Returns: A feature extractor model r i r5 2.head.weight 2.head.biasr8r9r:r;rOrXrNrEs r$r r sF(  $m4      r%c .td|fddddddgd|S)a]VisionTransformer-B architecture as described in `"An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale", `_. Patches: (H, W) -> (H/8, W/8) >>> import torch >>> from doctr.models import vit_b >>> model = vit_b(pretrained=False) >>> input_tensor = torch.rand((1, 3, 32, 32), dtype=tf.float32) >>> out = model(input_tensor) Args: pretrained: boolean, True if model is pretrained **kwargs: keyword arguments of the VisionTransformer architecture Returns: A feature extractor model r irZr5r\r]r^r_r`s r$r r sF$  $m4      r%r)F)copyrtypingrr.rdoctr.datasetsr doctr.models.modules.transformerr'doctr.models.modules.vision_transformerrutilsr __all__listrrKrL__annotations__Moduler Sequentialr3rJrXr r rQr%r$rms5  !!!!!!999999BBBBBB++++++ G  &$"4x())Y &$"4x())Y ++ d3S#X&'$"""""RY""",/</</</</< /</</