RЦi/SrSSKJrJrJr SSKrSSKJr SSKJr SSK J r SSK J r "S S \R5r"S S \R5r"S S\R5r"SS\R5rg)a^Coordinate Attention and Variants Coordinate Attention decomposes channel attention into two 1D feature encoding processes to capture long-range dependencies with precise positional information. This module includes the original implementation along with simplified and other variants. Papers / References: - Coordinate Attention: `Coordinate Attention for Efficient Mobile Network Design` - https://arxiv.org/abs/2103.02907 - Efficient Local Attention: `Rethinking Local Perception in Lightweight Vision Transformer` - https://arxiv.org/abs/2403.01123 Hacked together by / Copyright 2025 Ross Wightman )OptionalTypeUnionN)nn)create_act_layer)make_divisible) GroupNorm1c^\rSrSrSSSSS\R \R SSSS4 S\S \S \ \S \S \S \ S\ \RS\ \ \RS\ \\ \R4S\ 4U4SjjjrSrSrU=r$) CoordAttng?NUUUUUU?Fsigmoidchannelsrd_ratio rd_channels rd_divisor se_factorbias act_layer norm_layer gate_layerhas_skipc >XS.n [TU]5 XlU(d[X-U-USS9n[R "X4SSSUS.U D6UlUb U"U40U D6O[R"5UlU"5Ul [R "X14SSSUS.U D6Ul [R "X14SSSUS.U D6Ul [U 5Ul g)a5Coordinate Attention module for spatial feature recalibration. Introduced in "Coordinate Attention for Efficient Mobile Network Design" (CVPR 2021). Decomposes channel attention into two 1D feature encoding processes along the height and width axes to capture long-range dependencies with precise positional information. Args: channels: Number of input channels. rd_ratio: Reduction ratio for bottleneck channel calculation. rd_channels: Explicit number of bottleneck channels, overrides rd_ratio if set. rd_divisor: Divisor for making bottleneck channels divisible. se_factor: Applied to rd_ratio for final channel count (keeps params similar to SE). bias: Whether to use bias in convolution layers. act_layer: Activation module class for bottleneck. norm_layer: Normalization module class, None for no normalization. gate_layer: Gate activation, either 'sigmoid', 'hardsigmoid', or a module class. has_skip: Whether to add residual skip connection to output. device: Device to place tensors on. dtype: Data type for tensors. devicedtype round_limitrr) kernel_sizestridepaddingrN)super__init__rr rConv2dconv1Identitybn1actconv_hconv_wrgate)selfrrrrrrrrrrrrdd __class__s U/var/www/html/ai-image-ml/venv/lib/python3.13/site-packages/timm/layers/coord_attn.pyr&CoordAttn.__init__sH /  ()4J:k0R0PRP[P[P];ii k1QXY`dkhjk ii k1QXY`dkhjk $Z0 c4UnUR5up4pVURSSS9nURSSS9nURSS5n[R"Xx/SS9n UR U 5n UR U 5n URU 5n [R"XU/SS9upxURSS5nURURU55n URURU55n X+-U -n UR(aX-n U $)NT)keepdimdim) sizemean transposetorchcatr(r*r+splitr.r,r-r) r/xidentityNCHWx_hx_wya_ha_wouts r2forwardCoordAttn.forwardJsVVX affQf%ffQf%mmB# IIsja ( JJqM HHQK HHQK;;qa&a0mmB#ii C()ii C()ns" ==.C r4)r+r*r(r,r-r.r)__name__ __module__ __qualname____firstlineno__r Hardswish BatchNorm2dintfloatrboolrModulerstrr&rO__static_attributes__ __classcell__r1s@r2r r s&)-")+46NN6?"010101"# 01  01  0101BII01!bii101c4 ?23010101dr4r c^\rSrSrSrSSSSS\R SS SS4 S \S \S \ \S \S\S\ S\ \RS\ \\ \R4S\ 4U4SjjjrSrSrU=r$)SimpleCoordAttneaSimplified Coordinate Attention variant. Uses * linear layers instead of convolutions * no norm * additive pre-gating re-combination for reduced complexity while maintaining the core coordinate attention mechanism of separate height and width attention. g?NrrTrFrrrrrrrrrc P>XS.n [T U]5 XlU(d[X-U-USS9n[R "X4SU0U D6UlU"5Ul[R "X14SU0U D6Ul[R "X14SU0U D6Ul [U5Ul g)a Args: channels: Number of input channels. rd_ratio: Reduction ratio for bottleneck channel calculation. rd_channels: Explicit number of bottleneck channels, overrides rd_ratio if set. rd_divisor: Divisor for making bottleneck channels divisible. se_factor: Applied to rd_ratio for final channel count (keeps param similar to SE) bias: Whether to use bias in linear layers. act_layer: Activation module class for bottleneck. gate_layer: Gate activation, either 'sigmoid', 'hardsigmoid', or a module class. has_skip: Whether to add residual skip connection to output. device: Device to place tensors on. dtype: Data type for tensors. rrr rN) r%r&rr rLinearfc1r+fc_hfc_wrr.)r/rrrrrrrrrrrr0r1s r2r&SimpleCoordAttn.__init__ps8 /  ()r+rdr?re unsqueezerfr.rr/rCrDrIrJrLrMrNs r2rOSimpleCoordAttn.forwardsfffmfffmhhtxx a 345hhtxx a 345iin&&q!,66r:iin&&q!,66r:39-- ==.C r4)r+rdrerfr.r)rQrRrSrT__doc__rSiLUrWrXrrYrrZrr[r&rOr\r]r^s@r2r`r`es#)-$)+6?"(1(1(1"# (1  (1  (1(1BII(1c4 ?23(1(1(1Tr4r`c^\rSrSrSrSS\R S\SSS4S\S\S \ S \ \RS \ \ \ \R4S \\ \RS \ 4U4SjjjrSrSrU=r$)EfficientLocalAttnzEfficient Local Attention. Lightweight alternative to Coordinate Attention that preserves spatial information without channel reduction. Uses 1D depthwise convolutions and GroupNorm for better generalization. Paper: https://arxiv.org/abs/2403.01123 FrNrr"rrrrrc >XS.n [T U]5 Xpl[R"X4US4SUS-S4UUS.U D6Ul[R"X4SU4SSUS-4UUS.U D6UlUbU"U40U D6UlU"U40U D6UlO4[R"5Ul[R"5UlU"5Ul [U5Ul g)a  Args: channels: Number of input channels. kernel_size: Kernel size for 1D depthwise convolutions. bias: Whether to use bias in convolution layers. act_layer: Activation module class applied after normalization. gate_layer: Gate activation, either 'sigmoid', 'hardsigmoid', or a module class. norm_layer: Normalization module class, None for no normalization. has_skip: Whether to add residual skip connection to output. device: Device to place tensors on. dtype: Data type for tensors. rrr8rr"r#r$groupsrN) r%r&rrr'r,r-norm_hnorm_wr)r+rr.) r/rr"rrrrrrrr0r1s r2r&EfficientLocalAttn.__init__s0 /  ii  $a( A%q)    ii  K( q()      !$X44DK$X44DK++-DK++-DK;$Z0 r4cUnURSSS9nURSSS9nURURURU555nURUR UR U555nUR U5nUR U5nX%-U-nUR(aXr-nU$Nr6T)r<r7r8)r>r+rur,rvr-r.rrjs r2rOEfficientLocalAttn.forwardsffDf)ffDf)hht{{4;;s#345hht{{4;;s#345iiniinns" ==.C r4)r+r,r-r.rrurv)rQrRrSrTrlrrmr rWrYrrZrr[rr&rOr\r]r^s@r2roros !)+6?4>"515151 51 BII 51 c4 ?23 51!bii1515151nr4roc^\rSrSrSrS S\S\S\S\S\\\ \ R4S\4 U4S jjjr S r S rU=r$) StripAttniznMinimal Strip Attention. Lightweight spatial attention using strip pooling with optional learned refinement. ruse_convr"rrrc ~>XxS.n [T U]5 X`lX lU(aS[R "X4US4SUS-S4UUS.U D6Ul[R "X4SU4SSUS-4UUS.U D6UlO4[R"5Ul[R"5Ul[U5Ul g)a Args: channels: Number of input channels. use_conv: Whether to apply depthwise convolutions for learned spatial refinement. kernel_size: Kernel size for 1D depthwise convolutions when use_conv is True. bias: Whether to use bias in convolution layers. gate_layer: Gate activation, either 'sigmoid', 'hardsigmoid', or a module class. has_skip: Whether to add residual skip connection to output. device: Device to place tensors on. dtype: Data type for tensors. rrr8rrsN) r%r&rr}rr'r,r-r)rr.) r/rr}r"rrrrr_r0r1s r2r&StripAttn.__init__ s. /   ))(!,$)1- DK)) ,K1,- DK++-DK++-DK$Z0 r4cUnURSSS9nURSSS9nURU5nURU5nURX4-5nX%-nUR(aXb-nU$ry)r>r,r-r.r)r/rCrDrIrJa_hwrNs r2rOStripAttn.forwardBswffDf)ffDf)kk#kk#yy#o ==.C r4)r,r-r.rr})Tr6FrFNN)rQrRrSrTrlrWrYrr[rrrZr&rOr\r]r^s@r2r|r|s" 6?"313131 31  31 c4 ?23 313131jr4r|)rltypingrrrr@r create_actrhelpersr normr rZr r`ror|r4r2rsl )( (#K K\HbiiHVTTnM Mr4