RЦi"HSrSSKJrJr SSKrSSKJr SSKJs Jr SS\RS\ S\ S\ S\ S \ S \ S \ 4S jjr"S S\R5rSS\ S\ S \ 4Sjjr"SS\R5rSS\ S\\ \\ 4S\ S\\\ \\\ 44Sjjrg)a=DropBlock, DropPath PyTorch implementations of DropBlock and DropPath (Stochastic Depth) regularization layers. Papers: DropBlock: A regularization method for convolutional networks (https://arxiv.org/abs/1810.12890) Deep Networks with Stochastic Depth (https://arxiv.org/abs/1603.09382) Code: DropBlock impl inspired by two Tensorflow impl that I liked: - https://github.com/tensorflow/tpu/blob/master/models/official/resnet/resnet_model.py#L74 - https://github.com/clovaai/assembled-cnn/blob/master/nets/blocks.py Hacked together by / Copyright 2020 Ross Wightman )ListUnionNx drop_prob block_size gamma_scale with_noiseinplacecouple_channels scale_by_keepc URupp[X*5[X+5p[X1-U -U -5[X-5- [X- S-X- S--5- nX(aSOU X4n[R"5 [R "XR URS9RU5n[R"UX4SU S-U S-4S9nU S-S:Xd U S-S:XaUSU S-S-S2U S-S-S24nSU- nSSS5 U(a[R"5 [R"W5R5nURW5 SSS5 U(a"URW5RW5 U$UW-W-nU$U(a[R"5 WR5UR![R"S 9R%5R'S 5- nURUR!UR 55 SSS5 U(aURW5 U$UW-nU$!,(df  GNX=f!,(df  GN=f!,(df  NS=f) aDropBlock. See https://arxiv.org/pdf/1810.12890.pdf DropBlock with an experimental gaussian noise option. Args: x: Input tensor of shape (B, C, H, W). drop_prob: Probability of dropping a block. block_size: Size of the block to drop. gamma_scale: Scale factor for the drop probability. with_noise: If True, add gaussian noise to dropped regions instead of zeros. inplace: If True, perform operation in-place. couple_channels: If True, all channels share the same drop mask (per the original paper). If False, each channel gets an independent mask. scale_by_keep: If True, scale kept activations to maintain expected values. Returns: Tensor with dropped blocks, same shape as input. )dtypedevice) kernel_sizestridepaddingr.N?)rgHz>)shapeminfloattorchno_grademptyrr bernoulli_F max_pool2d empty_likenormal_mul_add_numeltofloat32sumadd)rrrrr r r r BCHWkhkwgamma noise_shape block_mask keep_masknoisenormalize_scales O/var/www/html/ai-image-ml/venv/lib/python3.13/site-packages/timm/layers/drop.py drop_block_2dr5s08JA!  Z!3 +)A-1 2U27^ CeQVVWZ\]\bef\fLgFh hE?a18K [[GGAHHMXXY^_ \\ 1WbAg&   6Q;"q&A+#C"q&Aa1 $EFJO  " ]]_$$Y/779E JJz "  FF9  " "5 ) HI %A H "+//"3illl6W6[6[6]6a6abf6g"g11!'':;!  FF9  HI A HM $_!s&BI-6I,;A6I> I), I;> J cd^\rSrSrSrS S\S\S\S\S\S\S \4U4S jjjrS r S r U=r $) DropBlock2dfa8DropBlock. See https://arxiv.org/pdf/1810.12890.pdf Args: drop_prob: Probability of dropping a block. block_size: Size of the block to drop. gamma_scale: Scale factor for the drop probability. with_noise: If True, add gaussian noise to dropped regions instead of zeros. inplace: If True, perform operation in-place. couple_channels: If True, all channels share the same drop mask (per the original paper). If False, each channel gets an independent mask. scale_by_keep: If True, scale kept activations to maintain expected values. rrrr r r r c >[T U]5 XlX0lX lX@lXPlX`lXplSS1n UH#n X;dM SSK n U RSU S35 M% g)N batchwisefastrz/DropBlock2d() got unexpected keyword argument '') super__init__rrrr r r r warningswarn) selfrrrr r r r kwargsdeprecated_argskr? __class__s r4r>DropBlock2d.__init__tsm "&$$ .*'/A' OPQsRSTUc UR(aUR(dU$[UURURURUR UR URURS9$)N)rrrr r r r ) trainingrr5rrr r r r rArs r4forwardDropBlock2d.forwards[}}DNNH nn((LL 00,,  rG)rr rrr r r g?rFFTT) __name__ __module__ __qualname____firstlineno____doc__rintboolr>rK__static_attributes__ __classcell__rEs@r4r7r7fs  #!$$!$("&VVV V  V  V"V VV6   rGr7rIcUS:XdU(dU$SU- nURS4SURS- --nURU5RU5nUS:aU(aUR U5 X-$)aDrop paths (Stochastic Depth) per sample (when applied in main path of residual blocks). This is the same as the DropConnect impl I created for EfficientNet, etc networks, however, the original name is misleading as 'Drop Connect' is a different form of dropout in a separate paper... See discussion: https://github.com/tensorflow/tpu/issues/494#issuecomment-532968956 ... I've opted for changing the layer and argument names to 'drop path' rather than mix DropConnect as a layer name and use 'survival rate' as the argument. rr)r)rndim new_emptyrdiv_)rrrIr keep_probr random_tensors r4 drop_pathr`suBhI I WWQZMDAFFQJ/ /EKK&11)[TU]5 XlX lgN)r=r>rr )rArr rEs r4r>DropPath.__init__s "*rGcX[XRURUR5$re)r`rrIr rJs r4rKDropPath.forwardsNNDMM4;M;MNNrGc6S[URS5S3$)Nz drop_prob=z0.3f)roundr)rAs r4 extra_reprDropPath.extra_reprsE$..3D9::rG)rr )rZT) rOrPrQrRrSrrUr>rKrlrVrWrXs@r4rbrbs0+%+T++ O;;rGrbdrop_path_ratedepths stagewisereturnc[U[5(aGU(a [S5e[R"SXSS9Vs/sHo3R 5PM nnU$[ U5nU(aD[R"SXSS9RU5Vs/sHo3R5PM nnU$[R"SXSS9Vs/sHo3R 5PM nnU$s snfs snfs snf)aGenerate drop path rates for stochastic depth. This function handles two common patterns for drop path rate scheduling: 1. Per-block: Linear increase from 0 to drop_path_rate across all blocks 2. Stage-wise: Linear increase across stages, with same rate within each stage Args: drop_path_rate: Maximum drop path rate (at the end). depths: Either a single int for total depth (per-block mode) or list of ints for depths per stage (stage-wise mode). stagewise: If True, use stage-wise pattern. If False, use per-block pattern. When depths is a list, stagewise defaults to True. Returns: For per-block mode: List of drop rates, one per block. For stage-wise mode: List of lists, drop rates per stage. z;stagewise=True requires depths to be a list of stage depthsrcpu)r) isinstancerT ValueErrorrlinspaceitemr&splittolist)rnrorprdpr total_depths r4calculate_drop_path_ratesr|s,&# Z[ [!&>RW!XY!XAvvx!XY &k ',~~a]b'c'i'ijp'qr'q!88:'qCrJ&+^^A~[`%ab%a668%aCbJZscsC(C- C2rM)rZFT)F)rStypingrrrtorch.nnnntorch.nn.functional functionalrTensorrrTrUr5Moduler7r`rbr|rGr4rs      $"K <<K K K  K  K  K K K \5 "))5 pE$t( ;ryy ;$ &&c49n%&& 4;T%[) )* &rG