RЦiW=[ SrSSKrSSKJr SSKJr SSKJrJrJ r J r J r J r J r SSKrSSKJrJr SSKJrJr SS KJrJr SS KJr SS KJr "S S 5rSSSSSSSSSSSSSSSSSSSSSSSSSS\\SSSSSSSSSSS\R:\R<"S5SS4+S\ \ \ \\4\4S\ \ \S\ \ \ S\ \S 4S!\S"\S#\!S$\ \S%\!S&\ S'\"S(\S)\!S*\ \"S+\ \ \ \ 4S,\ \ \ \ 4S-\ S.\ S/\ S0\ \ S1\ S2\ S3\ \"S4\S5\S6\"S7\ \ S 4S8\ \ S 4S9\ \ S:\ \"S;\ \S<\S=\!S>\S?\S@\SA\SB\!SC\!SD\RFSE\ \"\R<4SF\!SG\"SH\ \RHRJRL\44XSIjjr'g)JaNaFlex data loader for dynamic sequence length training. This module provides a specialized data loader for Vision Transformer models that supports: - Dynamic sequence length sampling during training for improved efficiency - Variable patch size training with probabilistic selection - Patch-level random erasing augmentation - Efficient GPU prefetching with normalization Hacked together by / Copyright 2025, Ross Wightman, Hugging Face N)suppress)partial)CallableDictIteratorListOptionalTupleUnion)IMAGENET_DEFAULT_MEANIMAGENET_DEFAULT_STD) _worker_init adapt_to_chs)NaFlexMapDatasetWrapperNaFlexCollator)PatchRandomErasing)create_transformc\rSrSrSr\\S\R"S5SSSSS 4 S \RRRS \ \ S 4S \ \ S 4S\S\RS\\R S\ S\S\S\SS4SjjrS\\ \\\R*4\R*44SjrS\4Sjr\S5r\S5rSrg)NaFlexPrefetchLoaderz;Data prefetcher for NaFlex format which normalizes patches.cudaNconstr rloadermean.stdchannelsdevice img_dtypere_probre_modere_count re_num_splitsreturnc XlXPlU=(d [RUl[ X$5n[ X45nSSU4n X@l[R"UV s/sHoS-PM sn XPRS9RU 5Ul [R"UV s/sHoS-PM sn XPRS9RU 5Ul US:a[UUU U US9Ul OSUl URS:H=(a [RR5UlURS:H=(a [R"R5Ulgs sn fs sn f) aInitialize NaFlexPrefetchLoader. Args: loader: DataLoader to prefetch from. mean: Mean values for normalization. std: Standard deviation values for normalization. channels: Number of image channels. device: Device to move tensors to. img_dtype: Data type for image tensors. re_prob: Random erasing probability. re_mode: Random erasing mode. re_count: Maximum number of erasing rectangles. re_num_splits: Number of augmentation splits. r )r dtyper) erase_probmode max_count num_splitsr Nrnpu)rr torchfloat32r!rrtensorviewrrrrandom_erasingtyper is_availableis_cudar.is_npu) selfrrrrr r!r"r#r$r%normalization_shapexs V/var/www/html/ai-image-ml/venv/lib/python3.13/site-packages/timm/data/naflex_loader.py__init__NaFlexPrefetchLoader.__init__s56  "3emmD+3) !X. LL" #dWd #F..JJN$ObJc <<! "cWc "6IIMNaIb  R<"4""( #D #'D {{f,J1H1H1J kkU*Guyy/E/E/G # $ "s E+&E0c ## SnUR(aJ[RRURS9n[ [RR US9nOcUR(aJ[RRURS9n[ [RR US9nOSn[nURGHuupEU"5 UR5HYupg[U[R5(dM&US:Xa UROSnXFRURSUS9XF'M[ URURSS9nUSn U R n U R"S:Xa#U upn U R%XS UR&5nOuU R"S :XaLU upnnnUUR&:XdS UR&S U35eU R%XS UR&5nO[)S U R"S35eUR+UR,5R/UR05nUR2b$UR3UUSUR5SS5S9nUR%U 5US'SSS5 U(dWW4v OSnUbUR(a7[RR7URS9R9U5 OGUR(a6[RR7URS9R9U5 UnUnGMx WW4v g!,(df  N=f7f)zIterate through the loader with prefetching and normalization. Yields: Tuple of (input_dict, targets) with normalized patches. T)r )streamNpatches)r non_blockingr))r rArz Expected z channels, got z&Unexpected patches tensor dimensions: z. Expected 3 or 5. patch_coord patch_valid)rDrEF)r6r/rStreamr rr?r7r.rritems isinstanceTensorr!toshapendimr2r ValueErrorsubrdivrr3getcurrent_stream wait_stream)r8firstr?stream_contextnext_input_dict next_targetkvr)patches_tensororiginal_shape batch_size num_patches patch_pixelsr@patch_hpatch_wr input_dicttargets r;__iter__NaFlexPrefetchLoader.__iter__Vs  <<ZZ&&dkk&:F$UZZ%6%6vFN [[YY%%T[[%9F$UYY%5%5fENF%N,0KK (O!+113DA!!U\\2223y.d-<-?-B-B#';;)-"'.C.*4*nnDKKdnS "1!;!/!5!5!&&!+&&2"11$3M$B$3$7$7 t$L2G.5\\.-I *U"X &((!<<JJ--T[[-AMMfU[[II,,DKK,@LLVT(J Fs-8v&  u"!s&CM!3MFMB:M! M M!c,[UR5$)zPGet length of underlying loader. Returns: Number of batches in the loader. )lenrr8s r;__len__NaFlexPrefetchLoader.__len__s 4;;c.URR$)zZGet sampler from underlying loader. Returns: Sampler from the underlying DataLoader. )rsamplerrfs r;rkNaFlexPrefetchLoader.sampler{{"""ric.URR$)zZGet dataset from underlying loader. Returns: Dataset from the underlying DataLoader. )rdatasetrfs r;roNaFlexPrefetchLoader.datasetrmri) rr r!r6r7rrr3r)__name__ __module__ __qualname____firstlineno____doc__r rr/r utilsdata DataLoaderr floatintr r)strr<rrrIrbrgpropertyrkro__static_attributes__rir;rrs<E '<%9#(<<#7/3"!"6HKK$$//6Hs #6Hucz" 6H  6H LL 6H  ,6H6H6H6H6H6HpL!(5c5<<.?)@%,,)N#OPL!\  ####rir)@iir Frrg?g?bilinear*Trall patch_sizepatch_size_choicespatch_size_choice_probstrain_seq_lens. max_seq_lenr[ is_trainingmixup_fnno_augr"r#r$re_splittrain_crop_modescaleratiohflipvflip color_jittercolor_jitter_probgrayscale_probgaussian_blur_prob auto_augmentnum_aug_repeatsnum_aug_splits interpolationrrcrop_pct crop_modecrop_border_pixels num_workers distributedrank world_sizeseedepochuse_prefetcher pin_memoryr!r persistent_workersworker_seedingr&c,U(GaUS:XdS5e[[40SS_SU _SU_SU_SU_S U_S U_S U_S U_S U_SU_SU_SU_SU_SU_SU_SU_SU_SU _SU _SU _SU&_SS_6n,[U5n-UU--n.[U[R R R5(aS5e[UU,UUUUU.UU$U!U"U#SU%S9n/[R R RU/SSU SU'[[U+S9U*S 9n0U&(a[U0UUU(U)U U U S!9n0U0$[SUUUU&SUUSS"9 Ul [US#9n1Sn2U!(aA[U[R R R5(dSS$KJn3 U3"U5n2[R R RUUSU U2U1U'SS%9n0U&(a [U0UUU(U)S&9n0U0$)'aCreate a data loader with dynamic sequence length sampling for training. Args: dataset: Dataset to load from. patch_size: Single patch size to use. patch_size_choices: List of patch sizes for variable patch size training. patch_size_choice_probs: Probabilities for each patch size choice. train_seq_lens: Training sequence lengths for dynamic batching. max_seq_len: Fixed sequence length for validation. batch_size: Batch size for validation and max training sequence length. is_training: Whether this is for training (enables dynamic batching). mixup_fn: Optional mixup function. no_aug: Disable augmentation. re_prob: Random erasing probability. re_mode: Random erasing mode. re_count: Maximum number of erasing rectangles. re_split: Random erasing split flag. train_crop_mode: Training crop mode. scale: Scale range for random resize crop. ratio: Aspect ratio range for random resize crop. hflip: Horizontal flip probability. vflip: Vertical flip probability. color_jitter: Color jitter factor. color_jitter_prob: Color jitter probability. grayscale_prob: Grayscale conversion probability. gaussian_blur_prob: Gaussian blur probability. auto_augment: AutoAugment policy. num_aug_repeats: Number of augmentation repeats. num_aug_splits: Number of augmentation splits. interpolation: Interpolation method. mean: Normalization mean values. std: Normalization standard deviation values. crop_pct: Crop percentage for validation. crop_mode: Crop mode. crop_border_pixels: Crop border pixels. num_workers: Number of data loading workers. distributed: Whether using distributed training. rank: Process rank for distributed training. world_size: Total number of processes. seed: Random seed. epoch: Starting epoch. use_prefetcher: Whether to use prefetching. pin_memory: Whether to pin memory. img_dtype: Image data type. device: Device to move tensors to. persistent_workers: Whether to use persistent workers. worker_seeding: Worker seeding mode. Returns: DataLoader or NaFlexPrefetchLoader instance. rz=Augmentation repeats not currently supported in NaFlex loaderrTrrrrrrrrrrrrrrrrrr"r#r$rnaflexFz IterableDataset Wrapper is a WIP) transform_factoryrrrseq_lensmax_tokens_per_batchrrrrrshufflerN)r)r[rrrkrworker_init_fnr)rrr!r r"r#r$) rrrrrrrrpatchify)r)OrderedDistributedSampler)r[rrrk collate_fnr drop_last)rrr!r )rrmaxrHr/rvrwIterableDatasetrrxrr transformrtimm.data.distributed_samplerr)4rorrrrrr[rrrr"r#r$rrrrrrrrrrrrrrrrrrrrrrrrrrrr!r rrrmax_train_seq_lenrnaflex_datasetrrrkrs4 r;create_naflex_loaderrsH!#d%dd##    ,        & 0 *  2 & (  ! "# $ % & 2' () *+ ,- .*/ 01 6 /),== gu{{//?? @ @ << <50 /!1$;#!5#! $!!,, #!"<O1-   )#! Fp MW-')!#  $ <  z'5;;3C3C3S3STT O/8G!!,, !#!!-   )# F Mri)(rumath contextlibr functoolsrtypingrrrrr r r r/ constantsr rrrrrrrnaflex_random_erasingrtransforms_factoryrrr0r rzryboolr{r)rvrwrxrr~rir;rs  III B.C50a#a#L=A269=*D!'+)-/3/3!-1 "$&&* '"7!5$(#',0!#!&+0<<+?#'#]_U5c?C#789_%T#Y/_"*$u+!6 _ c3h _  ___8$______ "#!_"eUl+,#_$eUl+,%_&'_()_*+_,$E?-_./_0"1_2sm3_45_67_89_:E3J;_<5#: =_>5/?_@C=A_B%SMC_FG_HI_JK_LM_NO_PQ_RS_TU_V;;W_Xc5<<'(Y_Z![_\]_^ u{{**,@@ A__ri