Αi#SSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSKrSSK r SSK r SSK r SSK r /r \RRS5r\R""\\R$5(d\ R&"5r\RR)\SSS5rSr\"\5 SrSSjrS rS \R44S jr\R84S jrSS jrg)N~z.cachepaddledatasetc[R"[5 g![a)nUR[R :waeSnAgSnAff=fN)osmakedirs DATA_HOMEOSErrorerrnoEEXIST)pathexcs U/var/www/html/banglarbhumi/venv/lib/python3.13/site-packages/paddle/dataset/common.py must_mkdirsr6s; I  99 $  %s AA  Ac^[R"5n[US5m[U4SjS5HnUR U5 M TR 5 UR 5$)Nrbc&>TRS5$)N)read)fsrmd5file..Ds affTlr)hashlibmd5openiterupdateclose hexdigest)fnamehash_md5chunkrs @rmd5filer%AsP{{}H UDA*C01GGI    rc[R"SU5R5n[U[5(a%[R"SU5R5n[ R R[U5n[ R RU5(d[ R"U5 [ R RXCcURS5SOU5n[ R RU5(a[U5U:XaU$SnSn[ R RU5(a[U5U:XGd[ R RU5(a/[RRS[U5SUS 35 Xg:aUS - nO[!S US U35e[RRS USUS35 [RRS5 ["R$"SUSSS9nUR&R)S5n U c7[+US5n [,R."UR0U 5 SSS5 O[+US5n Sn [3U 5n X- S -n U S:aU S-OS n Sn[4R6R8R;U SS9nUR=U S9HFnU RU5 US - nUR?U05 X-S:XdM5UR?U5 MH SSS5 SSS5 [ R RU5(dGM [U5U:XdGM[RRS5 [RBRE5 U$!,(df  N=f!,(df  N=f!,(df  N=f![@a nSnAGMSnAff=f)Nz^[a-zA-Z0-9_/\-]+$z^(?:(?!\.\.)[a-zA-Z0-9_/\.-])+$/rzfile z md5  zCannot download z within retry limit z Cache file z not found, downloading z zBegin to download GETT)timeoutfollow_redirectszcontent-lengthwbritem)name) chunk_sizez Download finished )#rematchgroup isinstancestrrrjoinr existsr splitr%sysstderrwrite RuntimeErrorhttpxstreamheadersgetrshutil copyfileobjrawintrhapi progressbar ProgressBar iter_bytesr Exceptionstdoutflush)url module_namemd5sum save_namedirnamefilenameretry retry_limitr total_lengthrr3 total_iter log_interval log_indexbardataes rdownloadr_Js7((0+>DDFK)S!!HH 0) %' ggll9k2G 77>>' " " Gww||'83#iH ww~~hGH$5$? EKww~~h''GH,=,G 77>>( # # JJ  uWX%6$7vfXRH I   QJE"3%';K=I  (#;C5 D  ./ sD4 yy}}-=> 'h-**155!4.-h-%) '*<'8 %1%>%B 0:RJ",Q%%& $kk55AA&VB%&LLJL$GDGGDM%NIJJy"5(71< # 9 5 %H.ww~~h''GH,=,GXJJ,-JJ O5.-.-4   sgO/-+O"N<:OB O O 4O<O/< O O O O O,(O/,O// PPc<[[R5Vs/sHoRS5(aMUPM snHRnS[[R "SU355;dM+[R "SU35R 5 MT gs snf)N__fetchzpaddle.dataset.)dirrr startswith importlib import_modulerb)xrPs r fetch_allrhs~v~~&&all4.@&  c  # #ok]$C D    # #ok]$C D J J L s BBz %05d.picklec[U5(d [S5e/nSn[U"55HMupgURU5 Xa:dMXa-S:XdM'[ X%-S5nU"XH5 /nUS- nSSS5 MO U(a [ X%-S5nU"XH5 SSS5 gg!,(df  M=f!,(df  g=f)a you can call the function as: split(paddle.dataset.cifar.train10(), line_count=1000, suffix="imikolov-train-%05d.pickle") the output files as: |-imikolov-train-00000.pickle |-imikolov-train-00001.pickle |- ... |-imikolov-train-00480.pickle :param reader: is a reader creator :param line_count: line count for each file :param suffix: the suffix for the output files, should contain "%d" means the id for each file. Default is "%05d.pickle" :param dumper: is a callable function that dump object to file, this function will be called as dumper(obj, f) and obj is the object will be dumped, f is a file object. Default is cPickle.dump. zdumper should be callable.rwr+N)callable TypeError enumerateappendr) reader line_countsuffixdumperlinesindex_fidrs rr;r;s, F  455 EG&(# Q ?q~2f&,u 1 -,$ &"C (A 5 ) ( -, ) (s%B) B;) B8 ; C c ^^^^UUUU4SjnU$)a Create a reader that yield element from the given files, select a file set according trainer count and trainer_id :param files_pattern: the files which generating by split(...) :param trainer_count: total trainer count :param trainer_id: the trainer rank id :param loader: is a callable function that load object from file, this function will be called as loader(f) and f is a file object. Default is cPickle.load c3># [T5(d [S5e[R"T5nUR5 /n[ U5H/up#UT-T :XdM[ SU35 UR U5 M1 UH*n[US5nT"U5nUShvN SSS5 M, gN!,(df  M@=f7f)Nzloader should be callable.z append file: rW)rkrlglobsortrmprintrnr) file_list my_file_listidxfnrrs files_patternloader trainer_count trainer_ids rro$cluster_files_reader..readers89 9IIm,   +GC]"j0 bT*+##B',Bb#!q    !s6AC "4C B8$B6%B8) C 6B88 C C )rrrrros```` rcluster_files_readerrs ! ! MrcU(a&[RRU5(aU$U(a*[RR R XU5$[US35e)Nz& not exists and auto download disabled)rrr:rrcommonr_ ValueError)rrOrrPr_s r_check_exists_and_downloadrsO t$$ ~~$$--cDDD6!GHIIrr)T)r ryrrerpickler4rDr<tempfiler@rpaddle.dataset__all__r expanduserHOMEaccessW_OK gettempdirr9r rr%r_rhdumpr;loadrrrrrrs     ww# yyrww    D GGLLx9 =  I AHM&36;;#N6<[[BJr