Αi.SrSSKrSSKrSSKrSSKrSSKrSSKJr /r Sr Sr Sr Sr \"SS S S S 9S 5r\"SS S S S 9S5r\"SS S S S 9S5r\"SS S S S 9S5r\"SS S S S 9S5rg)a IMDB dataset. This module downloads IMDB dataset from http://ai.stanford.edu/%7Eamaas/data/sentiment/. This dataset contains a set of 25,000 highly polar movie reviews for training, and 25,000 for testing. Besides, this module also provides API for building dictionary. N) deprecatedz6https://dataset.bj.bcebos.com/imdb%2FaclImdb_v1.tar.gz 7c2ac02c03563afcf9b574c7e56c153ac#b# [R"[RRR [ S[55nUR5nUb[URUR55(ayURU5R5RS5RS[ R"R%S55R'5R)5v UR5nUbMSSS5 g!,(df  g=f7f)zI Read files that match the given pattern. Tokenize and yield each file. imdbNs zlatin-1)tarfileopenpaddledatasetcommondownloadURLMD5nextboolmatchname extractfilereadrstrip translatestring punctuationencodelowersplit)patterntarftfs S/var/www/html/banglarbhumi/venv/lib/python3.13/site-packages/paddle/dataset/imdb.pytokenizer &s fnn++44S&#F G4 YY[nGMM"''*++$$R(TVVG_YtV%7%7%>%>y%IJUWUW Bn H G GsAD/ C D D/ D,(D/c [R"[5n[U5HnUHnX$==S- ss'M M UR 5Vs/sHoUSU:dM UPM nn[ USS9n[ [U65upx[[ [U[[U55555n [U5U S'U $s snf)zz Build a word dictionary from the corpus. Keys of the dictionary are words, and values are zero-based IDs of these words. cUS*US4$)Nr"r)xs rbuild_dict..Ls1Q4%1)key) collections defaultdictintr itemssortedlistzipdictrangelen) rcutoff word_freqdocwordr% dictionarywords_word_idxs r build_dictr=?s '',I D Oq O! &OO-?-q1-I? '>?JC$%HEDUE#e*$5678HE HW O @s  C $C z2.0.0zpaddle.text.datasets.Imdbr"z>Please use new dataset API which supports paddle.io.DataLoader)since update_tolevelreasonc\^^^TSm/mUU4SjnU"UTS5 U"UTS5 U4SjnU$)Nr*c >[U5H7nURUVs/sHnTRUT5PM snU45 M9 gs snfN)r appendget)routlabelr7wUNKr<s rloadreader_creator..load]s@G$C JJs;s!a-s;UC D%;sA rr"c3&># TShvN gN7frDr$)INSsrreaderreader_creator..readerds s r$) pos_pattern neg_patternr<rKrOrNrJs ` @@rreader_creatorrSSs@ 7 C CE c1c1 Mr(cl[[R"S5[R"S5U5$)z IMDB training set creator. It returns a reader creator, each sample in the reader is an zero-based ID sequence and label in [0, 1]. :param word_idx: word dictionary :type word_idx: dict :return: Training reader creator :rtype: callable zaclImdb/train/pos/.*\.txt$zaclImdb/train/neg/.*\.txt$rSrecompiler<s rtrainrYjs.$  01 01 r(cl[[R"S5[R"S5U5$)z IMDB test set creator. It returns a reader creator, each sample in the reader is an zero-based ID sequence and label in [0, 1]. :param word_idx: word dictionary :type word_idx: dict :return: Test reader creator :rtype: callable zaclImdb/test/pos/.*\.txt$zaclImdb/test/neg/.*\.txt$rUrXs rtestr[s.$  /0 /0 r(cB[[R"S5S5$)zQ Build a word dictionary from the corpus. :return: Word dictionary :rtype: dict z/aclImdb/((train)|(test))/((pos)|(neg))/.*\.txt$)r=rVrWr$r(r word_dictr^s   EF r(cj[RRR[S[ 5 g)Nr)r r r r r rr$r(rfetchr`s  NN""34r()__doc__r+rVrrpaddle.dataset.commonr paddle.utilsr__all__r rr r=rSrYr[r^r`r$r(rres # ?(2( ) K    " ) K    & ) K    & ) K     ) K  5  5r(