Ƒi%SrSSKrSSKJrJr SrSrSrSrSr S r S \ S \S-4S jr S \ S \S-4S jr S \ S \S-4SjrS\S \4SjrSS\S\S \4Sjjrg)aStage 1a+: UTF-16/UTF-32 detection for data without BOM. This stage runs after BOM detection but before binary detection. UTF-16 and UTF-32 encoded text contains characteristic null-byte patterns that would otherwise cause binary detection to reject the data. Note: ``from __future__ import annotations`` is intentionally omitted because this module is compiled with mypyc, which does not support PEP 563 string annotations. N)DETERMINISTIC_CONFIDENCEDetectionResulti gQ??gffffff?datareturncrUS[n[U5[:ag[U5nUbU$[ U5$)aDetect UTF-32 or UTF-16 encoding from null-byte patterns. UTF-32 is checked before UTF-16 since UTF-32 patterns are more specific. :param data: The raw byte data to examine. :returns: A :class:`DetectionResult` if a strong pattern is found, or ``None``. N) _SAMPLE_SIZElen_MIN_BYTES_UTF16 _check_utf32 _check_utf16)rsampleresults X/var/www/html/banglarbhumi/venv/lib/python3.13/site-packages/chardet/pipeline/utf1632.pydetect_utf1632_patternsr*sD-< F 6{%%& !F    c ^[T5[T5S-- nU[:agTSUmUS-n[U4Sj[S[T5S555n[U4Sj[S[T5S555nX2:Xa:XB- S:a2TR S5n[ U5(a[ S[SS9$[U4S j[S [T5S555n[U4S j[S [T5S555nXb:Xa;Xr- S:a3TR S 5n[ U5(a[ S [SS9$gg![a Nf=f![a gf=f)avCheck for UTF-32 encoding based on 4-byte unit structure. For valid Unicode (U+0000 to U+10FFFF = 0x0010FFFF): - UTF-32-BE: the first byte of each 4-byte unit is always 0x00 - UTF-32-LE: the last byte of each 4-byte unit is always 0x00 For BMP characters (U+0000 to U+FFFF), additionally: - UTF-32-BE: the second byte is also 0x00 - UTF-32-LE: the third byte is also 0x00 Nc3>># UHnTUS:XdMSv M g7frN.0irs r _check_utf32..TJ#9aT!W\#9  rc3D># UHnTUS-S:XdMSv M g7f)rrNrrs rrrVs$O$:qd1q5kQ>N$:s  rz utf-32-beencoding confidencelanguagec3>># UHnTUS:XdMSv M g7frrrs rrresI"8QDGqLqq"8r!c3>># UHnTUS:XdMSv M g7frrrs rrrgr r!z utf-32-le) r _MIN_BYTES_UTF32sumrangedecode_looks_like_textrrUnicodeDecodeError)r trimmed_len num_units be_first_nullbe_second_nulltext le_last_null le_third_nulls` rrr@svd)s4y1}-K%%   Dq IJ5CIq#9JJMOE!SY$:OON!n&@3&F ;;{+D%%&(7!&I%3t9a"8IILJ5CIq#9JJM ]%>%D ;;{+D%%&(7!& )"   ""    s$/E"/E% E"!E"% E21E2c^[[T5[5nXS--nU[:agUS-n[ U4Sj[ SUS555n[ U4Sj[ SUS555nX2- nXB- n/nU[ :aURSU45 U[ :aURSU45 U(dg[U5S:Xa>USSnTSURU5n [U 5(a[U[SS 9$gSn S n UH0upTSURU5n [U 5n X:dM,U n Un M2 U bU [:a[U [SS 9$g![a gf=f![a Mnf=f) aCheck for UTF-16 via null-byte patterns in alternating positions. UTF-16 encodes each BMP character as two bytes. For characters whose code-point high byte is 0x00 (Latin, digits, basic punctuation, many control structures), one of the two bytes in each unit will be a null. Even for non-Latin scripts (Arabic, CJK, Cyrillic, etc.) a significant fraction of code units still contain at least one null byte. Non-UTF-16 single-byte encodings never contain null bytes, so even a small null-byte fraction in alternating positions is a strong signal. When both endiannesses show null-byte patterns (e.g., Latin text where every other byte is null), we disambiguate by decoding both ways and comparing text-quality scores. r*Nc3>># UHnTUS:XdMSv M g7frrrs rr_check_utf16..K#:ad1gl#:r!rc3>># UHnTUS:XdMSv M g7frrrs rrr:r;r!rz utf-16-lez utf-16-ber#)minr r r r,r-_UTF16_MIN_NULL_FRACTIONappendr.r/rrr0 _text_quality_MIN_TEXT_QUALITY)r sample_lenr2 be_null_count le_null_countbe_fracle_frac candidatesr$r5 best_encoding best_quality_qualitys` rrrxs SY -Jq. J$$aIK5J#:KKMK5J#:KKM'G'G*,J**;01**;01  :!a=#  $++H5D%%&%7!&!%ML!   $++H5D %  !"L$M" \5F%F"/  5"   "   s$2E"E2" E/.E/2 F?Fr5cjU(dgUSSn[SU55nU[U5- [:$)z9Quick check: is decoded text mostly printable characters.FNc3^# UH#oR5(dUS;dMSv M% g7f) rN) isprintable)rcs rr#_looks_like_text..s Jv!AMAAvs- -)r,r _MIN_PRINTABLE_FRACTION)r5r printables rr/r/s8  $3ZFJvJJI s6{ "%< <10% control characters or >20% combining marks (category ``M*``). Nrr=LrMZsrPCg?g?r)r unicodedatacategoryord) r5rVrnlettersmarksspacescontrols ascii_lettersrRcatscores rrArAs(&5\F F AAvG E FHM ""1% q6S= qLG1v|"  Vs] QJE D[AM aKF Vs] MH|c y3 KE a3 &&E2v&1*   Lr)rN)__doc__r^chardet.pipelinerrr r+r r?rBrTbytesrrrstrboolr/intfloatrArrrrps F   % Od,B ,5u54!75pMuM4!7M`=3=4=99C9%9r