*pj*SrSSKJrJrJrJrJrJr SSKJ r SSK J r J r J r SSKJrJrJr SSKJr \S\ --rSrS \S \\S 4S \S \\\\\S-44SjrS \\S 4S \S \\\\\S-\\\4\\\\\\S-\\444SjrS \\S 4S \S \\\\\S-44SjrSS.S \S \\S 4S\S \\ 4Sjjr g)zStage 3: Statistical bigram scoring. Note: ``from __future__ import annotations`` is intentionally omitted because this module is compiled with mypyc, which does not support PEP 563 string annotations. ) BigramProfile_get_model_norms get_enc_index get_rowmaxscore_best_languagescore_with_profile)DetectionResult)_CONFUSION_BAND_CONFUSION_FLOOR_RATIO_STRICT_TIER_MAX_CONF)_COMMON_LATIN_ENCODINGS_DEMOTION_CANDIDATES_RARE_ARBITRATION_MARGIN) EncodingInfo@data candidates.profilereturnNc/nUH>n[XRUS9upVUS:dM!URURXV45 M@ U$)zFScore every candidate fully (no pruning). Returns (enc, score, lang).)r)rnameappend)rrrscoresencslangs X/var/www/html/pdf-tiff/venv/lib/python3.13/site-packages/chardet/pipeline/statistical.py _score_allr +sH 35F%dHHgF s7 MM388Q- . Mc [5n[5n[5nURnURnUR n/n/n UHn UR U R5n U cM#U R(a8[U 5H'un upnURU RXX45 M) Ml[U 5Hvun upnXOnSnUHnUUUUU-- nM UR U5nUc [S5nOUS:a UUU-- nOSnU RUXRXU45 Mx M U RSSS9 X4$)uZFlatten candidate model variants for pruned scoring. Returns ``(mb_entries, sb_entries)`` where multi-byte entries are ``(enc, lang, table, key, variant_index)`` and single-byte entries are ``(upper_bound, variant_index, enc, lang, table, key)`` sorted by descending bound. The upper bound multiplies each lead byte's total profile weight by the model's maximum weight for that lead byte — at most 256 terms versus one term per distinct bigram for a full score. ``variant_index`` is the variant's position in the encoding index, so exact score ties resolve to the same variant the full path keeps. rinfrc US$)Nr)es r!_split_variants..ls!A$r!Tkeyreverse) rrrrow_freq nonzero_rows input_normgetr is_multibyte enumeraterfloatsort)rrindexnormsrowmaxr,r-r. mb_entries sb_entriesrvariantsvirtabler*rmub_dotb1 model_normubs r_split_variantsrA9s^$ OE  E \FH''L##J@BJGIJ99SXX&      *3H*=&&T#!!388T#"BC+> &/&9 "B"cBF""R&8B<//#3J!5\c!zJ67   r2xxcB C':0OOO5  !!r!c d^^^^^^[5n[X5up40m0m0mSmSmSmS[S[S[S-S[SS4 UUUUUU4S jjnUHupgpn U"U[ XU 5Xz5 M UHHuppgpT[ - n T[:a[U T[-5n X:a OU"U[ XU 5Xz5 MJ T[ - n T[:a[U T[-5n TR5VVs/sHupX:dM UPM nnn/n[S U55(aUR[5 S U;aURS 5 U(akUHenURU;aM[!UR#UR/55H'un upxn U"UR[ XU 5Xz5 M) Mg UVs/sHNnTR#URS5S:dM%URTURTUR4PMP sn$s snnfs snf) uFScore candidates, skipping single-byte variants that provably cannot matter. Multi-byte variants are always scored fully — the orchestrator may later boost their confidence based on structural coverage, so no raw-score bound can rule them out. Single-byte variants are scored in descending upper-bound order (see :func:`_split_variants`) and skipped once their bound falls more than ``_PRUNE_MARGIN`` below the running second-best encoding score: such variants can affect neither the winner, nor position 1, nor any candidate within the confusion band of the top score. Encodings that ``postprocess_results`` inspects regardless of rank (the common Western Latin trio for niche-Latin demotion, KOI8-T for the KOI8-R promotion) are force-scored when their trigger could fire. Because confusion resolution can promote position 1 or any candidate within the band into position 0 before those triggers are evaluated, the trigger check covers every encoding near the top, not just the statistical winner. Returns (enc, score, lang) tuples for encodings scoring above zero, in candidate order. renc_namerrNr:rc>TRU5nUbX:dX:Xa UTU:agUTU'UTU'UTU'UT :XaUmgUT:aTm UmUm gUT :aUm ggN)r/) rDrrr:prev best_lang best_scorebest_vitop1top1_enctop2s rrecord_score_pruned..records~~h'  aiB'(BS _score_pruned..s 7h$ $hszkoi8-rzkoi8-t)rrAstrr2intr _PRUNE_MARGINr minr itemsanyextendr rrr1r/)rrr4r7r8rNrDrr;r*r:r@ threshold trigger_floorr&rnear_topforcedrrHrIrJrKrLrMs @@@@@@r _score_prunedr_ps/4 OE,ZAJ#%J')I GH D DcDjcd&+5&Bx+GC@$K+5/9*=( ' 'It.D'DEI > x+GC@$K/98=(M ##M42H+HI (..0G0daA4F0HGF 7h 777 -.8 h Cxxv%+4EIIchh4K*L&&T#sxx!3GC!H$S+M  C >>#((C (3 . >:chh'388)<= H s< H' H'$H-7-H-F) full_rankingr`c HU(aU(d/$[U5nURS:Xa/$U(d[UR5[:a [ XU5nO [ X5nURSSS9 UVVVs/sHupVn[XVUS9PM snnn$s snnnf)amScore all candidates and return results sorted by confidence descending. :param data: The raw byte data to score. :param candidates: Encoding candidates to evaluate. :param full_ranking: When ``True``, score every candidate fully so the returned list is complete (needed by ``detect_all``). When ``False`` (the default), single-byte candidates that provably cannot affect the top of the ranking may be skipped; the winner, position 1, and all candidates within the confusion band of the top score are identical to the full ranking. :returns: A list of :class:`DetectionResult` sorted by confidence. rc US$)Nr%)xs rr'"score_candidates..sadr!Tr))encoding confidencelanguage) rr.lennonzero_MIN_NONZERO_FOR_PRESCREENr r_r3r )rrr`rrrrrs rscore_candidatesrls$ z D!GS  s7??+.HHDg6z3 KKNDK1$ #MDT dC#  sB)!__doc__chardet.modelsrrrrrrchardet.pipeliner chardet.pipeline.confusionr r r chardet.pipeline.postprocessr rrchardet.registryrrVrkbytestuplelistrTr2r rUrAr_boolrlr%r!rrws-  *)1+>>     lC'(   %UC$J& '( 4"lC'(4" 4" sC$JsC/ 01uc3d E36 78:4"nnlC'(n n %UC$J& '(nj " "lC'(" "  / "r!