{-j ddlZddlZddlZddlZddlmZddlmZddlm Z m Z m Z m Z m Z ddlZddlmZddlmZmZmZddlmZmZd Z dMd e e eefd e e eefd edefdZdejde ejejffdZde ee e fdede ee e ffdZ dZ!dZ"dZ#dNdZ$dOdZ%dZ&dZ'dZ(Gdd eZ)Gd!d"eZ*d#Z+d$Z,d%Z-d&Z.d'Z/d(Z0d)Z1ej2e1d*e1d+ej3,Z4d-efd.Z5d/Z6d0e*fd1Z7d2edefd3Z8d4efd5Z9d-ede edffd6Z: dPd-ed9ed:ede e eeefdffd;Z; dQd>ed?ed@ed9edAedef dBZZ?ej2dEej>Z@ej2dFZAej2dFZBdGejde ee ffdHZCdIedJedKede ee ee fffdLZDdS)RN)Counter)deepcopy)AnyDictListTupleUnion)Image) BaseModelcomputed_fieldmodel_validator)calculate_overlap_ratio"calculate_projection_overlap_ratioc>|js|d}|S)Nr)is_validbuffer)polys n/var/www/html/banglarbhumi/venv/lib/python3.11/site-packages/paddlex/inference/pipelines/paddleocr_vl/uilts.py make_validr!s ={{1~~ Kunionpolygon1polygon2modereturnc ddlm}n#t$rtdwxYw||}||}t|}t|}||j}||j}|dkr||z S|dkrt|j|j}||z S|dkrt|j|j} || z Std|)a Calculate the overlap ratio between two polygons. Args: polygon1 (List[Tuple[int, int]]): First polygon represented as a list of points. polygon2 (List[Tuple[int, int]]): Second polygon represented as a list of points. mode (str, optional): Overlap calculation mode. Defaults to "union". Returns: float: Overlap ratio value between 0 and 1. r)PolygonzPlease install Shapely library.rsmalllargezUnknown mode: ) shapely.geometryr ImportErrorr intersectionarearminmax ValueError) rrrrpoly1poly2r#r small_area large_areas rcalculate_polygon_overlap_ratior,'s! =,,,,,,, ===;<<<= GH  E GH  E u  E u  E%%e,,1L KK   #E we## UZ00 j(( UZ00 j((0$00111s #coordsc>|dddf}|dddf}|dddf}|dddf}tj||z ||z z}tj|dddf|dddf}tj|dddf|dddf}tj|dddf|dddf}tj|dddf|dddf} tjd||z } tjd| |z } | | z} tj|dddf|dddf} tjdd5tj| dk| | z d}dddn #1swxYwY||fS) z Compute pairwise overlap ratio (mode="small") and bbox areas. This only replaces the numeric overlap/area calculation. The caller keeps the original sequential dropping order because it affects inline formulas. Nrrignore)divideinvalidg)npabsmaximumminimumerrstatewhere)r-x1y1x2y2areasinter_x1inter_y1inter_x2inter_y2inter_winter_h inter_arear*overlaps r_compute_pairwise_overlap_smallrGMs 1B 1B 1B 1B FBGR( ) )Ez"QQQW+r$'{33Hz"QQQW+r$'{33Hz"QQQW+r$'{33Hz"QQQW+r$'{33HjHx/00GjHx/00G7"JE!!!T'NE$'N;;J Hh 7 7 7II(:>: +BCHHIIIIIIIIIIIIIII E>s&FFFlayout_det_reslayout_shape_modect|}d|dD}|s||d<|Stjd|Dtj}|dddf|dddfz }|dddf|ddd fz }t |\}}t t t|D]} || d ks || d kr| t | d zt|D]W} | vs| vr || | f} || d d ks|| d d krU| d krO|| d d kr| || d d kr| | dkr|dkr:d|| vr0t|| d|| dd} | dkr|| d || d h} | hdzr!t| d krd| vs| hdkr|| || kr| B| Yfdt|D|d<|S)a. Remove overlapping boxes from layout detection results based on a given overlap ratio. Args: layout_det_res (Dict[str, List[Dict]]): Layout detection result dict containing a 'boxes' list. Returns: Dict[str, List[Dict]]: Filtered dict with overlapping boxes removed. c*g|]}|ddk|S)label reference.0boxs r z(filter_overlap_boxes..ys-   3w<;;V;V;V;V;Vrboxescg|] }|d S) coordinaterNrOs rrRz(filter_overlap_boxes..s:::Ss<(:::rdtypeNrrr0r/rLinline_formula?gffffff?rectpolygon_pointsr>sealchartimagetabler`c"g|] \}}|v | SrNrN)rPidxrQdropped_indexess rrRz(filter_overlap_boxes..s.(((S?0J0J0J0J0Jr) rr4arrayfloat64rGsetrangelenaddr, enumerate)rHrIlayout_det_res_filteredrSr-widthsheightsoverlap_matrixr>ij overlap_ratiopoly_overlap_ratiolabelsrcs @rfilter_overlap_boxesrtlsA'~66  .w7   E '+0(&& X::E:::"* M M MF AAAqD\F111a4L (FQQQTlVAAAqD\)G;FCCNEeeO 3u::  $+$+ !9q==GAJNN    " " "q1uc%jj))! +! +AO##qO';';*1a40Ma!%5558G$(888 3&&Qx(,<<<'++A...Qx(,<<<'++A...s""$..3CuQx3O3O)Ha!12E!H=M4NPW**&*C// (7+U1Xg->?????!CKKRSOOf,,;;;11 !8uQx''#''****#''***C! +D((((%e,,(((G$ #"rcbt|tjr|Stj|S)z Convert the input to a PIL Image. Args: img (PIL.Image or numpy.ndarray): Input image. Returns: PIL.Image: PIL Image object. ) isinstancer fromarrayimgs r to_pil_imagerzs-#u{## ?3  rcbt|tjrtj|S|S)z Convert the input to a numpy array. Args: img (PIL.Image or numpy.ndarray): Input image. Returns: numpy.ndarray: Numpy array image. )rvr r4rdrxs r to_np_arrayr|s+#u{##x}} Jrcvd|D}d|D}t|}t|}||fS)z Calculate width (max of all) and height (sum) for a vertical merge of images. Args: images (List[PIL.Image or np.ndarray]): List of images. Returns: Tuple[int, int]: (width, height) of merged image. c6g|]}t|jSrN)rzwidthrPrys rrRz"calc_merged_wh..s# 8 8 8#l3% 8 8 8rc6g|]}t|jSrN)rzheightrs rrRz"calc_merged_wh..s#:::C|C  ':::r)r&sum)imagesrlrmwhs rcalc_merged_whrsI9 8 8 8 8F::6:::G F A G A a4Krcenterautoc|sdSt|dkrt|dSt|tr|gt|dz z}t|t|dz krt dd|D}dgt|z}|dj}t dt|D]}||j}t||}||dz } | dkr||z dz} ||z dz} n| dkr ||z } ||z } nd} d} t |D]} || xx| z cc<| ||<|}td |D} tj d || fd }d}t|D]-\}}| ||||f||j z }.t|S) a+ Merge images vertically with given alignment. Args: images (List[PIL.Image or np.ndarray]): List of images to merge. aligns (str or List[str]): Alignment(s) for each merge step ('center', 'right', 'left'). Returns: np.ndarray: Merged image as numpy array. Nr/rz,The length of aligns must be len(images) - 1c,g|]}t|SrN)rzrs rrRz merge_images..s 666,s##666rrrrightc3$K|] }|jV dSN)rrs r zmerge_images..s$33#*333333rRGBrr)rhr|rvstrr'rrgr&rr newrjpaster)ralignsrI pil_images x_offsetsmerged_wroimg2_wstep_walignr:r<ktotal_hcanvasy_offsetrys r merge_imagesrs$ t 6{{a6!9%%%&#.S[[1_- 6{{c&kkAo%%GHHH76v666Jc*oo%I!}"H 1c*oo & &A$Xv&&q1u  H  8#)B6/a'BB g  ("B&BBBBq  A aLLLB LLLL ! 33 33333G Yux1? C CFHJ''3 S9Q<2333CJ v  rc> )*g}i}tD],\}}|dvr|||<|||f-g}g}g} g} d))fd} fd} t|D]\} \}}|s |g}|g} g} || dz \}}|d}|d}|d}|d}t||d}|dkos|d kom||kog|d|d koU|d|d koC|d|d z t|d |dz |d |dz d zk}|dko|d vo||ko|d |dkot |d|d z t|d |dz |d |dz dzko=)|d|d)|d |d z o | ||}|rd}n|r | ||}nd}|s|rA||| || ||| | f|g}|g} g} |r|| | fg}|D]<\}}t |t|}}|||||f=g}t *d}|tkr:d}|D]\}}}}||krt*fd|Drd}fd|D}|r|ng} t|\}!}"|!dkr|"|!z ntd}#|#d krpt|D]_\}$}%|% }|%d|d<d|d<||* |%`nt|| |}&t|D]n\}$}%|% }|$dkr|&nd|d<|$dkr| nd|d<|d|d<||* |%og}'t|dz|D]}(|(|vr|'|(|'D]2}(|||(* |(3|dz}n|r||vr4|*vr0|||* ||dz }|tk:|S)aK Merge blocks based on alignment and overlap logic, except for those with labels in non_merge_labels. Args: blocks (List[Dict]): List of block dicts. non_merge_labels (List[str]): Block labels that should not be merged. Returns: List[Dict]: List of processed (and possibly merged) blocks. rLc.t||z dkS)Nr5)a1a2s r is_alignedz merge_blocks..is_aligned4s27||q  rcp|d|drdS|d|drdSdS)NrleftrrrrN) block_bbox prev_bboxrs r get_alignmentz#merge_blocks..get_alignment7sJ :jmYq\ 2 2 6 Z 1 y| 4 4 78rc||d}||d}t|d|d}t|d|d}t|d|d}t|d|d}||||g} t|D]5\} } | ||fvs | d vr| d} t| | dkrdS6dS) NrQrr/rr0rLTF)r%r&rjr) block_idxprev_idxblocksrrr:r;r<r=min_boxrb other_block other_bboxnon_merge_labelss roverlapwith_other_boxz+merge_blocks..overlapwith_other_box?s8$U+ I&u- 1z!} - - 1z!} - - 1z!} - - 1z!} - -r2r" )& 1 1   C 8,,,w'/???$U+J&w ;;a??tt@urr/rQ horizontalrtextrr0g333333?)rrZrNFc3 K|]}|vV dSrrN)rPro used_indicess rrzmerge_blocks..s(#Q#QaA\$9#Q#Q#Q#Q#Q#QrTc,g|]}|dSrxrN)rPrors rrRz merge_blocks..s"@@@Qq %(@@@rinfry merge_alignsgroup_id)rjappendrr&r5r%rfrhallrfloatcopyrirrg)+rrrIblocks_to_mergenon_merge_blocksrbblock merged_groups current_groupcurrent_indicescurrent_alignsrrror prev_blockr prev_labelr block_labeliou_his_crossis_updown_align align_mode group_ranges group_indicesrstartend result_blocks group_foundimgsrrr aspect_ratiorpr merged_img insert_listn_idxrrs+`` @@r merge_blocksrsO''11 U >- - -$) S ! !  " "C< 0 0 0 0MMON!!!&%_553 3 !B C C C"GM"eONN@o~>???L!.AA v'']););sUC?@@@@M55L C F   1=   -E3 ve||#Q#Q#Q#Q=#Q#Q#Q Q Q|" @@@@-@@@)/7vvR %d++1()Qq1uuE%LL 1$$(1-(@(@44 9 &y 1 6 6 8 8'-i'8'?e 04n-%,,U333$((3333 4".dLBS!T!TJ(1-(@(@44 9 &y 1 6 6 8 856!VVzze @AQ Dn-,9!,<j)%,,U333$((3333 "519c2222E 000#**5111(,,E!(()9%)@AAA $$U++++Ag    " " "s,'>'>  !1#!6 7 7 7   S ! ! ! qO F  P rc ddldfd }d|D\}}}}||z }||z } |||f||fddj} d } ||| t|| d \} } }t d t j| | z}||| z d zz}|| |zd zz}||||f| | d |j|S)a Fill a rectangular area in the image with a white background and write the given token string. Paints directly on the provided image (in-place). Args: image (np.ndarray): Image to paint on (modified in-place). box (tuple): (x1, y1, x2, y2) coordinates of rectangle. token_str (str): Token string to write. Returns: np.ndarray: The same image (modified in-place). rN?cd\}}|}||z dkrI||zdz } |||d\\}} } |||zkr| ||zkr|}|}n|}||z dkI||| fS)N)g? g{Gz?rr/) thickness) getTextSize) rfontFace square_size fill_ratiorr optimal_scalemidrr_cv2s rget_optimal_font_scalez+paint_token..get_optimal_font_scales e dlT!!%<1$ChqIIIFQA;+++K*4L0L0L # dlT!!a""rc,g|]}t|SrN)int)rPvs rrRzpaint_token..s***c!ff***rr)colorr)rr/r)rrr)lineType)r) r rectangleFONT_HERSHEY_SIMPLEXr%r&mathfloorputTextLINE_AA)r_rQ token_strrr:r;r<r=box_wbox_hfontthickness_scale_ratio font_scaletext_wtext_hfont_thicknesstext_xtext_yrs @r paint_tokenrsJJJJ###### +*c***NBB GE GEMM%"bB8?bMQQQ  #D!7!74UE**s"""JDJz4I'IJJKKN56>a' 'F 56>a' 'FKK       Lrczd}ddl}|di}|\}}}} g|t|} || |}t |D]\} } | d\} }}}| |kr||kr||kr|| kry| t|| z ||z dkrZ| |z ||z ||z ||z g}dt| | zdz}t|||}| d ||<fd t |D}|||fS) a Replace figures in a table area with tokens, return new image and token map. Args: table_block_img (np.ndarray): Table image. table_box (list): Table bounding box [x_min, y_min, x_max, y_max]. figures (List[Dict]): List of figure dicts (must contain 'coordinate', 'path'). Returns: Tuple[np.ndarray, Dict[str, str], List[str]]: - New table image, - Token-to-img HTML map, - List of figure paths dropped. chd}g}d}t||krLtt||zs|||dz }t||kL|S)N>019rr/)rhrfrr)numexclude_digitsseqros rgen_random_mapz0tokenize_figure_of_table..gen_random_map sm( #hhnnAKK.0  1 FA#hhnn rrNirU[F]pathc.g|]\}}|v |dS)rrN)rProf drop_idxess rrRz,tokenize_figure_of_table..4s&PPP$!QZAfIr) randomseedrhshufflerrjrr%rr)table_block_img table_boxfiguresr r token_map table_x_min table_y_min table_x_max table_y_max random_map figure_idfigure figure_x_min figure_y_min figure_x_max figure_y_maxdraw_boxr drop_figuresrs @rtokenize_figure_of_tabler(s MMM KKI9B6Kk;JG --J NN:%**,,O&w//22 6AG AU> lL, K ' ' ++ ++ ++   i ( ( (<,. |0KLLrQQ{*{*{*{* H s:i#8999C?I)/8YOOO#)&>Ii PPPP)G*<*<PPPL I| 33rcBfd}d}tj|||S)z Replace tokens in a string with their HTML image equivalents. Args: table_res_str (str): Table string with tokens. figure_token_map (dict): Mapping from tokens to HTML img tags. Returns: str: Untokenized string. c|d}d|d}||d} |d}||dSg}|d|dddd d|}|jdkr|j}|d |zd zz }|S) Nr/rrrzImagez-    )groupgetrformatreplacejoincontent) matchtoken_idtokenimg_path img_blockimg_tags image_info ocr_contentfigure_token_mapimage_path_to_obj_maps rreplz(untokenize_figure_of_table..replDs;;q>> X   #''u{{1~~>>)--h==  ;;q>> !H OO/66$$UB//77cBB    8,,J B&&'/ f{2V;;  rz \[F(\d+)\])resub) table_res_strr=r>r?patterns `` runtokenize_figure_of_tablerD8s=(G 6'4 / //rceZdZUdZdZeed<dZeed<eed<eed<eed<eed<eed <d Z e ed <d Z e ed <d Z e ed <e dededefdZdS) TableCella TableCell represents a single cell in a table. Attributes: row_span (int): Number of rows spanned. col_span (int): Number of columns spanned. start_row_offset_idx (int): Start row index. end_row_offset_idx (int): End row index (exclusive). start_col_offset_idx (int): Start column index. end_col_offset_idx (int): End column index (exclusive). text (str): Cell text content. column_header (bool): Whether this cell is a column header. row_header (bool): Whether this cell is a row header. row_section (bool): Whether this cell is a row section. r/row_spancol_spanstart_row_offset_idxend_row_offset_idxstart_col_offset_idxend_col_offset_idxrF column_header row_header row_sectionbefore)rdatarct|trud|vr|S|ddd}t|s?|dd}|r|D]}||ddzz }|}||d<|S)z Create TableCell from dict, extracting 'text' property correctly. Args: data (Any): Input data. Returns: Any: TableCell-compatible dict. rbboxr7r+text_cell_bboxesNr-)rvrr0rhpopstrip)clsrQr text_cellsels rfrom_dict_formatzTableCell.from_dict_formatxs dD ! ! ~~ <##GR00Dt99 $!XX&8$?? 2(227 c 11zz||DL rN)__name__ __module__ __qualname____doc__rGr__annotations__rHrrMboolrNrOr classmethodrrZrNrrrFrF\s HcHc IIIM4JK_(###CC[$#rrFceZdZUdZgZeeed<dZe ed<dZ e ed<e e deeefdZ dS) TableDataz TableData holds a table's cells, row and column counts, and provides a grid property. Attributes: table_cells (List[TableCell]): List of table cells. num_rows (int): Number of rows. num_cols (int): Number of columns. table_cellsrnum_rowsnum_colsrc ~fdtjD}jD]}tt|jjt|jjD]P}tt|jjt|jjD] }||||<Q|S)z Returns a 2D grid of TableCell objects for the table. Returns: List[List[TableCell]]: Table as 2D grid. cRg|]"fdtjD#S)c Dg|]}tddz||dzS)r+r/)rrIrJrKrL)rF)rPrpros rrRz-TableData.grid...sQ   )*'(1u)*'(1u    r)rgrf)rProselfs @rrRz"TableData.grid..sY        t}--       r) rgrerdr%rIrJrKrfrL)rj table_datacellrorps` rgridzTableData.grids     4=))    $ , ,DD-t}==D+T];; , ,14=AA/??,,A(,JqM!$$ ,  ,rN)r[r\r]r^rdrrFr_rerrfr propertyrmrNrrrcrcs$&Ki%%%HcHc d4 ?+X^rrczzzzzzz+(?:|||||)z.*?(?=z|$))flagssc ddttttt t gzdz}tj||}tj ||}d|D}||fS)z Extract OTSL tags and text parts from the input string. Args: s (str): OTSL string. Returns: Tuple[List[str], List[str]]: (tokens, text_parts) (|)c:g|]}||SrNrV)rPr7s rrRz0otsl_extract_tokens_and_text..s%AAAE5;;==A%AAAr) r3OTSL_NL OTSL_FCEL OTSL_ECEL OTSL_LCEL OTSL_UCEL OTSL_XCELr@findallsplit)rprCtokens text_partss rotsl_extract_tokens_and_textrsx ))WiIy)T U U V   Z # #F'1%%JAAZAAAJ : rc &tdtj|fdD}g}d}d}|rKtd|D}|D]B}t ||kr-|t t ||k-Cg}d} |D]}|D]} || | t |krr|| | krf| dz } | t |krN|| ttt tttfvr ||| | dz } |t| t |kr|| tkr| dz } |}d} d} t|D]y\} }d}|tt fvr'd}d}d}|t kr || dz}d }| |zt |kr || |znd}d}|dzt |kr-|t ||dzkr||dz|}|ttfvr || ||dz|ttgz }|ttfvr || |||dzttgz }|t| |||||z|||z |tt tttfvr|dz }|tkr|dz }d}{||fS) a Parse OTSL text and tags into TableCell objects and tag structure. Args: texts (List[str]): List of tokens and text. tokens (List[str]): List of OTSL tags. Returns: Tuple[List[TableCell], List[List[str]]]: (table_cells, split_row_tokens) c6g|]\}}|t|SrN)list)rPxys rrRz$otsl_parse_texts..s8 Aq Qrc|kSrrN)z split_words rz"otsl_parse_texts..s Zrrc34K|]}t|VdSrrhrProws rrz#otsl_parse_texts..s(<.count_rightsk UmJ'<77 !OJ AIDS//// UmJ'<77  rcd}|}||||vr/|dz }|dz }|t|kr|S||||v/|Srr)rrrrr r_idx_iters r count_downz$otsl_parse_texts..count_down!sg Z '<77 !OJ AIDS[[(( Z '<77  rr+r)rrGrHrIrJrKrL)rw itertoolsgroupbyr&rhrryrxrzr{r|rjrFrV)textsrsplit_row_tokensrdrrmax_colsr new_textstext_idxr7rrror cell_textrGrH right_offsetnext_right_cellnext_bottom_cellrs @rotsl_parse_textsrsJ%f.G.G.G.GHH K E E<<+;<<<<<# & &Cc((X%% 9%%%c((X%% #  C & &  '''c%jj((U8_-E-EMH#e**,,x!!!!! I22"((x999 A    W % % %#e**$$xG)C)CA U##**4 Iy) ) )HHLy  !!a%L  ,-|+;c%jj+H+Ha,&''b  " qy3/00003/ :;;;;'7 'B5'I$9i"888KK$eaiI8N Iy#999JJ$eUQYI8N   "**%%).',x'7).',x'7    Iy)Y J J J QJE 7?? QJEE ( ((rrkc  |j}|j}t|jdkrdSd}|j}t |D]}|dz }t |D]}|||}|j|j} }|j|j } } | |ks| |kr9tj |j } |jrdnd} | }|dkr |d|dz }| dkr |d | dz }|d |d | d | d z }|d z }d|d}|S)z Export TableData to HTML table. Args: table_data (TableData): TableData object. Returns: str: HTML string. rr+zthtdr/z rowspan=""z colspan="<>zzz
)rerfrhrdrmrgrGrIrHrKhtmlescaperrVrM)rknrowsncolsbodyrmrorprlrowspanrowstartcolspancolstartr4celltag opening_tags rexport_to_htmlrYsd  E  E : !""a''r D ?D 5\\ u < .s&>>c3y>>>>>>>rc3&K|] }|dV dSrNrNrs rrz%otsl_pad_to_sqr_v2..s'==SK(======rrc3HK|]}t|dz VdSrr)rPrrs rrz%otsl_pad_to_sqr_v2..s5 S S3S%5%=!>!> S S S S S SrrNr+)rvrrVrwr~OTSL_FIND_PATTERNr}rhrj startswithrxrr&rrgrryr3)rlinesrow_datalinerrrrocell_strglobal_min_width max_total_len search_start search_endmin_total_cost optimal_widthcurrent_total_costrepaired_linesrcells current_len new_cellspaddingrs @rotsl_pad_to_sqr_v2r~sq h $ $$$ $~~Hh'!! NN7 # #EH     %--d33    NN $Y//  KAx""9-- a%#) P P     BJQs>>X>>>>>>PQAIPC==H======qM#L%}55J5\\NM|Z!^44"" S S S S( S S SSS  . ./N!MN22K %jj  & &n}n-II k][%@AGIbggi001111 << ' '' 11r otsl_contentct|}t|\}}t||\}}tt ||rt d|Dnd|}t |S)z Convert OTSL-v1.0 string to HTML. Only 6 tags allowed: , , , , , . Args: otsl_content (str): OTSL string. Returns: str: HTML table. c34K|]}t|VdSrrrs rrz'convert_otsl_to_html..s(;;3c#hh;;;;;;rr)rerfrd)rrrrcrhr&r)rr mixed_textsrdrrks rconvert_otsl_to_htmlrs&l33L6|DDFK$4[&$I$I!K!%&&?OV#;;*:;;;;;;UVJ * % %%rct|}td|dzdzD]%}||zdkr|d|}|||zz|kr|cS&dS)z Find the shortest substring that repeats to form the entire string. Args: s (str): Input string. Returns: str or None: Shortest repeating substring, or None if not found. r/rrN)rhrg)rpnro substrings r!find_shortest_repeating_substringrsp AA 1a1fqj ! !!! q5A::"1"IAF#q((    4rrr min_repeatsc^tt||z|dz dD]}|| d}|||zrbd}|}||r%|d| }|dz }||%t|||zz }|d|||fcSdS)a! Detect if string ends with a repeating phrase. Args: s (str): Input string. min_len (int): Minimum length of unit. min_repeats (int): Minimum repeat count. Returns: Tuple[str, str, int] or None: (prefix, unit, count) if found, else None. r/rNr)rgrhendswith)rprrrounitcounttemp_s start_indexs rfind_repeating_suffixrs3q66k*GaK < < 0 0!v ::d[( ) ) 0EF//$'' ! //$'' a&&EAI.K\k\?D%/ / / / 0 4rr r4line_thresholdchar_threshold min_countct||kr|S|}|s|Sd|vrUt|dkrBt|dd}|r.|\}}} t|| zt|dzkr|Sd|vrKt||kr8t|}|r't|t|z} | |kr|Sd|dD} | s|St| } | |kr|St | } | dd \} } | |kr | | z d kr| S|S) a Detect and truncate character-level, phrase-level, or line-level repetition in content. Args: content (str): Input text. line_threshold (int): Min lines for line-level truncation. char_threshold (int): Min repeats for char-level truncation. min_len (int): Min length for char-level check. Returns: Union[str, str]: (truncated_content, info_string) r,drr)rrrZc^g|]*}||+SrNrv)rPrs rrRz/truncate_repetitive_content..&s- J J JdTZZ\\ JTZZ\\ J J Jrr/rg?)rhrVrrr~r most_common)r4rrrrstripped_content suffix_matchprefixrepeating_unitrr total_lines line_countsmost_common_lines rtruncate_repetitive_contentrs& 7||i}}  ###,<(=(=(C(C,-=qVWXXX  ,8 )FNE>""U*S1A-B-BS-HHH  ###,<(=(=(G(G:;KLL  &())S-@-@@E&&%% K JgmmD&9&9 J J JE e**K^##%..K)55a88;e EK$7C#?#? Nrc<ddl}t|jdkr|||j}n|}|jtjkr| tj}t| }t| }||kr|Stj dtj}t||dzD] }t||z ||z z dz||<!|||}||dd|j\}} || } | |S|| \} } } }|| | |z| | | zf}|S)Nrr0rVr/r)rrhshapecvtColorCOLOR_BGR2GRAYrrWr4uint8astyperr&r%zerosrgLUT thresholdTHRESH_BINARY_INV findNonZero boundingRect)ryrgraymax_valmin_vallutrrQrbinaryr-rrrrcroppeds r crop_marginr 4sJJJ 39~~||C!344xxzz zRX{{28$$$((**ooG$((**ooG'  (3bh ' ' 'C 7GaK ( (@@a'kg&783>??A 774  D dCc.CDDIAv __V $ $F ~ !!&))JAq!Q!a!e)QQY&'G Nrz#<\|TEXT_START\|>(.*?)<\|TEXT_END\|>z!<\|LOC_BEGIN\|>(.*?)<\|LOC_END\|>z<\|LOC_(\d+)\|>r_c|jdd\}}|dkrq|dkrkt|} tjj}n#t $rtj}YnwxYw||dz|dzf|}t|S|S)zG Post-process the input image to extract location information. Nri)rrzr ResamplingLANCZOSAttributeErrorresizer|)r_rrpil_imgresample_filters rpre_process_for_spottingr]s ;rr?DAq4xxAHHu%% ,#.6OO , , ,#mOOO ,..!a%QAA7### LsAAA input_strrrc 6t|tsJt|}t|}g}g}t t |t |}t|D]}||} t||} t | dkrPttt| ddfdtdddD} fd| D} | | | | |r|stt|} d} d}|dzt | kr| ||dz}d|Dfd tdddD} fd | D} || |d}|} | | | | |d } |dz }|dzt | kd |}||d }||fS)zL Post-process the input string to extract text and location blocks. rNc6g|]}||dzfSr/rNrPrpvalss rrRz-post_process_for_spotting..s*>>>!Qa!e%>>>rrrcHg|]}|ddz z|ddz zfSrg@@r/rNrPprrs rrRz-post_process_for_spotting..s8CCC!!v !1Q4&=1#45CCCrcRg|]$}t|d%Sr)rr/)rPms rrRz-post_process_for_spotting..s(333C OO333rc6g|]}||dzfSrrNrs rrRz-post_process_for_spotting..s*BBBaDGT!a%[)BBBrcHg|]}|ddz z|ddz zfSrrNrs rrRz-post_process_for_spotting..s8GGGaAaD6MA%qtf}q'89GGGrrr.) rec_polys rec_texts)rvr ANNOT_TEXT_REr} LOC_BLOCK_REr%rhrgrV LOC_ITEM_RErmaprr LOC_TOKEN_REfinditerrrr3)rrrr loc_blocksr"r#rrotxt loc_itemsptsmatcheslast_endr/ text_span result_str spotting_resrs `` @rpost_process_for_spottingr3ns i % %%% %  ! !) , ,E%%i00JII CJJJ((A 1XX  Ahnn'' 1 66 y>>A   CYrr]++,,>>>>uQ1~~>>>CCCCCsCCC I|,,Y7788 !ec'll""AAI&E33U333DBBBB5Aq>>BBBCGGGGG3GGGC!(U1X^^-=-="=>I//##C   S ! ! !   S ! ! !Ry}}H FA!ec'll""Y''J!*CCL | ##r)r)rr)r)rr)rrrr)Errrr@ collectionsrrrtypingrrrrr numpyr4PILr pydanticr r r layout_parsing.utilsrrrrrrr,ndarrayrGrtrzr|rrrrr(rDrFrcrwrxryrzr{r|NON_CAPTURING_TAG_GROUPcompileDOTALLrrrrrrrrrr Sr$r%r&r(rr3rNrrr?sK  00000000000000?????????? #2#25c?##25c?##2 #2 #2#2#2#2L J 2:rz !">B#d4j)B#>AB# #tDz/B#B#B#B#J       "9999x]]]]@;;;|;4;4;4|!0!0!0H33333 333l..... ...d       GBJBB&=BBB") C*p)p)p)f"y""""J727272727272t&s&&&&*sDy1A(23 +. 5c3  %&: 66 666 6  6  6666r   F A24HH rz>EE bj+,, rz,-- BJ4T ?"2$2$2$"2$ 3S$Y  2$2$2$2$2$2$r