jOddlmZddlZddlmZddlmZmZm Z m Z m Z m Z m Z mZddlmZmZmZmZddlmZdgZGddeZGd d eZdS) )AnyN) fclusterdata)BlockDocument KIEDocumentKIEPageLinePage PredictionWord)estimate_page_angleresolve_enclosing_bboxresolve_enclosing_rbbox rotate_boxes) NestedObjectDocumentBuilderceZdZdZ ddedededed d f d Zed ej d e ej ej ffd Z d ej de e d e e e fdZd ej d e e e fdZed ej de e e d e e e e fdZd ej dej de e eefde eeefd e ef dZd efdZ dde ej d e ej de ej de e e eefde e e e fde eeefde eeefd zde eeefd zd efdZd S) raImplements a document builder Args: resolve_lines: whether words should be automatically grouped into lines resolve_blocks: whether lines should be automatically grouped into blocks paragraph_break: relative length of the minimum space separating paragraphs export_as_straight_boxes: if True, force straight boxes in the export (fit a rectangle box to all rotated boxes). Else, keep the boxes format unchanged, no matter what it is. TFQ? resolve_linesresolve_blocksparagraph_breakexport_as_straight_boxesreturnNc>||_||_||_||_dSNrrrr)selfrrrrs W/var/www/html/Carbon-Document/venv/lib/python3.11/site-packages/doctr/models/builder.py__init__zDocumentBuilder.__init__s*+,.(@%%%boxesc|jdkr^t|t| dd}tj|d|dfd}|dddfd |dddfztj|dddf|dddfz z z|fS) aSort bounding boxes from top to bottom, left to right Args: boxes: bounding boxes of shape (N, 4) or (N, 4, 2) (in case of rotated bbox) Returns: tuple: indices of ordered boxes of shape (N,), boxes If straight boxes are passed tpo the function, boxes are unchanged else: boxes returned are straight boxes fitted to the straightened rotated boxes so that we fit the lines afterwards to the straigthened page )r$g@) loc_predsangle orig_shape min_angleNr) ndimrr np concatenateminmaxmedianargsort)r!s r _sort_boxeszDocumentBuilder._sort_boxes*s :?? *5111' E NEIIaLL%))A,,#?DDEaaad a%1+o %1+aaaQRd :S0T0TTT]]__afffr word_idcscg}fd|dfDtdkr|ndg}ddD]`}d}||d}||df|dz }||jkrd}|r||g}||a|||S) aSplit a line in sub_lines Args: boxes: bounding boxes of shape (N, 4) word_idcs: list of indexes for the words of the line Returns: A list of (sub-)lines computed from the original line (words) c g|] }| Sr7).0idxr4s r z6DocumentBuilder._resolve_sub_lines..MsVVVYs^VVVr rr+r)NTr*F)r2tolistlenappendr) rr!r4linessub_linei horiz_breakprev_boxdists ` r_resolve_sub_linesz"DocumentBuilder._resolve_sub_linesAsVVVVuY\/B/J/J/L/L/S/S/U/UVVV  y>>A   LL # # # #!! ~Hqrr] # #"  ".QT{Xa[0$..."'K"LL***!H"""" LL " " " r c||\}}tj|dddf|dddfz }g}|dg}||dddg}|ddD]}d}t ||ddg|t |z z } | |dz krd}|r-||||g}d}|||||ddgz }t |dkr)|||||S)zOrder boxes to group them in lines Args: boxes: bounding boxes of shape (N, 4) or (N, 4, 2) in case of rotated bbox Returns: nested list of box indices Nr#r)rTr+F) r3r-r1meanabsr<extendrDr=) rr!idxsy_medr>words y_center_sumr9 vert_breaky_dists r_resolve_lineszDocumentBuilder._resolve_linesgs&&u-- e %1+aaad 344a T!W~q!f-2244 8 6 6CJsQF+0022\CJJ5NNOOF !!"  ! T44UEBBCCC LL    E#J1v.3355 5LL u::>> LL00>> ? ? ? r r>c jdkr!tjfdD}n,fdD}tjd|D}jdkrtj|ddddf|ddddfzdz |ddddf|ddddfzdz |ddddf|ddddfzdz |ddddf|ddddfzdz |ddddf|ddddfzdz |ddddf|ddddfzdz fd }ntj|dddf|dddfzdz |dddf|dddfzdz |dddf|dddfzdz |dddf|dddfzdz |dddf|dddffd }t |d d d d}i}t |D]=\}}||vr|||7|g||<>fd|D} | S)zOrder lines to group them in blocks Args: boxes: bounding boxes of shape (N, 4) or (N, 4, 2) lines: list of lines, each line is a list of idx Returns: nested list of box indices r#cFg|]}tfd|DS)c Hg|]}t|ddddfSrtupler8r9r!s rr:z>DocumentBuilder._resolve_blocks...s4(Q(Q(QSuS!!!QQQY/?)@)@(Q(Q(Qr )rr8liner!s rr:z3DocumentBuilder._resolve_blocks..sF000((Q(Q(Q(QD(Q(Q(QRR000r cFg|]}tfd|DS)c tg|]4}t|ddft|ddff5S)Nr+rSrUs rr:z>DocumentBuilder._resolve_blocks...sJ'g'g'g[^uS"1"W~)>)>eCQRQSQSGn@U@U(V'g'g'gr )rrVs rr:z3DocumentBuilder._resolve_blocks..sG''g'g'g'gbf'g'g'ghhr c,g|]\\}}\}}||||fSr7r7)r8x1y1x2y2s rr:z3DocumentBuilder._resolve_blocks..s.#\#\#\9M(2rHRRR$4#\#\#\r Nrr)r+r*)axisg?distance euclidean)tdepth criterionmetricc,g|]}fd|DS)c g|] }| Sr7r7)r8r9r>s rr:z>DocumentBuilder._resolve_blocks...s///#5:///r r7)r8blockr>s rr:z3DocumentBuilder._resolve_blocks..s.NNNE///////NNNr ) r,r-asarraystackr enumeratekeysr=values) r!r> box_lines _box_lines box_featuresclusters_blocksline_idx cluster_idxblockss `` r_resolve_blockszDocumentBuilder._resolve_blockss= :??$&J0000!000%%II !J #\#\Q[#\#\#\]]I :??')xqqq!Qw')AAAq!G*<<Aqqq!Qw')AAAq!G*<<Aqqq!Qw')AAAq!G*<<Aqqq!Qw')AAAq!G*<<Aqqq!Qw')AAAq!G*<<Aqqq!Qw')AAAq!G*<<A  ( ( (LL8qqq!t_yA6!;qqq!t_yA6!;qqq!t_yA6!;qqq!t_yA6!;aaadOaaadO    L 1 [fggg(*%.x%8%8 2 2 !Hkgllnn,, $++H5555(0z $$ONNNW^^=M=MNNN r objectness_scores word_predscrop_orientationscfjdtkr-tdjddtjddkrgS}|jr|||jdkr|n |ddddf}|jrCt|dkr0||jdkr|n |ddddf|}n<|g}n8||jdkr|n |ddddfdg}|g}fd|D}|S) aGather independent words in structured blocks Args: boxes: bounding boxes of all detected words of the page, of shape (N, 4) or (N, 4, 2) objectness_scores: objectness scores of all detected words of the page, of shape N word_preds: list of all detected words of the page, of shape N crop_orientations: list of dictoinaries containing the general orientation (orientations + confidences) of the crops Returns: list of block elements rIncompatible argument lengths: , r#Nr`r)cLg|] }tfd|D!S)cLg|] }tfd|D!S)c g|]}jdkr`tg|td|Dt ||RnVtg||df|dff|df|dffft ||RS)r#c34K|]}t|VdSrrSr8pts r zQDocumentBuilder._build_blocks.....s(FFBeBiiFFFFFFr rr)r+)r,r rTr;floatr8r9r!rzrxrys rr:zGDocumentBuilder._build_blocks....s-zQ #CFF%*2C2C2E2EFFFFF/455*#. #CQ-sAv7%Q-sTUv9WX/455*#. r )r )r8rWr!rzrxrys rr:z..sl"! $r )r)r8r>r!rzrxrys rr:z1DocumentBuilder._build_blocks..sl   (' ""#     r ) shaper< ValueErrorrrOr,rrwr3) rr!rxryrz_boxesr>rsrvs ```` r _build_blockszDocumentBuilder._build_blockss& ;q>S__ , ,bu{1~bbQTU_Q`Q`bbcc c ;q>Q  I   ''&+2B2BqqqRTSTRTu VVE" "s5zzA~~..9I9IvvvVWVWVWY[Z[Y[V[}^cdd '%% q0@0@fffQQQPRQRPRUmTTUVWXEgG       (!)   . r cFd|jd|jd|jd|jS)Nzresolve_lines=z, resolve_blocks=z, paragraph_break=z, export_as_straight_boxes=r)rs r extra_reprzDocumentBuilder.extra_reprsS HT/ H H$BU H H#3 H H(,(E H H r pages text_preds page_shapes orientations languagesc t|t|cxkr't|cxkrt|ksMnt|t|cxkr't|cxkrt|krnntdt|tr|ndgt|z} t|tr|ndgt|z} jr0t|dkr|djdkr d|D}fdt |tt||||||| | D} t| S)ayRe-arrange detected words into structured blocks Args: pages: list of N elements, where each element represents the page image boxes: list of N elements, where each element represents the localization predictions, of shape (*, 4) or (*, 4, 2) for all words for a given page objectness_scores: list of N elements, where each element represents the objectness scores text_preds: list of N elements, where each element is the list of all word prediction (text + confidence) page_shapes: shape of each page, of size N crop_orientations: list of N elements, where each element is a dictionary containing the general orientation (orientations + confidences) of the crops orientations: optional, list of N elements, where each element is a dictionary containing the orientation (orientation + confidence) languages: optional, list of N elements, where each element is a dictionary containing the language (language + confidence) Returns: document object 7All arguments are expected to be lists of the same sizeNrr#cg|]?}tj|d|dfd@S)r))r-r.r/r0)r8p_boxess rr:z,DocumentBuilder.__call__..Hs>bbbQXQQ(H!LLbbbr c xg|]6\ }}}}}}}}} t| |||||||| 7Sr7)r r) r8page_idxr page_boxes loc_scoresryword_crop_orientations orientationlanguagers rr:z,DocumentBuilder.__call__..Js{   udE:z:G]_jlt ""*       r ) r<r isinstancelistrr,zipranger) rrr!rxrrrzrr _orientations _languages_pagess ` r__call__zDocumentBuilder.__call__s< u::Z \ \ \ \C0A,B,B \ \ \ \cJ[F\F\ \ \ \ \`cdi`j`jnq o o a>a>a>a> " # #a>a>a>a>'*+<'='=a>a>a>a>a>VWW W'|T:: SLLU@S #-Y"="=VYYD6CPUJJCV  ( cSZZ!^^Qx}!!bb\abbb    y|c%jj!!!! y y   6r )TFrFNN)__name__ __module__ __qualname____doc__boolrr staticmethodr-ndarrayrTr3rintrDrOrwstrdictrrrrrrr7r rrrs#$!&). A A A A A #' A  A A A Ag2:g% BJ0F*Gggg\g,$ $tCy$TRVWZR[_$$$$L+BJ+4S ?++++Z?rz?$tCy/?d4PTUXPY?F[???\?B>z>:>sEz*+ >  S#X/ > e >>>>@ C    5915H H BJH BJH   + H eCJ/01 H %S/* H  S#X/H 4S>*T1H S#X'$.H  H H H H H H r ceZdZdZ ddeejdeeeejfdeeeejfdeeeee ee ffdee e e fdeeeeeee ffd eeee fdzd eeee fdzd e fd Zdejdejd ee ee fdeeee fd eef dZdS)KIEDocumentBuilderaImplements a KIE document builder Args: resolve_lines: whether words should be automatically grouped into lines resolve_blocks: whether lines should be automatically grouped into blocks paragraph_break: relative length of the minimum space separating paragraphs export_as_straight_boxes: if True, force straight boxes in the export (fit a rectangle box to all rotated boxes). Else, keep the boxes format unchanged, no matter what it is. Nrr!rxrrrzrrrc .t|t|cxkr't|cxkrt|ksMnt|t|cxkr't|cxkrt|krnntdt|tr|ndgt|z} t|tr|ndgt|z} jrt|dkrt t |djdkrzg} |D]s} i} | D]E\}}tj | d| dfd| |<F| | t| }fdt|t!t||||||| | D}t#|S)aRe-arrange detected words into structured predictions Args: pages: list of N elements, where each element represents the page image boxes: list of N dictionaries, where each element represents the localization predictions for a class, of shape (*, 5) or (*, 6) for all predictions objectness_scores: list of N dictionaries, where each element represents the objectness scores for a class text_preds: list of N dictionaries, where each element is the list of all word prediction page_shapes: shape of each page, of size N crop_orientations: list of N dictonaries, where each element is a list containing the general crop orientations (orientations + confidences) of the crops orientations: optional, list of N elements, where each element is a dictionary containing the orientation (orientation + confidence) languages: optional, list of N elements, where each element is a dictionary containing the language (language + confidence) Returns: document object rNrr#r)c  g|]B\ }}} }}t|  fdD||||CS)c ri|]3}|||||4Sr7)r)r8krrrrrys r z:KIEDocumentBuilder.__call__...sZ t))"1 "1 "1 .q1 r )rrm) r8rrrrrrrrryrs @@@@rr:z/KIEDocumentBuilder.__call__..s   "udE:z:G]_jlt! (__..     r )r<rrrrnextiterrnr,itemsr-r.r/r0r=rrr)rrr!rxrrrzrrrrstraight_boxesrstraight_boxes_dictrboxrs` rrzKIEDocumentBuilder.__call__sst< u::Z \ \ \ \C0A,B,B \ \ \ \cJ[F\F\ \ \ \ \`cdi`j`jnq o o a>a>a>a> " # #a>a>a>a>'*+<'='=a>a>a>a>a>VWW W&|T:: SLLU@S #-Y"="=VYYD6CPUJJCV  ( 'SZZ!^^Dq**++,,1Q66>@$??G*,'")--//]]313SWWUVZZ@XZ[1\1\+A.."))*=>>>>&    "y|c%jj!!!! y y#   <6"""r rycRjdtkr-tdjddtjddkrgS}||jdkr|n |ddddf\}}fd|D}|S)aGather independent words in structured blocks Args: boxes: bounding boxes of all detected words of the page, of shape (N, 4) or (N, 4, 2) objectness_scores: objectness scores of all detected words of the page word_preds: list of all detected words of the page, of shape N crop_orientations: list of orientations for each word crop Returns: list of block elements rr|r}r#Nr`c g|]}jdkrtt|d|dtd|Dt ||njt|d|d|df|dff|df|dffft ||S)r#rr)c34K|]}t|VdSrrSrs rrz>KIEDocumentBuilder._build_blocks...s(GGRuRyyGGGGGGr )value confidencegeometryobjectness_scorecrop_orientationr+)r,r rTr;rrs rr:z4KIEDocumentBuilder._build_blocks..s!   zQ  oa(%c?1-GG5:3D3D3F3FGGGGG!&'8'=!>!>!23!7      oa(%c?1- a=%Q-85a=%PSUVPV-:XY!&'8'=!>!>!23!7    r )rr<rr3r,) rr!rxryrzrrI_ predictionss ```` rrz KIEDocumentBuilder._build_blockss$ ;q>S__ , ,bu{1~bbQTU_Q`Q`bbcc c ;q>Q  I""V[A-=-=666!!!RaR%=QQa       !   $r r)rrrrrr-rrrrTrrrrrr rr7r rrrhs$5915Q#Q#BJQ#Dbj)*Q# S"*_ 56 Q# c4c5j(9#::;< Q# %S/* Q# S$tCH~*>%> ?@Q#4S>*T1Q#S#X'$.Q# Q#Q#Q#Q#f-z-:-sEz*+ -  S#X/ - j  ------r r)typingrnumpyr-scipy.cluster.hierarchyrdoctr.io.elementsrrrrr r r r doctr.utils.geometryr rrrdoctr.utils.reprr__all__rrr7r rrs"000000aaaaaaaaaaaaaaaaaaaassssssssssss))))))  R R R R R lR R R j KKKKKKKKKKr