{-jfFddlmZmZmZmZmZmZddlZddl m Z ddl m Z ddl mZddlmZdd lmZmZdd lmZdd lmZd d lmZd dlmZd dlmZd dlmZddl m!Z!ddl m"Z"m#Z#ej$GddeZ%e dGddeZ&dS))AnyDictListOptionalTupleUnionN)logging)pipeline_requires_extra)ImageBatchSampler) ReadImage) HPIConfigPaddlePredictorOption) DetResult) benchmark)(AutoParallelImageSimpleInferencePipeline) BasePipeline) CropByBoxes) OCRResult)LayoutParsingResult)get_sub_regions_ocr_ressorted_layout_boxesc1eZdZdZddddddddedeedeedeeeefd eed e d ee eeefe fd dffd Z ded dfdZ d1dededededededeedeedeedeedeedeed efdZded e fdZdee dee d ee d!ee d"ee d ef d#Z d2d$e eeeejeejfdee dee d%ee d ee d!ee d"ee d&ee eefd'ee d(ee eeeefefd)eedeedeedeedeedeedeed*eed+eed,eed-eed.eed/eed ef0d0ZxZS)3_LayoutParsingPipelinezLayout Parsing PipelineNFdeviceengine engine_config pp_optionuse_hpip hpi_configconfigrr r!r"r#r$returnc tjd||||||d|||td|_t d|_t|_dS)aInitializes the layout parsing pipeline. Args: config (Dict): Configuration dictionary containing various settings. device (Optional[str], optional): The device to use for prediction. Defaults to `None`. engine (Optional[str], optional): Inference engine. Defaults to `None`. engine_config (Optional[Dict[str, Any]], optional): Engine-specific config. Defaults to `None`. pp_option (Optional[PaddlePredictorOption], optional): Paddle predictor options. Defaults to `None`. use_hpip (bool, optional): Whether to use HPIP. Defaults to `False`. hpi_config (Optional[Union[Dict[str, Any], HPIConfig]], optional): HPIP configuration. Defaults to `None`. rr) batch_sizeBGR)formatN) super__init__inintial_predictorr batch_samplerr img_readerr_crop_by_boxes) selfr%rr r!r"r#r$kwargs __class__s s/var/www/html/banglarbhumi/venv/lib/python3.11/site-packages/paddlex/inference/pipelines/layout_parsing/pipeline.pyr-z_LayoutParsingPipeline.__init__&s2  '!       '''.!<<<#5111)mmc@|dd|_|dd|_|dd|_|dd|_|jrF|diddd i}|||_|d id d d i}i}|ddx}||d<|ddx}||d<|ddx} ||d<|ddx} ||d<|j|fi||_|didddi} || |_ |jrF|didddi} || |_ |jrF|didddi} || |_ |jrF|didddi} || |_ dS)zInitializes the predictor based on the provided configuration. Args: config (Dict): A dictionary containing the configuration for the predictor. Returns: None use_doc_preprocessorTuse_table_recognitionuse_seal_recognitionuse_formula_recognition SubPipelinesDocPreprocessorpipeline_config_errorz+config error for doc_preprocessor_pipeline! SubModulesLayoutDetectionmodel_config_errorz"config error for layout_det_model! thresholdN layout_nmslayout_unclip_ratiolayout_merge_bboxes_mode GeneralOCRz&config error for general_ocr_pipeline!SealRecognitionz+config error for seal_recognition_pipeline!TableRecognitionz,config error for table_recognition_pipeline!FormulaRecognitionz.config error for formula_recognition_pipeline!) getr8r9r:r;create_pipelinedoc_preprocessor_pipeline create_modellayout_det_modelgeneral_ocr_pipelineseal_recognition_pipelinetable_recognition_pipelineformula_recognition_pipeline) r2r%doc_preprocessor_configlayout_det_config layout_kwargsrBrCrDrEgeneral_ocr_configseal_recognition_configtable_recognition_configformula_recognition_configs r5r.z)_LayoutParsingPipeline.inintial_predictorPs&%+JJ/Et$L$L!%+ZZ0G%N%N"$*JJ/Et$L$L!'-zz2KT'R'R$  $ &,jj&D&D&H&H!+-Z'' # .2-A-A'..D *#JJ|R88<<  !#G H   *..{DAA AI N)2M+ &+// dCC CJ P*4M, '#4#8#89NPT#U#U U  4GM/ 0(9(=(=*D))  $   9QM4 5 1 12C U U} U U#ZZ;;??  $&N O  %)$8$89K$L$L!  $ &,jj&D&D&H&H!+-Z'' # .2-A-A'..D *  % '-zz."'E'E'I'I"+-[(( $ /3.B.B(//D +  ' )/NB)G)G)K)K$+-]** & 150D0D*11D - r6imagelayout_det_resoverall_ocr_restable_res_list seal_res_listformula_res_listtext_det_limit_side_lentext_det_limit_typetext_det_threshtext_det_box_threshtext_det_unclip_ratiotext_rec_score_threshc g} i}d}d}d}tj|}g}t|dD]\}}i}|d}|d}||d<||d<d|d<|||d krHt |dkr3t ||ks Jd ||d |d<|d z }n|d krGt |dkr3t ||ks Jd||d|d<|d z }n|dkrZt |dkrFt ||ks Jdd||d|d<|d z }nqt||gd\}}|D]:}||d|g||<|||;d|d|d<| || D]}t |d kr|D]}tj |j |j dz}| |d}d|D\}}} }!|||!|| ddf|||!|| ddf<t||||| | | | d}"d|"d| |d<t||d}#t|#d|#dD]+\}$}%i}|$|d<d|d<|%|d<| |,t!| |j d  } | S)!a Retrieves the layout parsing result based on the layout detection result, OCR result, and other recognition results. Args: image (list): The input image. layout_det_res (DetResult): The detection result containing the layout information of the document. overall_ocr_res (OCRResult): The overall OCR result containing text information. table_res_list (list): A list of table recognition results. seal_res_list (list): A list of seal recognition results. formula_res_list (list): A list of formula recognition results. text_det_limit_side_len (Optional[int], optional): The maximum side length of the text detection region. Defaults to `None`. text_det_limit_type (Optional[str], optional): The type of limit for the text detection region. Defaults to `None`. text_det_thresh (Optional[float], optional): The confidence threshold for text detection. Defaults to `None`. text_det_box_thresh (Optional[float], optional): The confidence threshold for text detection bounding boxes. Defaults to `None` text_det_unclip_ratio (Optional[float], optional): The unclip ratio for text detection. Defaults to `None`. text_rec_score_thresh (Optional[float], optional): The score threshold for text recognition. Defaults to `None`. Returns: list: A list of dictionaries representing the layout parsing result. rboxes coordinatelabel block_bbox block_label block_contentformulazThe number of formula regions of layout parsing pipeline and formula recognition pipeline are different! rec_formulartablezThe number of table regions of layout parsing pipeline and table recognition pipeline are different! pred_htmlsealzThe number of seal regions of layout parsing pipeline and seal recognition pipeline are different!z, rec_textsT)return_match_idxN )dtypec,g|]}t|Sr+)int).0is r5 zA_LayoutParsingPipeline.get_layout_parsing_res.. s%:%:%:c!ff%:%:%:r6)r`rarbrcrdreF) flag_within rec_boxes other_text)w)nparray enumeratelowerappendlenjoinrrJvaluesonesshapervlistrOzipr)&r2rZr[r\r]r^r_r`rarbrcrdrelayout_parsing_resmatched_ocr_dict formula_index table_index seal_index object_boxesobject_box_idxbox_infosingle_box_resboxriocr_res_in_box matched_idxes matched_idxlayout_box_idsidxwht_imx1y1x2y2 sub_ocr_resocr_without_layout_boxes ocr_rec_box ocr_rec_texts& r5get_layout_parsing_resz-_LayoutParsingPipeline.get_layout_parsing_ress|B     (1.2I(J(J3 63 6 $NHN<(CW%++--E+.N< (,1N= ).0N? +    $ $ $ !!'((1,,,-- ===I>==7G}6U%7N?3"Q&M'!!~&&**N++k999G:997E[6Q#7N?3 1$K&}%%))M**Z777F8777;ii%j1+>77N?3!OJ0G#cUT111- $1MMK'++K>>F9G8H(55(5<<^LLLL26))N;""Q&&)CWU[ DDDsJF,S1,?C%:%:c%:%:%:NBB.3BrE2b5!!!O.DF2b5"R%?+"&11"4K0C,;0C2G2G2 # # #K@Dyy#K0@@&s+O<<$; \u$ $ $  *- $[ 13KK3X* *  6 6 %K N+6N< (,8N= ).:N? +  % %n 5 5 5 501Cu{ST~VVV!!r6 input_paramsc|dr|jstjddS|dr|jstjddS|dr|jstjddSdS) a4 Check if the input parameters are valid based on the initialized models. Args: input_params (Dict): A dictionary containing input parameters. Returns: bool: True if all required models are initialized according to input parameters, False otherwise. r8zRSet use_doc_preprocessor, but the models for doc preprocessor are not initialized.Fr:zRSet use_seal_recognition, but the models for seal recognition are not initialized.r9zTSet use_table_recognition, but the models for table recognition are not initialized.T)r8r errorr:r9)r2rs r5check_model_settings_validz1_LayoutParsingPipeline.check_model_settings_valid,s . / 8Q  Md   5 . / 8Q  Md   5 / 0 9S  Mf   5tr6use_doc_orientation_classifyuse_doc_unwarpingr:r9r;c| ||j}n |dus|durd}nd}||j}||j}||j}t ||||S)a' Get the model settings based on the provided parameters or default values. Args: use_doc_orientation_classify (Optional[bool]): Whether to use document orientation classification. use_doc_unwarping (Optional[bool]): Whether to use document unwarping. use_seal_recognition (Optional[bool]): Whether to use seal recognition. use_table_recognition (Optional[bool]): Whether to use table recognition. Returns: dict: A dictionary containing the model settings. NTF)r8r:r9r;)r8r:r9r;dict)r2rrr:r9r;r8s r5get_model_settingsz)_LayoutParsingPipeline.get_model_settingsKs( ( /4E4M#'#< +t337HD7P7P'+$$',$  '#'#< ($($> ! " *&*&B #!5!5"7$;     r6inputuse_textline_orientationlayout_thresholdrCrDrEseal_det_limit_side_lenseal_det_limit_typeseal_det_threshseal_det_box_threshseal_det_unclip_ratioseal_rec_score_threshc+zK||||||}||sddiVt||D]\}}||jd}|dr,t ||||d}nd|i}|d}t |||| | | d}t | ||| | ||||d} |d r8t | |d d d d | | d}!|!d }"ng}"|d r> PNO O O O"+D,>,>u,E,E"F"Ft 6t 6 FJ//**>??BK45 C'+22#5Q*;3((  ($$)5k'B$%9,%G "!%%*.)(;-E &N#))*-E,C(;$3(;*?*?*     O56 $ $33.5:*/-2&+(7'54 ! ! ! "//?!@!#45 ##22.5:*/-2'50G,?(7,?.C.C3      !-_ = " 78 &"&55.-25:*/'5 6###$33E#F  #% #::&- /-+!1(?$7 /$7&;&; ;    )4Q7(5a8(<"0#2"0!.$4$4"0  N&n55 5 5 5 5it 6t 6r6)NNNNNN)NNNNNNNNNNNNNNNNNNNNNN)__name__ __module__ __qualname____doc__rrstrrrboolrrr-r.rrrryfloatrrrrrrndarrayrrr __classcell__)r4s@r5rr"sg!! !% $2659AE(,(,(,(, (,  (,  S#X/ (,12(,(,U4S>9#<=>(, (,(,(,(,(,(,TVV$VVVV@26-1+//31515B"B"B""B"# B"  B"  B"B""*#B"&c]B""%B"&e_B" (B" (B" B"B"B"B"Ht>* &.tn* $D>* 'tn * (~ * "*$ *  * * * * ^8<,037/304269=%)QU2615-1+//3151515-1+//315151@6@6S$s)RZbj1AAB@6'/tn@6$D> @6 #+4. @6 'tn @6 (~@6"*$@6#5#56@6TN@6&eE53F,L&MN@6#+3-@6"*#@6&c]@6"%@6 &e_!@6" (#@6$ (%@6&"*#'@6(&c])@6*"%+@6,&e_-@6. (/@60 (1@64 5@6@6@6@6@6@6@6@6r6rocrc0eZdZdgZedZdZdS)LayoutParsingPipelinelayout_parsingctS)N)r)r2s r5 _pipeline_clsz#LayoutParsingPipeline._pipeline_cls>s%%r6cdS)Nrr+)r2r%s r5_get_batch_sizez%LayoutParsingPipeline._get_batch_sizeBsqr6N)rrrentitiespropertyrrr+r6r5rr:sD !H &&X&r6r)'typingrrrrrrnumpyrutilsr utils.depsr common.batch_samplerr common.readerrmodelsrrmodels.object_detection.resultrutils.benchmarkr _parallelrbaser componentsr ocr.resultrresultrrr time_methodsrrr+r6r5rs;:::::::::::::::222222555555&&&&&&66666666777777((((((@@@@@@$$$$$$""""""''''''???????? T6T6T6T6T6\T6T6T6nD r6