j1ddlmZddlZddlZddlmZddlZddlmZm Z m Z m Z m Z m Z ddlmZmZddlmZmZd-d.dZe d/d0dZ d1d2d,ZdS)3) annotationsN)Path) IS_JETSONLOGGER TORCH_VERSIONThreadingLockedis_dgx is_jetson)check_tensorrt check_version) TORCH_2_4 TORCH_2_9Fonnxtypes.ModuleTypecudaboolreturnintctr=tjjjjdz }t rt|d}|r|dz}nUdtj ddd}ddddd d d d d d ddddd d  |d}t||j S) z@Return max ONNX opset for this torch version with ONNX fallback..N )z1.8z1.9z1.10z1.11z1.12z1.13z2.0z2.1z2.2z2.3z2.4z2.5z2.6z2.7z2.8)r torchrutils _constantsONNX_MAX_OPSETrminjoinrsplitgetdefsonnx_opset_version)rropsetversions b/var/www/html/Carbon-Document/venv/lib/python3.11/site-packages/ultralytics/utils/export/engine.pybest_onnx_opsetr-s  +:Q>  #rNNE   QJE((=.s33BQB788  #gr  ! " udi2244 5 55rmodeltorch.nn.Moduleim'torch.Tensor | tuple[torch.Tensor, ...] output_file Path | strr* input_nameslist[str] | None output_namesdynamic dict | Nonestrc |dg}|dg}trddini}tjj|||fd|d|||d|t |S)aExport a PyTorch model to ONNX format. Args: model (torch.nn.Module): The PyTorch model to export. im (torch.Tensor | tuple[torch.Tensor, ...]): Example input tensor(s) for tracing. output_file (Path | str): Path to save the exported ONNX file. opset (int): ONNX opset version to use for export. input_names (list[str] | None): List of input tensor names. Defaults to ``["images"]``. output_names (list[str] | None): List of output tensor names. Defaults to ``["output0"]``. dynamic (dict | None): Dictionary specifying dynamic axes for inputs and outputs. Returns: (str): Path to the exported ONNX file. Notes: Setting `do_constant_folding=True` may cause issues with DNN inference for torch>=1.12. Nimagesoutput0dynamoFT)verbose opset_versiondo_constant_foldingr5r7 dynamic_axes)r r rexportr:)r/r1r3r*r5r7r8kwargss r, torch2onnxrE.s6j !{ "+ 3h  F J    !       {  r.rrH onnx_filePath | str | None workspace int | Nonehalfint8shapetuple[int, int, int, int]dlametadatar?prefixc 4 tdstrtd ddln"#t$rtddlYnwxYwt jddt jd d tjd | d jd|p!t| d} j j } | rj j j| _| } | }t%pddz}t%jddddk}|r'|dkr!|jj|n |dkr||_dt%jjz}| || jo|}| jo|}t:st=dtj| dd|s|st=djj |_!t%|_"|#j$j%&| }|'|stQd|fdtSj*D}fdtSj+D}|D]0}tj| d|j,d|j-d|j.1|D]0}tj| d|j,d|j-d|j.1|r| /}d|dddf}g|ddfd |ddDR}|D] }|0|j,|||!!|1||r|s|2|tj| d"|rd#nd$|rd%nd&zd'||r|#j$j3j4j5|_6Gfd(d)j7}||tqt| d*+|_9n!|r|#j$j:|r| ;|}|tQd,ty|d-5}| st{j>| }|?t|Ad.d/d0|?|B|?|dddn #1swxYwYn| C|5}ty|d-5}| st{j>| }|?t|Ad.d/d0|?|B|?|Ddddn #1swxYwYdddn #1swxYwYtq|S)1amExport a YOLO model to TensorRT engine format. Args: onnx_file (str): Path to the ONNX file to be converted. output_file (Path | str | None): Path to save the generated TensorRT engine file. workspace (int | None): Workspace size in GB for TensorRT. half (bool, optional): Enable FP16 precision. int8 (bool, optional): Enable INT8 precision. dynamic (bool, optional): Enable dynamic input shapes. shape (tuple[int, int, int, int], optional): Input shape (batch, channels, height, width). dla (int | None): DLA core to use (Jetson devices only). dataset (ultralytics.data.build.InfiniteDataLoader, optional): Dataset for INT8 calibration. metadata (dict | None): Metadata to include in the engine file. verbose (bool, optional): Enable verbose logging. prefix (str, optional): Prefix for log messages. Returns: (str): Path to the exported engine file. Raises: ValueError: If DLA is enabled on non-Jetson devices or required precision is not set. RuntimeError: If the ONNX file cannot be parsed. Notes: TensorRT version compatibility is handled for workspace size and engine building. INT8 calibration requires a dataset and generates a calibration cache. Metadata is serialized and written to the engine file if provided. )jetpackz10.15rNz>=7.0.0T)hardz!=10.2.0z5https://github.com/ultralytics/ultralytics/pull/24367)msg z starting export with TensorRT z...z.enginei@rr z.DLA is only available on NVIDIA Jetson devicesz enabling DLA on core ztDLA requires either 'half=True' (FP16) or 'int8=True' (INT8) to be enabled. Please enable one of them and try again.zfailed to load ONNX file: c:g|]}|S) get_input.0inetworks r, zonnx2engine..s' F F Fqg"" F F Fr.c:g|]}|Sr]) get_outputr_s r,rczonnx2engine..s'IIIw!!!$$IIIr.z input "z " with shape z output " rc3\K|]&}ttdpd|zV'dS)rN)rmax)r`drLs r, zonnx2engine..s>"V"Vq3s1in1'='='A#B#B"V"V"V"V"V"Vr.)r$optriz building INT8FP1632z engine as cNeZdZdZ ddfd Zdd Zdd Zdd ZddZddZ dS)%onnx2engine..EngineCalibratoraCustom INT8 calibrator for TensorRT engine optimization. This calibrator provides the necessary interface for TensorRT to perform INT8 quantization calibration using a dataset. It handles batch generation, caching, and calibration algorithm selection. Attributes: dataset: Dataset for calibration. data_iter: Iterator over the calibration dataset. algo (trt.CalibrationAlgoType): Calibration algorithm type. batch (int): Batch size for calibration. cache (Path): Path to save the calibration cache. Methods: get_algorithm: Get the calibration algorithm to use. get_batch_size: Get the batch size to use for calibration. get_batch: Get the next batch to use for calibration. read_calibration_cache: Use existing cache instead of calibrating again. write_calibration_cache: Write calibration cache to disk. rIcacher:rNonecj|||_t||_ jjn jj|_|j |_ t||_ dS)z;Initialize the INT8 calibrator with dataset and cache path.N) IInt8Calibrator__init__datasetiter data_iterCalibrationAlgoTypeENTROPY_CALIBRATION_2MINMAX_CALIBRATIONalgo batch_sizebatchrrs)selfrxrsrRtrts r,rwz.onnx2engine..EngineCalibrator.__init__sr #,,T222& !%g+AA0C %/ !%[[ r.trt.CalibrationAlgoTypec|jS)z%Get the calibration algorithm to use.)r~rs r, get_algorithmz3onnx2engine..EngineCalibrator.get_algorithms y r.rc|jpdS)z*Get the batch size to use for calibration.r)rrs r,get_batch_sizez4onnx2engine..EngineCalibrator.get_batch_sizeszQ&r.list[int] | Nonec t|jddz }|jjdkr|dn|}t |gS#t$rYdSwxYw)zOGet the next batch to use for calibration, as a list of device memory pointers.imggo@cpurN)nextrzdevicetypetordata_ptr StopIteration)rnamesim0ss r, get_batchz/onnx2engine..EngineCalibrator.get_batchs} //6>D.2k.>%.G.G4776???TD 0011$   44 sA%A(( A65A6 bytes | Nonec|jr)|jjdkr|jSdSdS)zSUse existing cache instead of calibrating again, otherwise, implicitly return None..cacheN)rsexistssuffix read_bytesrs r,read_calibration_cachez.EngineCalibrator.read_calibration_cache sK:$$&&34:+<+H+H:0022233+H+Hr.bytesc:|j|}dS)z Write calibration cache to disk.N)rs write_bytes)rrs_s r,write_calibration_cachez=onnx2engine..EngineCalibrator.write_calibration_cachesJ**511r.N)rI)rsr:rrt)rr)rr)rr)rr)rsrrrt) __name__ __module__ __qualname____doc__rwrrrrr)rRrsr,EngineCalibratorrrs  .  ) ) ) ) ) ) ) )" ! ! ! ! ' ' ' '     3 3 3 3  2 2 2 2 2 2r.rr)rxrsz3TensorRT engine build failed, check logs for errorswblittle) byteordersigned)Er r r tensorrt ImportErrorr __version__rinfor with_suffixLoggerINFOSeverityVERBOSE min_severityBuildercreate_builder_configrr&set_memory_pool_limitMemoryPoolType WORKSPACEmax_workspace_sizeNetworkDefinitionCreationFlagEXPLICIT_BATCHcreate_networkplatform_has_fast_fp16platform_has_fast_int8r ValueError DeviceTypeDLAdefault_device_typeDLA_coreset_flag BuilderFlag GPU_FALLBACK OnnxParserparse_from_file RuntimeErrorrange num_inputs num_outputsnamerPdtypecreate_optimization_profile set_shapeadd_optimization_profileset_calibration_profilermProfilingVerbosityDETAILEDprofiling_verbosityrvr:int8_calibratorFP16build_serialized_networkopenjsondumpswritelento_bytesencode build_engine serialize) rJr3rLrNrOr8rPrRrxrSr?rTloggerbuilderconfigworkspace_bytesis_trt10flagparserinputsoutputsinpoutprofile min_shape max_shaperenginetmetarbrs ` ` @@r, onnx2enginer]sX vxx w #/948888#/:3jkkkk KPVPPCOPPPQQQGi!g677O3?((a00344:H4Oa''$$S%7%A?SSSS 1  $3! C5DEE ED$$T**G  ) 2dD  ) 2dD  OMNN N v==S===>>> D G &)^%7"c((4555^^GV , ,F  ! !) , ,EC CCDDDG F F FE'2D,E,E F F FFIIIIeG4G.H.HIIIGVV vTTsxTTSYTTTTUUUUWW vUUUUciUU#)UUVVVV45577a"b) WeBQBiW"V"V"V"VERSRTRTI"V"V"VWW  Q QC   chI5i  P P P P''000  4 4  * *7 3 3 3 K6rrt%XVVAWSW9Xrreprrsss L.,---%(%;%D"? 2? 2? 2? 2? 2? 2? 2? 2s2? 2? 2? 2D"2!1d9oo11(;;<<" " "  .,---(11'6BB >TUU U +t $ $ #z(++D **1*NNOOO &&& GGFOOO                 ! !'6 2 2 (fd;PT>U>U (YZ#z(++D **1*NNOOO &&& GGF$$&& ' ' '  ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( {  sY8AA6B V  VV/ZBY) Z)Y- -Z0Y- 1ZZZ)F)rrrrrr)rNNN)r/r0r1r2r3r4r*rr5r6r7r6r8r9rr:) NNFFFrFNNNFrI)rJr:r3rKrLrMrNrrOrr8rrPrQrRrMrSr9r?rrTr:rr:) __future__rrtypespathlibrr ultralytics.utilsrrrrr r ultralytics.utils.checksr r ultralytics.utils.torch_utilsr rr-rErr]r.r,rsI#"""""  bbbbbbbbbbbbbbbbBBBBBBBB>>>>>>>>66666< $(%)+++++`&* '7  RRRRRRRr.