5pjb SSKrSSKrSSKJrJr SSKJr SSKJr SSKJ r J r SSK J r J r SSKJrJrJr \ (aSS KJr SS KJr SS KJr \R0"\5rS \S \4SjrSrSrSSS.S\S\S-S\S-S \4Sjjr "SS\!\5r""SS\!\5r#"SS\!\5r$\"SS55r%\"SS55r&g) N) dataclassfield)datetime)Enum) TYPE_CHECKINGOptional)InferenceEndpointErrorInferenceEndpointTimeoutError) get_sessionloggingparse_datetimeHfApiInferenceClientAsyncInferenceClient custom_imagereturncSU;aSU0$U$)aBuild the `model.image` payload of an Inference Endpoint from a user-provided image dict. `model.image` is a union keyed by variant (`{"vLLM": {...}}`, `{"custom": {...}}`, ...). Only a flat container dict has a top-level `url` (required server-side, and no variant is named `url`), so dicts with one are wrapped in `{"custom": ...}`. Everything else is forwarded as-is, so variants added to the API later work without a release. urlcustom)rs `/var/www/html/pdf-tiff/venv/lib/python3.13/site-packages/huggingface_hub/_inference_endpoints.py_build_endpoint_image_payloadrs ,'' )sGLangvLLM)r)tensor_parallel_sizedata_parallel_sizeimager r!c <[[U5S5nUc [S5eUS[4US[44HFupEnUcM X6;dM[ R "SUSUSSRU5S 3[5 MH U0XE0nUbXUS 'UbX US 'U$) aWrite the parallelism sizes into a `model.image` payload. They are engine settings, so they live inside the engine config (`{"vLLM": {"url": ..., "tensorParallelSize": 8}}`) rather than at the top level. Returns a new image dict, the input is left untouched. Nz=Cannot set the parallelism sizes: the image payload is empty.r r!`z!` is not a known setting of the 'zA' image: the API will silently drop it. Engines that support it: z, .tensorParallelSizedataParallelSize) nextiter ValueError_TENSOR_PARALLEL_IMAGE_KEYS_DATA_PARALLEL_IMAGE_KEYSwarningswarnjoin UserWarning)r"r r! image_keyvaluename supporteds r_set_parallelism_in_imager5)sT%[$'IXYY 57RS 13LM#Y  !; MMD6:9+F--1YYy-A,B!E  #,5+, -E'1Ei-.%/Ai+, Lrc4\rSrSrSrSrSrSrSrSr Sr S r S r g ) InferenceEndpointStatusKpending initializingupdating updateFailedrunningpausedfailed scaledToZerorN) __name__ __module__ __qualname____firstlineno__PENDING INITIALIZINGUPDATING UPDATE_FAILEDRUNNINGPAUSEDFAILEDSCALED_TO_ZERO__static_attributes__rrrr7r7Ks*G!LH"MG F F#Nrr7c$\rSrSrSrSrSrSrSrg)InferenceEndpointTypeVpublic protected authenticatedprivaterN) rArBrCrDPUBlIC PROTECTED AUTHENTICATEDPRIVATErMrrrrOrOVs FI#MGrrOc\rSrSrSrSrSrg)InferenceEndpointScalingMetric]pendingRequests hardwareUsagerN)rArBrCrDPENDING_REQUESTSHARDWARE_USAGErMrrrrZrZ]s ($NrrZc#>\rSrSr%Sr\"SS9r\\S'\\S'\"SS9r \\S'\"SS9r \ \S'\"SS9r \\S '\"SS9r \S -\S '\"SSS 9r\\S '\"SSS 9r\\S'\"SSS 9r\\S'\"SSS 9r\\S'\"SSS 9r\\S'\"SSS 9r\\S'\"SS9r\\S'\"SSS9r\\-S -\S'\"SSS9rS\S'\S9S\S\S\\-S -S\SSS4 Sjj5rS:Sjr\S;Sj5r \SS#jr$S S S S S S S S S S S S S S S S S$.S%\S -S&\S -S'\S -S(\"S -S)\"S -S*\"S -S\S -S \S -S\S -S\S -S+\S -S,\%\S -S-\%\S -S.\"S -S/\"S -S0\\\4S -SS4"S1jjr&S>S2jr'S?S3\SS4S4jjr(S>S5jr)S:S6jr*S:S7jr+S8r,g )@InferenceEndpointba5 Contains information about a deployed Inference Endpoint. Args: name (`str`): The unique name of the Inference Endpoint. namespace (`str`): The namespace where the Inference Endpoint is located. repository (`str`): The name of the model repository deployed on this Inference Endpoint. status ([`InferenceEndpointStatus`]): The current status of the Inference Endpoint. url (`str`, *optional*): The URL of the Inference Endpoint, if available. Only a deployed Inference Endpoint will have a URL. framework (`str`): The machine learning framework used for the model. revision (`str`): The specific model revision deployed on the Inference Endpoint. task (`str`): The task associated with the deployed model. created_at (`datetime.datetime`): The timestamp when the Inference Endpoint was created. updated_at (`datetime.datetime`): The timestamp of the last update of the Inference Endpoint. type ([`InferenceEndpointType`]): The type of the Inference Endpoint (public, authenticated, private). raw (`dict`): The raw dictionary data returned from the API. token (`str` or `bool`, *optional*): Authentication token for the Inference Endpoint, if set when requesting the API. Will default to the locally saved token if not provided. Pass `token=False` if you don't want to send your token to the server. Example: ```python >>> from huggingface_hub import get_inference_endpoint >>> endpoint = get_inference_endpoint("my-text-to-image") >>> endpoint InferenceEndpoint(name='my-text-to-image', ...) # Get status >>> endpoint.status 'running' >>> endpoint.url 'https://my-text-to-image.region.vendor.endpoints.huggingface.cloud' # Run inference >>> endpoint.client.text_to_image(...) # Pause endpoint to save $$$ >>> endpoint.pause() # ... # Resume and wait for deployment >>> endpoint.resume() >>> endpoint.wait() >>> endpoint.client.text_to_image(...) ``` F)initr3 namespace repositorystatus health_routeNr)reprrc frameworkrevisiontask created_at updated_attype)rhraw)rhcompare_tokenr_apitokenapircNUc SSKJn U"5nUc URnU"XX4S9$)z&Initialize object from raw dictionary.r r)rordrqrr)hf_apirrs)clsrordrsrtrs rfrom_rawInferenceEndpoint.from_raws0 ; %'C =IIEsHHrc$UR5 g)z$Populate fields from raw dictionary.N)_populate_from_rawselfs r __post_init__InferenceEndpoint.__post_init__s !rctURc [S5eSSKJn U"URURS9$)zReturns a client to make predictions on this Inference Endpoint. Returns: [`InferenceClient`]: an inference client pointing to the deployed endpoint. Raises: [`InferenceEndpointError`]: If the Inference Endpoint is not yet deployed. Cannot create a client for this Inference Endpoint as it is not yet deployed. Please wait for the Inference Endpoint to be deployed using `endpoint.wait()` and try again.r rmodelrs)rr inference._clientrrq)r}rs rclientInferenceEndpoint.clientsA 88 (o  7((++  rctURc [S5eSSKJn U"URURS9$)a Returns a client to make predictions on this Inference Endpoint. Returns: [`AsyncInferenceClient`]: an asyncio-compatible inference client pointing to the deployed endpoint. Raises: [`InferenceEndpointError`]: If the Inference Endpoint is not yet deployed. rr rr)rr "inference._generated._async_clientrrq)r}rs r async_clientInferenceEndpoint.async_clientsB 88 (o  M#((++  rtimeout refresh_everycUbUS:a [S5eUS::a [S5e[R"5nUR[R:Xa[ SUR S35eUR[R:Xa[ SUR S35eUR[R:XaURbURRS5SURRS53n[5RX@RR!UR"S9S 9nUR$S :Xa[&R)S 5 U$Ub'[R"5U- U:a [+S 5e[&R)S URSUS35 [R,"U5 UR/5 GM)aWait for the Inference Endpoint to be deployed. Information from the server will be fetched every 1s. If the Inference Endpoint is not deployed after `timeout` seconds, a [`InferenceEndpointTimeoutError`] will be raised. The [`InferenceEndpoint`] will be mutated in place with the latest data. Args: timeout (`int`, *optional*): The maximum time to wait for the Inference Endpoint to be deployed, in seconds. If `None`, will wait indefinitely. refresh_every (`int`, *optional*): The time to wait between each fetch of the Inference Endpoint status, in seconds. Defaults to 5s. Returns: [`InferenceEndpoint`]: the same Inference Endpoint, mutated in place with the latest data. Raises: [`InferenceEndpointError`] If the Inference Endpoint ended up in a failed state. [`InferenceEndpointTimeoutError`] If the Inference Endpoint is not deployed after `timeout` seconds. rz`timeout` cannot be negative.z!`refresh_every` must be positive.zInference Endpoint z> failed to deploy. Please check the logs for more information.z> failed to update. Please check the logs for more information./)rs)headersz'Inference Endpoint is ready to be used.z>aealal0m77 ! r running_okcURRURURXRS9nUR UlUR 5 U$)aResume the Inference Endpoint. This is an alias for [`HfApi.resume_inference_endpoint`]. The current object is mutated in place with the latest data from the server. Args: running_ok (`bool`, *optional*): If `True`, the method will not raise an error if the Inference Endpoint is already running. Defaults to `True`. Returns: [`InferenceEndpoint`]: the same Inference Endpoint, mutated in place with the latest data. )r3rdrrs)rrresume_inference_endpointr3rdrqror{)r}rrs rresumeInferenceEndpoint.resumesPii11dnnS^S^2 77 ! rcURRURURURS9nUR UlUR 5 U$)a;Scale Inference Endpoint to zero. An Inference Endpoint scaled to zero will not be charged. It will be resumed on the next request to it, with a cold start delay. This is different from pausing the Inference Endpoint with [`InferenceEndpoint.pause`], which would require a manual resume with [`InferenceEndpoint.resume`]. This is an alias for [`HfApi.scale_to_zero_inference_endpoint`]. The current object is mutated in place with the latest data from the server. Returns: [`InferenceEndpoint`]: the same Inference Endpoint, mutated in place with the latest data. r)rr scale_to_zero_inference_endpointr3rdrqror{rs r scale_to_zeroInferenceEndpoint.scale_to_zerosMii88diiSWSaSaimitit8u77 ! rcvURRURURURS9 g)a4Delete the Inference Endpoint. This operation is not reversible. If you don't want to be charged for an Inference Endpoint, it is preferable to pause it with [`InferenceEndpoint.pause`] or scale it to zero with [`InferenceEndpoint.scale_to_zero`]. This is an alias for [`HfApi.delete_inference_endpoint`]. rN)rrdelete_inference_endpointr3rdrqr|s rdeleteInferenceEndpoint.deletes, ++dnn\`\g\g+hrc(URSUlURSSUlURSSUlURSR S5UlURSUlURSSUlURSS UlURSS Ul [URSS 5Ul [URSS 5Ul URS Ul g)zmPopulate fields from raw dictionary. Called in __post_init__ + each time the Inference Endpoint is updated. r3rrerfstater healthRouterirjrk createdAt updatedAtrnN)ror3rerfrrrgrirjrkrrlrmrnr|s rr{$InferenceEndpoint._populate_from_raws HHV$ ((7+L9hhx(1 88H%))%0 HH]3'*;7)*5 HHW%f- (();K)HI(();K)HIHHV$ r) rlrirgr3rorerjrfrkrnrmr)NN)rN)rr)rr)N)rra)T)-rArBrCrD__doc__rr3str__annotations__rerfr7rgrrirjrkrlrrmrnrOrodictrqboolrr classmethodrrxr~propertyrrintrrlistrrrrrr{rMrrrrarabs9x5!D#!N'J'&+&7F #75)L#)'Ct'E2Is2u51Hc15u-D#- e%8J8 e%8J8"'U"?D ?5!C!!&5% @FC$J @ue4D'4bf I I#& I/2TzD/@ INVW^N_ I  I I"  *  *3C$J3c3J]3j #'$($("&"&,0!% $#$(.2+/+/)-)-)d4Z d Tz d Tz d4Zd4Zd #Tzd$Jd:d*dDjdTkd  9t+!d"S D(#d$"Dj%d& $J'd(c3h$&)d* +dL$1D*$i%rrac \rSrSr%Sr\\S'\\S'\\S'\\S'\\S'\\S'\\S '\\S '\S -\S '\\S '\S -\S'\\S'\\S'\\S'\\S'\ S\ S\S\SS4Sj5r Sr g )InferenceEndpointHardwareia Contains information about a hardware configuration available for Inference Endpoints. The `vendor`, `region`, `accelerator`, `instance_type` and `instance_size` fields are exactly the values to pass to [`create_inference_endpoint`] (or `hf endpoints deploy`) to deploy on this hardware. Args: id (`str`): Unique hardware identifier, e.g. `"aws-us-east-1-nvidia-l4-x1"`. vendor (`str`): The cloud provider hosting the hardware, e.g. `"aws"`. region (`str`): The cloud region the hardware is available in, e.g. `"us-east-1"`. accelerator (`str`): The type of hardware accelerator, e.g. `"cpu"`, `"gpu"` or `"neuron"`. instance_type (`str`): The cloud instance type, e.g. `"nvidia-l4"`. instance_size (`str`): The instance size multiplier, e.g. `"x1"`. architecture (`str`): Human-readable hardware description, e.g. `"Nvidia L4"`. num_accelerators (`int`): Number of accelerator units per replica. num_cpus (`int`, *optional*): Number of vCPUs per replica. memory_gb (`float`): RAM per replica, in GB. gpu_memory_gb (`int`, *optional*): Total GPU memory per replica, in GB (i.e. summed over `num_accelerators`). `None` for non-GPU hardware. price_per_hour (`float`): Cost per replica per hour, in USD. status (`str`): Availability of the hardware: `"available"`, `"low_availability"`, `"not_available"`, `"reserved"` or `"deprecated"`. max_accelerators (`int`): Maximum number of accelerators of this type the namespace is allowed to run. used_accelerators (`int`): Number of accelerators of this type currently used by the namespace. Example: ```python >>> from huggingface_hub import list_inference_endpoints_hardware >>> hardware = list_inference_endpoints_hardware() >>> hardware[0] InferenceEndpointHardware(id='aws-us-east-1-nvidia-l4-x1', vendor='aws', region='us-east-1', ...) ``` idvendorregionrrr architecturenum_acceleratorsNnum_cpus memory_gb gpu_memory_gbprice_per_hourrfmax_acceleratorsused_acceleratorsrorcUSnU"USUUUSUSUSUSUSURS5US URS 5US US US USS9$)zhInitialize object from a raw compute dictionary, nested under a vendor and a region in the API response.quotarr instanceType instanceSizernumAcceleratorsnumCpusmemoryGb gpuMemoryGb pricePerHourrfmaxAcceleratorsusedAccelerators)rrrrrrrrrrrrrfrr)r)rwrorrrs rrx"InferenceEndpointHardware.from_raw5sG 4yM*n-n-^, !23WWY'*o''-0~.x="#45#$67  rr) rArBrCrDrrrrfloatrrrxrMrrrrrs.` G K KDj: K 4 C  A\  rr)'rr- dataclassesrrrenumrtypingrrhuggingface_hub.errorsr r utilsr r rrvrrrrr get_loggerrArrrr+r,rr5rr7rOrZrarrrrrs (*X772H   H %    1% (,%)  *d   D$c4$C%S$%  M%M% M%`  U U  U r