5pjrE SrSSKrSSKJr SSKrSSKJrJrJrJ r SSK J r J r SSK JrJrJrJrJrJrJrJrJrJr SSKJrJr SS KJrJr \"S S 9r\"S S 9r\\ \"S S 94r!\\ S-\"S S 94r"\\ S-\"SS 94r#SSSSSSSSSS. r$\\ S-\"S\"\%"\$55SS94r&\\ S-\"SS 94r'\\(S-\"SS 94r)\\(S-\"SSS 94r*\\(S-\"S S!S 94r+\RYS"S#S$/S%9SS&\#S'\S(S4S)jj5r-S*S+1r.S,\S(\/4S-jr0\RYS.S/S0/S%9SS&\#S1\\ S-\"S2S 94S3\\ S-\"S4S 94S5\\ S-\"S6S 94S7\\ S-\"S8S 94S9\\/\"S:S;SS?S@/SA9SSSSSSSSSSSSSSSSSSSSSSSB.SC\!SD\\ \"SES 94SF\\ \"SGS 94S5\\ \"SHS 94SI\\ \"SJS 94S7\\ \"SKS 94S3\\ \"SLS 94S1\\ \"SMS 94S&\#SN\\ S-\"SOS 94S'\SP\\(\"SQS 94SR\\(\"SSS 94ST\\(S-\"SUS 94SV\\S-\"SWS 94SX\\2S-\"SYS 94SZ\S[\\ S-\"S\S]S 94S^\&S_\'S`\)Sa\*Sb\+Sc\\ S-\"SdSeS 94Sf\\ S-\"SgShS 94Si\Sj\Sk\Sl\Sm\\ S-\"Sn\"\5SoS94S(S4>Spjj5r3\RYS>Sq/SA9SSD\\ \"SES 94SC\"S5\\ S-\"SrS 94S&\#S'\S(S4 Ssjj5r4SS'\S(S4Stjjr5\RYS"Su/SA9"\55 \RYSvSwSx9"\55 \Rm\SySz9 \RYS{/S%9SSC\!S&\#S'\S(S4S|jj5r7\RY/S}QS%9SSC\!S&\#SD\\ S-\"SES 94S5\\ S-\"SHS 94SI\\ S-\"SJS 94S7\\ S-\"SKS 94SF\\ S-\"S~S 94SZ\\ S-\"SS 94SN\\ S-\"SOS 94S[\\ S-\"S\SS 94S^\&S_\'S`\)Sa\*Sb\+Sc\\ S-\"SdSS 94Sf\\ S-\"SgSS 94SP\\(S-\"SQS 94SR\\(S-\"SSS 94ST\\(S-\"SUS 94SV\\S-\"SWS 94SX\\2S-\"SYS 94S'\S(S40Sjj5r8\RYS/S%9SSC\!S&\#S\\/\"SSS 94S'\S(S4 Sjj5r9\RYS/S%9SSC\!S&\#S'\S(S4Sjj5r:\RYS/S%9SSC\!S&\#S\\/\"SSS 94S'\S(S4 Sjj5r;\RYS/S%9SSC\!S&\#S'\S(S4Sjj5rg)z2CLI commands for Hugging Face Inference Endpoints.N) Annotated)InferenceEndpointHardwareInferenceEndpointScalingMetricInferenceEndpointType_set_parallelism_in_image)CLIErrorHfHubHTTPError) EnvFileOptEnvOpt RevisionOptSecretsFileOpt SecretsOpt SoftChoiceTokenOpt get_hf_api parse_env_map typer_factory)ArgumentOption)_dataclass_to_dictoutz(Manage Hugging Face Inference Endpoints.)helpz.Interact with the Inference Endpoints catalog.zEndpoint name.z_The namespace associated with the Inference Endpoint. Defaults to the current user's namespace.customhfServellamacppsGLangteitgi tgiNeuronvLLM vLLMNeuron) rzhf-serversglangrrz tgi-neuronvllmz vllm-neuron--enginezbManaged engine image to run --custom-image with (e.g. 'vllm'). Defaults to an arbitrary container.) click_typerzVHealth check route exposed by the container (e.g. '/health'). Requires --custom-image.zDPort the container listens on (e.g. 30000). Requires --custom-image.--tensor-parallel-sizezZNumber of accelerators to shard a single model copy across (vLLM and SGLang engines only).--data-parallel-sizezFNumber of model copies to run, one per accelerator (vLLM engine only).z list | lszhf endpoints lsz"hf endpoints ls --namespace my-org)examples namespacetokenreturncR[US9nURXS9n/nUGHnURnURS05nURS05n URS05n URS 05n URURS S 5[U [5(aU RS S 5OS [U[5(aURS S 5OS [U [5(aU RSS 5OS [U [5(aU RSS 5OS [U [5(aU RSS 5OS [U [5(aU RSS 5OS [U [5(aU RSS 5OS S.5 GM [R"US S9 g![aPn[R"SU35 [ R RURRS9UeSnAff=f)z6Lists all Inference Endpoints for the given namespace.r+r*r+zListing failed: codeNstatusmodelcomputeprovidername repositorystatetask framework instanceTypevendorregion)r6r3r2r:r;instancer=r>)id_key)rlist_inference_endpointsr rerrorclick exceptionsExitresponse status_coderawgetappend isinstancedicttable) r*r+api endpointsrBresultsendpointrHr2r3r4r5s c/var/www/html/pdf-tiff/venv/lib/python3.13/site-packages/huggingface_hub/cli/inference_endpoints.pylsrSks 5 !CP0090R Gll2&$'')R(77:r*+8B5$8O8O<4UW5?5M5M&**Wb1SU1;E41H1H &"-b;EeT;R;RUYY{B7XZ?I'SW?X?XGKK;^`8B8T8R8R(,,x4XZ8B8T8R8R(,,x4XZ  $IIgf%/ P $UG,-##)C)C#D%OPsG H&A H!!H& deprecated not_availablehwcUR[;=(a& URUR- UR:$)zhWhether the namespace can deploy one replica on this hardware right now: usable status and enough quota.)r2_UNDEPLOYABLE_STATUSESmax_acceleratorsused_acceleratorsnum_acceleratorsrVs rR_is_deployabler]s8 //uB4G4G"J^J^4^bdbubu4uhardwarezhf endpoints hardwarez4hf endpoints hardware --vendor aws --accelerator gpur=z>Only show hardware hosted by this cloud provider (e.g. 'aws').r>zEOnly show hardware available in this cloud region (e.g. 'us-east-1'). acceleratorzGOnly show hardware with this accelerator (e.g. 'cpu', 'gpu', 'neuron'). instance_typezr`rar[r\s rRhardware..s(BIIr~~rGWGWY[YlYl mr^)keyquota/) r=r>r`ra instance_size memory_gb gpu_memory_gbprice_per_hourrir2id)headersr@c3J# UHn[U5(dMUv M g7fre)r]).0rVs rR hardware..s?2N2,>BBs# #zmDeploy on one of these, e.g.: hf endpoints deploy my-endpoint --repo --framework --vendor z --region z --accelerator z --instance-type z --instance-size zMNone of these can be deployed on right now, see the QUOTA and STATUS columns.zGUse '--all' to also show hardware that cannot be deployed on right now.z--vendorz--regionz --acceleratorz--instance-typez 'z ' (valid: , )zNo such hardware: z; .zCNo hardware matches all of these filters at once. Try dropping one.)r!list_inference_endpoints_hardwarer=lowerr>r`rar]sortrrZrYrrMnexthintrkjoinsorted)r*r=r>r`rarbr+rN hardware_listrVmatchingvisibleitemsflagvaluevalidvunknowns rRr_r_s?V 5 !C99I9[M  B Nbiioo/6<<>A ^ryy0FLLNB  BNN$8$8$:k>O>O>Q$Q  "b&6&6&<&<&>-BUBUBW&W  %GHbN24FrHGG LLmLnls skreg  #w23G3G2H"J]J]I^0_&` `krE sII   $??FFS  {*RYYKr~~FVW!//00A"BRBRAS U  `a  Z[ V-%H-Bii-%HIV-%H-Bii-%HI +/W2/WX"M}3]}4D4D}3]^ '  '"U  D&+[[]e:Te1779e:T%T DtfBugZ &-(@'A C'    !7!3 4A 6W  mH tD&I%H/W3]:T  sk3M-M-M6-M'M3MM-3M )MM #M M M) -M) M$M) "'M) $M) deployzChf endpoints deploy my-endpoint --repo gpt2 --framework pytorch ...zhf endpoints deploy my-endpoint --repo openai/gpt-oss-120b --framework custom --engine vllm --custom-image vllm/vllm-openai:v0.23.0 --tensor-parallel-size 8 ...)r6r))r*r:r+ min_replica max_replicascale_to_zero_timeoutscaling_metricscaling_thresholdrevision custom_imageengine health_routeporttensor_parallel_sizedata_parallel_sizecontainer_commandcontainer_argsenvenv_filesecrets secrets_file endpoint_typer6repozeThe name of the model repository associated with the Inference Endpoint (e.g. 'openai/gpt-oss-120b').r;z@The machine learning framework used for the model (e.g. 'vllm').z?The hardware accelerator to be used for inference (e.g. 'cpu').rkzNThe size or type of the instance to be used for hosting the model (e.g. 'x4').zYThe cloud instance type where the Inference Endpoint will be deployed (e.g. 'intel-icl').zTThe cloud region in which the Inference Endpoint will be created (e.g. 'us-east-1').zVThe cloud provider or vendor where the Inference Endpoint will be hosted (e.g. 'aws').r:zCThe task on which to deploy the model (e.g. 'text-classification').rzVThe minimum number of replicas (instances) to keep running for the Inference Endpoint.rzRThe maximum number of replicas (instances) to scale to for the Inference Endpoint.rzFThe duration in minutes before an inactive endpoint is scaled to zero.rz!The metric reference for scaling.rzeThe scaling metric threshold used to trigger a scale up. Ignored when scaling metric is not provided.rrz--custom-imagezhDocker image URL for the container to run (e.g. 'nexagi/sglang:v0.5.12'). Requires '--framework custom'.rrrrrrz--container-commandzpOverride the container entrypoint, as a quoted string split into tokens (e.g. "python -m sglang.launch_server").rz--container-argsz~Arguments appended to the container entrypoint, as a quoted string split into tokens (e.g. "--tp 8 --reasoning-parser qwen3").rrrrrz--typezTEndpoint access type. Defaults to 'authenticated' (token-gated, publicly reachable).c [UUUUUUS9n[UU5R5VV s0sHunn UU =(d S_M n!nn [UU5R5VV s0sHunn UU =(d S_M n"nn 0n#UbUU#S'UbUU#S'U(a[R"U5U#S'U(a[R"U5U#S'U!(aU!U#S'U"(aU"U#S '[ U S 9n$U$R "S0S U_S U_S U_SU_SU_SU_SU_SU_SU_SU _SU _SU _SU _SU_SU_SU _SU_U#D6n%[R"U%R5 [R"SUS35 gs sn nfs sn nf)zDeploy an Inference Endpoint from a Hub repository. Run `hf endpoints hardware` to list the valid `--vendor`, `--region`, `--accelerator`, `--instance-type` and `--instance-size` combinations. rrrrrr7Ntyperrrrrr.r6r8r;r`rkrar>r=r*r:r+rrrrrrzUse 'hf endpoints describe z!' to check the deployment status.) _build_custom_imagerrshlexsplitrcreate_inference_endpointrrLrHr|)&r6rr;r`rkrar>r=r*r:r+rrrrrrrrrrrrrrrrrrrcustom_image_dictrhrenv_map secrets_mapparamsrNrQs& rRrrs\,! 1- 3@X2N2T2T2VW2VJCsEKR2VGW6CG\6Z6`6`6bc6b U3 #6bKcF &v$!2~&+kk2C&D"##(;;~#> u 'y 5 !C,,   $  $   &, 4!" %H(HHX\\HH*4&0Q RSQXcs E4%E:zChf endpoints catalog deploy --repo meta-llama/Llama-3.2-1B-InstructzPThe hardware accelerator to be used for inference (e.g. 'cpu', 'gpu', 'neuron').c8[US9nURUUUUUS9n[R"UR5 g![aPn[R"SU35 [ R RURRS9UeSnAff=f)z4Deploy an Inference Endpoint from the Model Catalog.r.)repo_idr6r`r*r+zDeployment failed: r0N) r&create_inference_endpoint_from_catalogr rrBrCrDrErFrGrLrH)rr6r`r*r+rNrQrBs rRdeploy_from_catalogrs& 5 !C P==# > HHX\\ P 'w/0##)C)C#D%OPs? B A BBc [US9nURUS9n[R"SU05 g![aPn[R"SU35 [ R RURRS9UeSnAff=f)zList available Catalog models.r.zCatalog fetch failed: r0Nmodels) rlist_inference_catalogr rrBrCrDrErFrGrL)r+rNrrBs rR list_catalogrs 5 !CP++%+8 HHh  P *5'23##)C)C#D%OPs3 B A BB zhf endpoints catalog lsz list-catalogT)r6hiddencatalogr6z!hf endpoints describe my-endpointc2[US9nURXUS9n[R"UR5 g![aPn[R"SU35 [ R RURRS9UeSnAff=f)z+Get information about an existing endpoint.r.r6r*r+zFetch failed: r0N) rget_inference_endpointr rrBrCrDrErFrGrLrHr6r*r+rNrQrBs rRdescribers 5 !CP--4TY-Z HHX\\ P N5'*+##)C)C#D%OP< BA BB)z/hf endpoints update my-endpoint --min-replica 2z8hf endpoints update my-endpoint --tensor-parallel-size 8zdhf endpoints update my-endpoint --container-args "--enable-auto-tool-choice --tool-call-parser lfm2"zBThe machine learning framework used for the model (e.g. 'custom').zrThe specific model revision to deploy on the Inference Endpoint (e.g. '6c0e6080953db56375760c0471a8c5f2929baf11').aDocker image URL for the container to run (e.g. 'nexagi/sglang:v0.5.12'). Replaces the image currently configured on the endpoint rather than patching it, so pass the engine and container settings you want to keep along with it, run 'hf endpoints describe NAME' first to see them.zOverride the container entrypoint, as a quoted string split into tokens (e.g. "python -m sglang.launch_server"). Replaces the current value; pass an empty string to clear it.a_Arguments appended to the container entrypoint, as a quoted string split into tokens (e.g. "--enable-auto-tool-choice --tool-call-parser lfm2"). Replaces the arguments currently set on the endpoint rather than adding to them, so include the ones you want to keep, run 'hf endpoints describe NAME' first to see them. Pass an empty string to clear them.c[U U U U U bU OSU bUOSS9n[US9nURUUUUUUUUb[R"U5OSUb[R"U5OSU cU OSU cUOSUUUUUUUUUS9n[ R"UR5 g![ aPn[ R"SU35 [RRURRS9UeSnAff=f)zUpdate an existing endpoint.Nrr.)r6r*r8r;rr:rrrrrr`rkrarrrrrr+zUpdate failed: r0)rrupdate_inference_endpointrrr rrBrCrDrErFrGrLrH)r6r*rr`rkrar;rr:rrrrrrrrrrrrrr+rrNrQrBs rRupdaters)|,! 5A5M1SW1=1I-t  5 !CP00*@Q@]ekk*;rs 8   <   ), F G!QR   "#%  $J  !#  $J n      $J d#456 q  $J e $J S "$J  i $J  U '8:^&_`"&& & &a&F'80T>#     5c c  d TU W c  d [\ ^  c  d ]^ `c  d RS Uc $   u  %c 4 5c 6 7c c L M Oj#    #'26.2  #' wT wT  x   wT S  wT R  wT( a  )wT4 l  5wT@  g   AwTL  i   MwTZ[wT\  d V   ]wTh iwTj i  kwTv e  wwTB% d Y  CwTN&-4  OwTZ!  x  [wTfgwTh d  {  iwTv wwTx!ywTz {wT|0}wT~,wT@! d  !;   AwTT d  <   UwTh iwTjkwTlmwTn!owTp d  !"78g  qwT@ AwTwTt (.s-tu "  x     d c     !vF !  !  ! 0I/JKLYN40>9-=>?#       @  #  #'26.2  e[ [[  d x   [ d R  [ d a  [* d l  +[6 d U  7[B d F  C[N  d V   O[Z d  o   [[p q[r!s[t u[v0w[x,y[z! d  !4   {[P d  f   Q[h d i  i[t d e  u[@% d Y  A[L&-4  M[X!  x  Y[d e[f g[[|;<=# 0 00  w9: < 0 0 0>0,:;<#       =  ;<=#  '  'j    >6BCD#       E &#'+%)/*/ $J/* / * / * /d / D[/r^