j6&J dZddlmZddlZddlmZddlmZddlm Z m Z m Z ddl m Z e d Ze d Zeeejd fZeedzejd fZeedzejd fZedddg dVdede ddfdZeddgddddddddddedeeejd fdeeejd fdeeejd fd!eeejd" fd#eeejd$ fd%eeejd& fd'eeejd( fded)eedzejd* fde d+eeejd, fd-eeejd. fd/eedzejd0 fd1eedzejd2 fd3eedzejd4 fddf"d5Zedd6g dWdeeejd fdedeedzejd7 fdede ddf d8Z dXde ddfd9Zedd:geed;d<=eeed>?ed@g dVdedede ddfdAZedBg dYdededeedzejd fdeedzejd fd!eedzejd" fd#eedzejd$ fdeedzejdC fdDeedzejdE fd)eedzejd* fd+eedzejd, fd-eedzejd. fd/eedzejd0 fd1eedzejd2 fd3eedzejd4 fde ddf dFZedGg dZdededIee ejdJdK fde ddf dLZ!edMg dVdedede ddfdNZ"edOg dZdededPee ejdQdR fde ddf dSZ#edTg dVdedede ddfdUZ$dS)[z2CLI commands for Hugging Face Inference Endpoints.) AnnotatedN)InferenceEndpointScalingMetric)HfHubHTTPError)TokenOpt get_hf_api typer_factory)outz(Manage Hugging Face Inference Endpoints.)helpz.Interact with the Inference Endpoints catalog.zEndpoint name.z_The namespace associated with the Inference Endpoint. Defaults to the current user's namespace.z list | lszhf endpoints lsz"hf endpoints ls --namespace my-org)examples namespacetokenreturnct|} |||}nI#t$r<}tjd|t j|jj|d}~wwxYwg}|D]}|j }| di}| di} | di} | d i} | | d d t| tr| d d nd t|tr| d d nd t| tr| dd nd t| tr| dd nd t| tr| dd nd t| tr| dd nd t| tr| dd nd dtj|d dS)z6Lists all Inference Endpoints for the given namespace.r)r rzListing failed: codeNstatusmodelcomputeprovidername repositorystatetask framework instanceTypevendorregion)rrrrrinstancerr )id_key)rlist_inference_endpointsrr errortyperExitresponse status_coderawgetappend isinstancedicttable) r rapi endpointsr$resultsendpointr)rrrrs j/var/www/html/Carbon-Document/venv/lib/python3.11/site-packages/huggingface_hub/cli/inference_endpoints.pylsr4$sP 5 ! ! !CE009E0RR EEE ,U,,---jen8999uDEG  l2&&$$'')R((77:r**++8B5$8O8OW<444UW5?5M5MU&**Wb111SU1;E41H1HP &"---b;EeT;R;RZUYY{B777XZ?I'SW?X?X`GKK;;;^`8B8T8R8RZ(,,x444XZ8B8T8R8RZ(,,x444XZ   Igf%%%%%%s* A07A++A0deployzChf endpoints deploy my-endpoint --repo gpt2 --framework pytorch ...)rr )r rr min_replica max_replicascale_to_zero_timeoutscaling_metricscaling_thresholdrrepozeThe name of the model repository associated with the Inference Endpoint (e.g. 'openai/gpt-oss-120b').rz@The machine learning framework used for the model (e.g. 'vllm'). acceleratorz?The hardware accelerator to be used for inference (e.g. 'cpu'). instance_sizezNThe size or type of the instance to be used for hosting the model (e.g. 'x4'). instance_typezYThe cloud instance type where the Inference Endpoint will be deployed (e.g. 'intel-icl').r zTThe cloud region in which the Inference Endpoint will be created (e.g. 'us-east-1').rzVThe cloud provider or vendor where the Inference Endpoint will be hosted (e.g. 'aws').rzCThe task on which to deploy the model (e.g. 'text-classification').r6zVThe minimum number of replicas (instances) to keep running for the Inference Endpoint.r7zRThe maximum number of replicas (instances) to scale to for the Inference Endpoint.r8zFThe duration in minutes before an inactive endpoint is scaled to zero.r9z!The metric reference for scaling.r:zeThe scaling metric threshold used to trigger a scale up. Ignored when scaling metric is not provided.ct| }||||||||||| | | | ||| }tj|jdS)z3Deploy an Inference Endpoint from a Hub repository.r)rrrr<r=r>r rr rrr6r7r9r:r8N)rcreate_inference_endpointr r-r))rr;rr<r=r>r rr rrr6r7r8r9r:r/r2s r3r5r5Gsyl 5 ! ! !C,, ## %+3!-H$HX\zChf endpoints catalog deploy --repo meta-llama/Llama-3.2-1B-InstructzPThe hardware accelerator to be used for inference (e.g. 'cpu', 'gpu', 'neuron').c"t|} ||||||}nI#t$r<}tjd|t j|jj|d}~wwxYwtj |j dS)z4Deploy an Inference Endpoint from the Model Catalog.r)repo_idrr<r rzDeployment failed: rN) r&create_inference_endpoint_from_catalogrr r$r%r&r'r(r-r))r;rr<r rr/r2r$s r3deploy_from_catalogrEs& 5 ! ! !C E==# >   EEE ///000jen8999uDEHX\- A37A..A3ct|} ||}nI#t$r<}tjd|t j|jj|d}~wwxYwtj d|idS)zList available Catalog models.rzCatalog fetch failed: rNmodels) rlist_inference_catalogrr r$r%r&r'r(r-)rr/rHr$s r3 list_catalogrJs 5 ! ! !CE++%+88 EEE 2522333jen8999uDEHh      s) A/7A**A/zhf endpoints catalog lsz list-catalogT)rhiddencatalogrz!hf endpoints describe my-endpointct|} ||||}nI#t$r<}tjd|t j|jj|d}~wwxYwtj |j dS)z+Get information about an existing endpoint.rrr rzFetch failed: rN) rget_inference_endpointrr r$r%r&r'r(r-r)rr rr/r2r$s r3describerRs 5 ! ! !CE--49TY-ZZ EEE *5**+++jen8999uDEHX\+ A17A,,A1z/hf endpoints update my-endpoint --min-replica 2zBThe machine learning framework used for the model (e.g. 'custom').revisionzrThe specific model revision to deploy on the Inference Endpoint (e.g. '6c0e6080953db56375760c0471a8c5f2929baf11').c6t|} ||||||||||| | | | | |}nI#t$r<}tjd|t j|jj|d}~wwxYwtj |j dS)zUpdate an existing endpoint.r)rr rrrTrr<r=r>r6r7r8r9r:rzUpdate failed: rN) rupdate_inference_endpointrr r$r%r&r'r(r-r))rr r;r<r=r>rrTrr6r7r8r9r:rr/r2r$s r3updaterWs^ 5 ! ! !CE00#''##"7)/1  " EEE +E++,,,jen8999uDEHX\s$7 A=7A88A=zhf endpoints delete my-endpointFyesz--yeszSkip confirmation prompts.cTtjd|d|t|} ||||nI#t$r<}tjd|t j|jj |d}~wwxYwtj d |d | dS) z)Delete an Inference Endpoint permanently.zDelete endpoint 'z'?)rXrrOzDelete failed: rNz Deleted 'z'.rM) r confirmrdelete_inference_endpointrr$r%r&r'r(result)rr rXrr/r$s r3deleter]dsK,D,,,#6666 5 ! ! !CE %%49E%RRRR EEE +E++,,,jen8999uDEJ#4###$//////sA B 7BB zhf endpoints pause my-endpointct|} ||||}nI#t$r<}tjd|t j|jj|d}~wwxYwtj |j dS)zPause an Inference Endpoint.rrOzPause failed: rN) rpause_inference_endpointrr r$r%r&r'r(r-r)rQs r3pauser`{s 5 ! ! !CE//TYV[/\\ EEE *5**+++jen8999uDEHX\rSzhf endpoints resume my-endpointfail_if_already_runningz--fail-if-already-runningzWIf `True`, the method will raise an error if the Inference Endpoint is already running.c"t|} ||||| }nI#t$r<}tjd|t j|jj|d}~wwxYwtj |j dS)zResume an Inference Endpoint.r)rr r running_okzResume failed: rN) rresume_inference_endpointrr r$r%r&r'r(r-r))rr rarr/r2r$s r3resumeres 5 ! ! !C E0022 1   EEE +E++,,,jen8999uDEHX\rFz&hf endpoints scale-to-zero my-endpointct|} ||||}nI#t$r<}tjd|t j|jj|d}~wwxYwtj |j dS)z$Scale an Inference Endpoint to zero.rrOzScale To Zero failed: rN) r scale_to_zero_inference_endpointrr r$r%r&r'r(r-r)rQs r3 scale_to_zerorhs 5 ! ! !CE77TY^c7dd EEE 2522333jen8999uDEHX\rS)NN)NNNN)N)NNNNNNNNNNNNNN)NFN)%__doc__typingrr%$huggingface_hub._inference_endpointsrhuggingface_hub.errorsr _cli_utilsrrr _outputr ie_cli catalog_appstrArgumentNameArgOptionNameOpt NamespaceOptcommandr4intfloatr5rErJ add_typerrRrWboolr]r`rerhrAr3r}s 88 OOOOOO111111;;;;;;;;;; F G G Gm!QRRR  EN()))+  $JEL&''')  $JEL n  '8:^&_``"&&& & &&&a`&DX)n(opp\#  ehhh h   x     h  S    h  R    h(  a    )h4  l    5h@   g     AhL   i     MhZ[h\  d  V     ]hh ihj  i    khv  e    whB% d  Y    ChN&- 4    OhZ!   x    [hf ghhhqphV (.s-tuu "   x       d  c       !vuF ! !  !  ! ! ! !L 0I/JKKLYYY0N400>>>9---=>??#          @?  KLMM# Wdd dd  d  x     d d  R    d d  a    d* d  l    +d6 d  U    7dB d  F    CdN  d  V     OdZ d  i    [df d  e    gdr% d  Y    sd~&- 4    dJ!   x    KdV WdX YdddNMdN;<==# 00 00   W#?@@@ B 0 0 000>=0,:;<<#          =<  ;<==#  '  'j      >=6BCDD#          ED   rA