ՑiYJSSKrSSKrSSKrSSKrSSKJrJrJr SSK J r J r SSK J r SSKJr SSKJr /SQr"S S \ 5rS r"S S 5rSqSrSrSrSS\4SjjrSSjr\"S5"SS\55rg)N)AnyDictOptional) BaseModelmodel_validator)Literal)logging)class_requires_deps)fastdeploy-server vllm-server sglang-servermlx-vlm-serverllama-cpp-serverc\rSrSr%Sr\S\S'Sr\\ \S'Sr \ \S'Sr \\ \ \4\S '\"S S 9S 5rS rg) GenAIConfig$native)rr r rrrbackendN server_urlmax_concurrency client_kwargsafter)modecUR[;a/URc"[S[ UR5S35eU$)Nz(`server_url` must not be `None` for the z backend.)rSERVER_BACKENDSr ValueErrorreprselfs e/var/www/html/banglarbhumi/venv/lib/python3.13/site-packages/paddlex/inference/models/common/genai.pycheck_server_urlGenAIConfig.check_server_url1sA <[R"5Tl[R"TR5 TRR 5 TRR 5 TR5 TRR 5 STl g!TR5 TRR 5 STl f=f)NT) asyncionew_event_loopr6set_event_loopr;set run_forever_cleanup_loop_internalr<r8r sr" _run_loop,_AsyncThreadManager.start.._run_loopzs..0DI  " "499 -    ! ! # $ %%' ++-((,,.# ++-((,,.# s B##3CT)targetdaemon) is_runningr=r8r;clearr<r9Threadr7startwait)r!rHs` r"rO_AsyncThreadManager.startgs ??   $  !   &&( $" &&iE    r%crURcg[R"UR5nU(ak[R"S[ U5S35 UHnUR 5 M URR[R"USS065 URRURR55 [URS5(a3URRURR55 URR5 [R"S5 g![a#n[R"SU35 SnANYSnAff=f!URR5 [R"S5 f=f) a Perform cleanup operations within the event loop thread. IMPORTANT: This method MUST be called from the event loop thread (i.e., in the finally block of _run_loop) because asyncio operations like `all_tasks()`, `task.cancel()` are NOT thread-safe. Cleanup sequence: 1. Cancel all remaining tasks 2. Wait for cancellation to complete 3. Shutdown async generators (prevents ResourceWarning) 4. Shutdown default executor (Python 3.9+) 5. Close the event loop Nz Cancelling z pending tasks during cleanupreturn_exceptionsTshutdown_default_executorz!Error during event loop cleanup: zEvent loop closed successfully)r6rB all_tasksr debuglencancelrun_until_completegathershutdown_asyncgenshasattrrT Exceptionwarningclose)r!pendingtaskes r"rG*_AsyncThreadManager._cleanup_loop_internals; 99   <'' 2G CL>9VWX$DKKM$  ,,NNGDtD II ( ()E)E)G Htyy"=>> ,,TYY-P-P-RS IIOO  MM: ;  E OO?sC D D E IIOO  MM: ;s*DE FE<7F<FF2F6timeoutc^^^TR5(dgSTlUU4Sjn[R"U"5TR5nUR TS-S9 TRRTRR5 TRR!S S9nU(d[R"S 5 TR"bNTR"R%SS9 TR"R'5(a[R"S 5 STlSTlg![ RRa! [R"STS-S35 GN[a$n[R"SU35 SnAGN6SnAff=f![a GNf=f) a! Gracefully stop the event loop. This method performs a graceful shutdown sequence: 1. Sets shutting_down flag to reject new tasks 2. Schedules a graceful shutdown coroutine in the event loop that: - Waits for pending tasks to complete (with timeout) - Cancels tasks that don't complete in time 3. Signals the loop to stop 4. Waits for cleanup to complete in the loop thread 5. Joins the background thread Args: timeout: Maximum seconds to wait for pending tasks to complete. Tasks not completed within this time will be cancelled. Default is 5.0 seconds. Thread Safety: This method is safe to call from any thread. It uses only thread-safe mechanisms (run_coroutine_threadsafe, call_soon_threadsafe, Events) to communicate with the event loop thread. NTc># [R"5n[R"TR5Vs/sH"nXLdM UR 5(aM UPM$ nnU(d[ R "S5 g[ R "S[U5STS35 [R"UT[RS9IShvN up4U(a][ R"S[U5STS 35 UHnUR5 M [R"US S 06IShvN [ R "S 5 gs snfNN"7f) z Graceful shutdown coroutine that runs inside the event loop. All asyncio operations here are thread-safe because this coroutine executes in the event loop thread. z,No pending tasks to wait for during shutdownNzGraceful shutdown: waiting for z pending tasks (timeout=zs))rd return_whenzGraceful shutdown: cancelling z$ tasks that did not complete within srSTz%Graceful shutdown coroutine completed) rB current_taskrUr6doner rVrWrP ALL_COMPLETEDr^rXrZ)ritr`rj still_pendingrar!rds r"_graceful_shutdown4_AsyncThreadManager.stop.._graceful_shutdowns:#//1L!**49955A(125   LM MM1#g,@#9B(  )0 g6K6K)# D4S5G4HI44;9A? *DKKM* nnmLtLLL MMA BC #Ms<8EEEE$A-EEA%E7E8!EEg@rdz"Graceful shutdown timed out after zs, forcing loop stopz Error during graceful shutdown: @z-Event loop cleanup did not complete within 5szYBackground thread did not terminate in time. Some resources may not be properly released.)rLr=rBrun_coroutine_threadsafer6result concurrentfutures TimeoutErrorr r^r]call_soon_threadsafestop RuntimeErrorr<rPr7joinis_alive)r!rdrnfuturerbcleanup_completeds`` r"rx_AsyncThreadManager.stopsq.  #* CX D556H6JDIIVF MM'C-M 0  II * *499>> : !4499#9F OOK L ;; " KK  S  ){{##%%C   =!!..  OO4Ws]OD$$  D OO>qcB C C D    s/8D0!/F0;F. F7FF F,+F,cUR5(d [S5eUR(a [S5e[R"XR 5nU$)a) Submit a coroutine to be executed in the background event loop. This is the primary method for bridging sync and async code. The coroutine will be scheduled to run in the background thread's event loop. Args: coro: A coroutine object to be executed Returns: concurrent.futures.Future: A future that can be used to: - Wait for the result: future.result(timeout=...) - Check completion: future.done() - Cancel the task: future.cancel() Raises: RuntimeError: If the event loop is not running or is shutting down Thread Safety: This method is safe to call from any thread. It uses `asyncio.run_coroutine_threadsafe()` which is explicitly documented as thread-safe. Example: future = manager.run_async(fetch_data()) result = future.result(timeout=30) zEvent loop is not runningzxEvent loop is shutting down, cannot accept new tasks. Please ensure all async operations complete before calling stop().)rLryr=rBrrr6)r!coror|s r" run_async_AsyncThreadManager.run_async3sV:  :; ;   U  11$ B r%cURSL=(a7 URR5(+=(a UR(+$)z Check if the event loop is currently running and accepting tasks. Returns: bool: True if the loop is running and not closed/stopped N)r6 is_closedr8r s r"rL_AsyncThreadManager.is_running]s4yy$WTYY-@-@-B)BW4<rOrGfloatrxrrLpropertyrr.r&r%r"r3r3AsG6$'!R0SSKJn [TU] 5 XlX0lUc[ UR5SS9nX@lSU;aSUS'U"SSU0UD6Ul [R"UR 5Ul g) Nr) AsyncOpenAI rpapi_keynullbase_urlr&) rrsuperr>r_max_concurrencyr_get_model_name _model_name_clientrB Semaphore _semaphore)r!rrr model_namekwargsr __class__s r"r>GenAIClient.__init__s} '  /  "4#7#7#92FJ% F " &F9 "?H?? !++D,A,ABr%cUR$r)rr s r" openai_clientGenAIClient.openai_clients ||r%Frc J^U4Sjn[U"STRUS.UD6US9$)Nc># TRIShvN TRRRR"U0UD6IShvN sSSS5IShvN $NMNN !,IShvN (df  g=f7fr)rrchat completionscreate)argsrr!s r"&_create_chat_completion_with_semaphoreRGenAIClient.create_chat_completion.._create_chat_completion_with_semaphoresV!\\..::AA''sVBA"B4A( A$ A( BA&B$A(&B(A?.A1 / A?;B)modelmessagesrr&)rr)r!rrrrs` r"create_chat_completion"GenAIClient.create_chat_completions?  2 &&!   (   r%cH[URR5SS9 g)Nrp)rrr_r s r"r_GenAIClient.closes$,,$$&2r%c# URRR5IShvN nUR SR $N![an[ SU35UeSnAff=f7f)Nz@Failed to get the model list from the OpenAI-compatible server: r)rmodelslistr]rydataid)r!rrbs r"rGenAIClient._get_model_namesk <<..3355F {{1~   6 RSTRUV  s7A,'A A A A, A A)A$$A))A,)rrrrr)rN) r'r(r)r*r>rrrr_rr. __classcell__)rs@r"rrs?BFC&AF "3!!r%rr)FN)rBrconcurrent.futuresrtr9typingrrrpydanticrrtyping_extensionsrutilsr utils.depsr rrr1r3rrrrrrrobjectrr&r%r"rs &&/%.),p#p#f ! A&$&E&&;|X4!&4!4!r%