͑ixSSKJr SSKrSSKrSSKJr SSKJrJrJ r J r SSK r SSK Jr SSKrSSKJr SSKJr SS KJrJr \(aSS KJrJr SS KJr \"S 5r\ "S 5r/SQr"SS\5rSSjrS SjrS!Sjr "SS5r!\RD4S"Sjjr#S#Sjr$S$Sjr%S%Sjr&S%Sjr'\RPS&Sj5r)S'S(Sjjr*S)Sjr+S%Sjr,g)*) annotationsN)Enum) TYPE_CHECKINGAnyCallableTypeVar) ParamSpec)_C_ops)core) LayerHelperin_dynamic_or_pir_mode) GeneratorSequence)Tensor_InputT_RetT) DebugModeTensorCheckerConfigcheck_numerics enable_operator_stats_collection!disable_operator_stats_collectioncollect_operator_statsenable_tensor_checkerdisable_tensor_checkercompare_accuracycheck_layer_numericsc(\rSrSrSrSrSrSrSrSr g) r8a The DebugMode is a feature that helps to present the state of the TensorCheckerConfig. Each DebugMode has a specific meaning, which is explained below: - DebugMode.CHECK_NAN_INF_AND_ABORT: This mode prints or saves information about Tensors that contain NaN/Inf and interrupts the program. - DebugMode.CHECK_NAN_INF: This mode prints or saves critical information about Tensors that contain NaN/Inf but allows the program to continue running. - DebugMode.CHECK_ALL_FOR_OVERFLOW: This mode checks the output of the FP32 operator and prints or saves information about key Tensors that exceed the FP16 representation range, such as overflow or underflow. - DebugMode.CHECK_ALL: This mode prints or saves output Tensor key information for all operators. rr N) __name__ __module__ __qualname____firstlineno____doc__CHECK_NAN_INF_AND_ABORT CHECK_NAN_INFCHECK_ALL_FOR_OVERFLOW CHECK_ALL__static_attributes__r"T/var/www/html/banglarbhumi/venv/lib/python3.13/site-packages/paddle/amp/debugging.pyrr8s  MIr-rc^SU4SjjnU$)a This decorator is used to check the numerical values of the layer's input and output data. Args: func (callable): The function to be decorated. Returns: callable: The decorated function. Raises: None. Example: .. code-block:: python >>> import paddle >>> class MyLayer(paddle.nn.Layer): ... def __init__(self, dtype): ... super().__init__() ... self._w = self.create_parameter([2, 3], dtype=dtype) ... self._b = self.create_parameter([2, 3], dtype=dtype) ... @paddle.amp.debugging.check_layer_numerics ... def forward(self, x): ... # return 1/x * self._w + self._b open it you will see the error log ... return x @ self._w + self._b ... >>> dtype = 'float32' >>> x = paddle.rand([10, 2, 2], dtype=dtype) # type: ignore[arg-type] >>> model = MyLayer(dtype) >>> x[0] = float(0) >>> loss = model(x) >>> adam = paddle.optimizer.Adam(parameters=model.parameters()) >>> loss.backward() >>> adam.step() >>> # error log >>> # [PRECISION] [ERROR] in [device=gpu:0, op=divide, tensor=, dtype=fp32], numel=40, num_nan=0, num_inf=4, num_zero=0, max=inf, min=1.048930e+00, mean=inf >>> # Traceback (most recent call last): >>> # File "tmp.py", line 16, in >>> # loss = model(x) >>> # File "/paddle/nn/layer/layers.py", line 1254, in __call__ >>> # return self.forward(*inputs, **kwargs) >>> # File "/paddle/amp/debugging.py", line 116, in wrapper >>> # out_data = func(self, *modified_args, **kwargs) >>> # File "test.py", line 10, in forward >>> # return 1/x * self._w+ self._b >>> # RuntimeError: (PreconditionNotMet) There are NAN or INF (num_nan=0, num_inf=4, num_zero=0) in [device=gpu:0, op=divide, tensor=, dtype=fp32]. c(>U(aUSn[U[R5(d [S5eSUl[ U5n[ R"US5US'T"U/UQ70UD6n[ R"US5nU$[S5e)Nrz)First input of this layer must be tensor.Fr zNo elements found in args.) isinstancepaddler RuntimeError stop_gradientlistr enable_check_model_nan_infdisable_check_model_nan_inf)selfargskwargs start_data modified_argsout_dataoutfuncs r.wrapper%check_layer_numerics..wrappers aJj&--88"#NOO',J $ JM%@@QOM! D;=;F;H44XqACJ;< &set_checked_op_list..s$HuUz%checked_op_list must be list or tuple) r1r5tuplejoinr2baser set_checked_op_list ValueError)checked_op_list check_op_lists r.rPrPsV" oe} 5 5HH$H$HHM KK   0 0 ?DE E #r-cUbh[U[[45(aBSRSU55n[R R RU5 g[S5eg)NrDc3$# UHov M g7frFr"rGs r.rJ&set_skipped_op_list..s#GeErLz%skipped_op_list must be list or tuple) r1r5rMrNr2rOr set_skipped_op_listrQ)skipped_op_list skip_op_lists r.rWrWsV" oe} 5 588#G#GGL KK   0 0 >DE E #r-c\rSrSr%SrSrS\S'S\S'S\S 'S \S 'S \S 'S \S'S\S'S\S'S\S'S\S'S\S'S\S'\RSSSSS4S Sjjr S!Sjr S"Sjr S#Sjr S$Sjr S$SjrSrg)%ra5 The purpose of this class is to collect the configuration for checking NaN and Inf values in the tensors of a module or operator. It takes the following arguments: Args: enable(bool): Indicating whether to enable the detection of NaN and Inf values in tensors. The default value is False, which means that these tools will not be used. debug_mode(DebugMode, optional): A parameter that determines the type of debugging to be used. Default is DebugMode.CHECK_NAN_INF_AND_ABORT. output_dir(string|None, optional): The path to store collected data. If this parameter is set to None, the data will be printed to the terminal. Default is None. checked_op_list(list|tuple|None, optional): Specifies a list of operators that need to be checked during program execution, for example, checked_op_list=['elementwise_add', 'conv2d'], indicating that the output results of elementwise_add and conv2d should be checked for nan/inf during program execution. Default is None. skipped_op_list(list|tuple|None, optional): Specifies a list of operators that do not need to be checked during program execution, for example, skipped_op_list=['elementwise_add', 'conv2d'], indicating that the output results of elementwise_add and conv2d should not be checked for nan/inf during program execution. None is None. debug_step(list|tuple|None, optional): A list or tuple used primarily for nan/inf checking during model training. For example, debug_step=[1,5] indicates that nan/inf checking should only be performed on model training iterations 1 to 5. Default is None. stack_height_limit(int, optional): An integer value specifying the maximum depth of the call stack. This feature supports printing the call stack at the error location. Currently, only enabling or disabling call stack printing is supported. If you want to print the corresponding C++ call stack when NaN is detected in GPU Kernel, set stack_height_limit to 1, otherwise set it to 0. Default is 1. Examples: .. code-block:: python >>> import paddle >>> checker_config = paddle.amp.debugging.TensorCheckerConfig(enable=True, debug_mode=paddle.amp.debugging.DebugMode.CHECK_NAN_INF) >>> paddle.amp.debugging.enable_tensor_checker(checker_config) >>> x = paddle.to_tensor([1, 0, 3], place=paddle.CPUPlace(), dtype='float32', stop_gradient=False) >>> y = paddle.to_tensor([0.2, 0, 0.5], place=paddle.CPUPlace(), dtype='float32') >>> res = paddle.pow(x, y) >>> paddle.autograd.backward(res, retain_graph=True) >>> paddle.amp.debugging.disable_tensor_checker() >>> # [PRECISION] [ERROR] in [device=cpu, op=elementwise_pow_grad, tensor=, dtype=fp32], numel=3, num_nan=1, num_inf=0, num_zero=0, max=2.886751e-01, min=2.000000e-01, mean=-nan >>> # when DebugMode.CHECK_NAN_INF_AND_ABORT and stack_height_limit = 1 >>> # Traceback (most recent call last): >>> # res = paddle.pow(x, y) >>> # File "/usr/local/lib/python3.8/dist-packages/paddle/tensor/math.py", line 447, in pow >>> # return _C_ops.elementwise_pow(x, y) rintcurrent_step_idboolenabler debug_mode str | None output_dirSequence[str] | NonerRrXSequence[int] | None debug_stepstack_height_limitz int | None start_stepend_stepseed initial_seedNr cXlX lX0lX@lXPlX`lXplSUlSUlSUl SUl Ub[U[[45(ap[UR 5S:Xa UR SUR S:deUR uUlUl[URS5UlO [!S5e["R$"5(aO['["R("55H,n["R*"U5R5Ul M. Oh["R,"5(aN['["R."55H,n["R0"U5R5Ul M. ["R2"5R5Ul URR4[6R8;a%[!SUR[6R85e[;UR5 [=UR5 UR(aUR?UR5 gg)N{r r rz debug_step must be list or tuplezdebug_mode in DebugMode) r_r`rbrRrXrerfrgrhrirjr1rMr5lenmaxrQr is_compiled_with_cudarangeget_cuda_device_countdefault_cuda_generatoris_compiled_with_xpuget_xpu_device_countdefault_xpu_generatordefault_cpu_generatornamer __members__rPrW _set_seed) r8r_r`rbrRrXrerfis r.__init__TensorCheckerConfig.__init__s $$..$"4    !*udm44(A-*T__Q-??@26."%dooq"9 !CDD  % % ' '45578$($?$?%,.!9 & & ( (44467$($>$>q$A$N$N$P!8!668EEG ??  y'<'< <)%%  D001D001 ;; NN4;; ' r-cURUR:waURUlUR[R"[R5R :dURS:a[ S5 SUl[R"UR5 [RRUR5 [R"UR5 [ SUR5 [R"5(a%[R"SU05 [ SU5 gg)Nrz.[Warning: Seed must be between 0 and 2**32 - 1rlz(AMP Debugging TensorCheckerConfig: seed FLAGS_cudnn_deterministicz@AMP Debugging TensorCheckerConfig: FLAGS_cudnn_deterministic is ) rjrinpiinfouint32rnprintr2randomr ro set_flags)r8flags r.ryTensorCheckerConfig._set_seed(s    )))DI 99rxx *.. .$))a- B CDI  DII tyy! DII 8$))D  % % ' '   94@ A R  (r-c[R"SU05 U(a[R"SURR05 URb3[R R RUR5 [UR[5(a4[R R RUR5 g[S5eg)NFLAGS_check_nan_infFLAGS_check_nan_inf_levelzstack_height_limit must be int) r2rr`rIrbrOr set_nan_inf_debug_pathr1rfr\set_nan_inf_stack_limitrQ)r8 check_flags r._set_envTensorCheckerConfig._set_env@s/<=    ,doo.C.CD  *   77H$11C99   88++!!ABB! r-cUR(aqURbcURbVUR[R:d[RUR:ag[=RS- slgg)NFr T)r_rgrhrr]r8s r.update_and_check_step_id,TensorCheckerConfig.update_and_check_step_idTs[ ;;*t}}/HOO&9&I&II*::dmmK '771<7r-c^UR(aURUR5 ggrF)r_rrs r.start_check_nan_inf'TensorCheckerConfig.start_check_nan_infas ;; MM$++ & r-c&URS5 g)NF)rrs r.stop_check_nan_inf&TensorCheckerConfig.stop_check_nan_infes er-) rRr`rer_rhrjrbrirXrfrg)r_r^r`rrbrarRrcrXrcrerdrfr\)rr\rBNone)rr\rBr)rBr^rBr)r#r$r%r&r'r]__annotations__rr(r{ryrrrrr,r"r-r.rrs)XOS L))))$$ I !* A A!%0404+/"#>(>(>( >( . >( . >()>( >(@0C( 'r-rc ,SnSn[5(a%[R"UUUURUU5$[ S 0[ 5D6nUR SS9nUR SS9nURSSU0UUURUUS.U/U/S .S 9 Xx4$) a This function is used to debugging a tensor, finding the number of NaNs, Infs and zeros in the tensor. Args: tensor(Tensor): The target tensor to check. op_type(str): The OP or API name which produce the target tensor. var_name(str): The name of target tensor. debug_mode(paddle.amp.debugging.DebugMode, optional): The mode of debugging to be used. Default is DebugMode.CHECK_NAN_INF_AND_ABORT. Returns: (Tensor, Tensor): A tuple of tensors containing - **stats** (Tensor): Returns the number of NaNs, Infs and zeros of input tensor. The shape is [3] and dtype is int64. - **values** (Tensor): Returns the maximum, minimum and mean value of input tensor. The shape is [3] and dtype is float. Examples: .. code-block:: python >>> import paddle >>> checker_config = paddle.amp.debugging.TensorCheckerConfig( ... enable=True, debug_mode=paddle.amp.debugging.DebugMode.CHECK_NAN_INF) >>> x = paddle.to_tensor([1, 0, 3], place=paddle.CPUPlace(), dtype='float32') >>> y = paddle.to_tensor([0.2, 0, 0.5], place=paddle.CPUPlace(), dtype='float32') >>> res = paddle.pow(x, y) >>> paddle.amp.debugging.check_numerics(res, "pow", "res") rint64)dtypefloattensor)op_typevar_namecheck_nan_inf_levelrfrb)statsvalues)typeinputsattrsoutputs)r)rr rrIr locals"create_variable_for_type_inference append_op) rrrr`rfrbhelperrrs r.rrisHJ$$            6VX 6F  5 5G 5 DE  6 6W 6 EF   f  #-#3#3"4$  !'fX6  =r-c:[R"S/5nUS$)NFLAGS_low_precision_op_list)r2 get_flags)flagss r._get_operator_stats_flagrs"   ;< =E . //r-c[SRS55 Sn[SRS5SSRS5SSRS 5SSRS 5SS RS 55 Ub[U[5(a[ U5HnXn[U[ 5(aUnOB[U[ 5(aURS5nO[SUS[U5S35e[SUSSUSSSUSSSUSSSUSS3 5 US- nM [SRS[ U5-S-55 g )a| Parse and print the stats of operators, mainly including the calls of dtypes such as different fp32, fp16, bf16 and others. Args: op_count_dict(dict): a dict to record the number of calls for different operator and dtype. An example is {'conv2d': '1,0,0,0', 'elementwise_add': '1,0,0,0'} or {'conv2d': [1, 0, 0, 0], 'elementwise_add': [1, 0, 0, 0]}. z <{:-^120}>z op list rz<{:-^40}z Op Name |z{:-^17}z FP16 Calls z BF16 Calls z FP32 Callsz{:-^17}>z Other Calls NrDzInput z/ is expected to be a list of str, but received .z z<40z| z<17r r r!z <{:-^120}> z op count:  ) rformatr1dictsortedr5strsplitrQr) op_count_dict total_opsrrIcalleds r._print_operator_statsrss ,  k *+I +& ( ( ' /*  Z t%D%Dm,G!*E%&&E3''S) UG#RSWX]S^R__`a WSMVAYsO3vayoSPQSVWZ[abc[dehZij  NI- .   I > D EFr-c[RRR5 [R"SS05 g)a Enable to collect the number of operators for different data types. The statistical data are categorized according to four data types, namely float32, float16, bfloat16 and others. This function is used in pair with the corresponding disable function. Examples: .. code-block:: python >>> # doctest: +REQUIRES(env:GPU) >>> import paddle >>> paddle.device.set_device('gpu') >>> conv = paddle.nn.Conv2D(3, 2, 3) >>> x = paddle.rand([10, 3, 32, 32]) >>> paddle.amp.debugging.enable_operator_stats_collection() >>> # AMP list including cast, conv2d, elementwise_add, reshape >>> with paddle.amp.auto_cast(enable=True, level='O2'): ... out = conv(x) >>> # Print to the standard output. >>> paddle.amp.debugging.disable_operator_stats_collection() >>> # <------------------------------------------------------- op list --------------------------------------------------------> >>> # <--------------- Op Name ---------------- | -- FP16 Calls --- | -- BF16 Calls --- | --- FP32 Calls--- | -- Other Calls --> >>> # cast | 1 | 0 | 2 | 0 >>> # conv2d | 1 | 0 | 0 | 0 >>> # elementwise_add | 0 | 0 | 1 | 0 >>> # reshape | 0 | 0 | 1 | 0 >>> # <----------------------------------------------------- op count: 4 ------------------------------------------------------> rr N)r2rOr clear_low_precision_op_listrr"r-r.rrs/D KK002 3Q78r-c[5(dg[RRR 5n[ U5 [R "SS05 g)a Disable the collection the number of operators for different data types. This function is used in pair with the corresponding enable function. The statistical data are categorized according to four data types, namely float32, float16, bfloat16 and others, and will be printed after the function call. Examples: .. code-block:: python >>> import paddle >>> conv = paddle.nn.Conv2D(3, 2, 3) >>> x = paddle.rand([10, 3, 32, 32]) >>> paddle.amp.debugging.enable_operator_stats_collection() >>> # AMP list including cast, conv2d, elementwise_add, reshape >>> with paddle.amp.auto_cast(enable=True, level='O2'): ... out = conv(x) >>> # Print to the standard output. >>> paddle.amp.debugging.disable_operator_stats_collection() >>> # <------------------------------------------------------- op list --------------------------------------------------------> >>> # <--------------- Op Name ---------------- | -- FP16 Calls --- | -- BF16 Calls --- | --- FP32 Calls--- | -- Other Calls --> >>> # cast | 1 | 0 | 2 | 0 >>> # conv2d | 1 | 0 | 0 | 0 >>> # elementwise_add | 0 | 0 | 1 | 0 >>> # reshape | 0 | 0 | 1 | 0 >>> # <----------------------------------------------------- op count: 4 ------------------------------------------------------> Nrr)rr2rOr get_low_precision_op_listrr)rs r.rrsE@ $ % %KK$$>>@M-( 3Q78r-c#<# [5 Sv [5 g7f)a9 The context switcher to enable to collect the number of operators for different data types. The statistical data are categorized according to four data types, namely float32, float16, bfloat16 and others, and will be printed when exiting the context. Examples: .. code-block:: python >>> import paddle >>> conv = paddle.nn.Conv2D(3, 2, 3) >>> x = paddle.rand([10, 3, 32, 32]) >>> with paddle.amp.debugging.collect_operator_stats(): ... # AMP list including cast, conv2d, elementwise_add, reshape ... with paddle.amp.auto_cast(enable=True, level='O2'): ... out = conv(x) >>> # Print to the standard output. >>> # <------------------------------------------------------- op list --------------------------------------------------------> >>> # <--------------- Op Name ---------------- | -- FP16 Calls --- | -- BF16 Calls --- | --- FP32 Calls--- | -- Other Calls --> >>> # cast | 1 | 0 | 2 | 0 >>> # conv2d | 1 | 0 | 0 | 0 >>> # elementwise_add | 0 | 0 | 1 | 0 >>> # reshape | 0 | 0 | 1 | 0 >>> # <----------------------------------------------------- op count: 4 ------------------------------------------------------> N)rrr"r-r.rr.s>%& %'scrUSLdS5e[RRRUUUUSS9 g)a This is a precision comparison tool that can be used to compare log data of float16 and float32. Args: dump_path(str): The path of the running log, such as the log for execution using the float32 data type. another_dump_path(str): the path of another running log ,such as the log for execution using the float16 data type. output_filename(str): the excel file name of compare output. loss_scale(float, optional): the loss_scale during the training phase. Default is 1. dump_all_tensors(bool, optional): dump all tensor, It is currently not support. Default is False. Examples: .. code-block:: python >>> import paddle >>> from paddle.base import core >>> try: ... import xlsxwriter as xlw ... except ImportError: ... import subprocess ... subprocess.check_call( ... ['python', '-m', 'pip', 'install', 'xlsxwriter==3.0.9'] ... ) ... import xlsxwriter as xlw ... if core.is_compiled_with_cuda(): ... paddle.set_flags( ... {"FLAGS_check_nan_inf": 1, "FLAGS_check_nan_inf_level": 3} ... ) ... path = "workerlog_log_dir" ... paddle.base.core.set_nan_inf_debug_path(path) ... x = paddle.to_tensor( ... [2, 3, 4, 0], dtype="float32" ... ) ... y = paddle.to_tensor( ... [1, 5, 2, 0], dtype="float32" ... ) ... z1 = x + y ... out_excel = "compare_accuracy_out_excel.csv" ... paddle.amp.debugging.compare_accuracy( ... path, path, out_excel, loss_scale=1, dump_all_tensors=False ... ) FzIt is currently not supported.)dump_all_tensorsN)r2ampaccuracy_comparer) dump_pathanother_dump_pathoutput_filename loss_scalers r.rrRsEb u $F&FF $ JJ00 1r-cpUR5(aUR5 gUR5 g)aZ The enable_tensor_checker(checker_config) function enables model-level accuracy checking and is used in combination with disables_tensor_checker() to achieve model-level precision checking by checking the output Tensors of all operators within the specified range. Args: checker_config(TensorCheckerConfig): Checker_config is to collect the configuration for checking NaN and Inf values in the tensors of a module or operator. Note: If disable_tensor_checker() is called before backward(), the gradient operator will not be checked. If disable_tensor_checker() is called before optimizer.step(), the optimizer and other weight update related operators will not be checked. Examples: .. code-block:: python >>> import paddle >>> checker_config = paddle.amp.debugging.TensorCheckerConfig(enable=True, debug_mode=paddle.amp.debugging.DebugMode.CHECK_NAN_INF) >>> paddle.amp.debugging.enable_tensor_checker(checker_config) >>> x = paddle.to_tensor([1, 0, 3], place=paddle.CPUPlace(), dtype='float32', stop_gradient=False) >>> y = paddle.to_tensor([0.2, 0, 0.5], place=paddle.CPUPlace(), dtype='float32') >>> res = paddle.pow(x, y) >>> paddle.autograd.backward(res, retain_graph=True) >>> paddle.amp.debugging.disable_tensor_checker() >>> #[PRECISION] [ERROR] in [device=cpu, op=elementwise_pow_grad, tensor=, dtype=fp32], numel=3, num_nan=1, num_inf=0, num_zero=0, max=2.886751e-01, min=2.000000e-01, mean=-nan >>> # when DebugMode.CHECK_NAN_INF_AND_ABORT and stack_height_limit = 1 >>> # Traceback (most recent call last): >>> # File "tp.py", line 8, in >>> # res = paddle.pow(x, y) >>> # File "/usr/local/lib/python3.8/dist-packages/paddle/tensor/math.py", line 447, in pow >>> # return _C_ops.elementwise_pow(x, y) N)rrr)checker_configs r.rrs,F..00**,))+r-c4[R"SS05 g)ah disable_tensor_checker() is used to disable accuracy checking, and is used together with enable_tensor_checker(config) to achieve model-level precision checking by checking the output Tensors of all operators within the specified range. Note: If disable_tensor_checker() is called before backward(), the gradient operator will not be checked; If disable_tensor_checker() is called before optimizer.step(), the optimizer and other weight update related operators will not be checked. Examples: .. code-block:: python >>> import paddle >>> checker_config = paddle.amp.debugging.TensorCheckerConfig(enable=True, debug_mode=paddle.amp.debugging.DebugMode.CHECK_NAN_INF) >>> paddle.amp.debugging.enable_tensor_checker(checker_config) >>> x = paddle.to_tensor([1, 0, 3], place=paddle.CPUPlace(), dtype='float32', stop_gradient=False) >>> y = paddle.to_tensor([0.2, 0, 0.5], place=paddle.CPUPlace(), dtype='float32') >>> res = paddle.pow(x, y) >>> paddle.autograd.backward(res, retain_graph=True) >>> paddle.amp.debugging.disable_tensor_checker() >>> # [PRECISION] [ERROR] in [device=cpu, op=elementwise_pow_grad, tensor=, dtype=fp32], numel=3, num_nan=1, num_inf=0, num_zero=0, max=2.886751e-01, min=2.000000e-01, mean=-nan >>> # when DebugMode.CHECK_NAN_INF_AND_ABORT and stack_height_limit = 1 >>> # Traceback (most recent call last): >>> # res = paddle.pow(x, y) >>> # File "/usr/local/lib/python3.8/dist-packages/paddle/tensor/math.py", line 447, in pow >>> # return _C_ops.elementwise_pow(x, y) rrN)r2rr"r-r.rrs> +Q/0r-)r?Callable[_InputT, _RetT]rBr)rRrcrBr)rXrcrBr) rrrrrrr`rrBztuple[Tensor, Tensor])rBr)rzdict[str, str | list[int]]rBrr)rBzGenerator[None, None, None])r F) rrrrrrrrrr^rBr)rrrBr)- __future__r contextlibrenumrtypingrrrrnumpyrtyping_extensionsr r2r paddle.baser frameworkr rcollections.abcrrrrr__all__rrrPrWrr(rrrrrcontextmanagerrrrrr"r-r.rs\# ' ;3 I  ,H "HHVFFyy@&== D D DD D  DN0 (GV#9L%9P  ( (N" 8888 8  8  8v&,R1r-