~Uh8ddlmZGddeZGddeZGddeZGdd eZGd d eZGd d eZy))Optimizer2Statec6eZdZ dfd ZxZS)AdamWc <t |d|||||||| | | |  y)a Base AdamW optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. lr (`float`, defaults to 1e-3): The learning rate. betas (`tuple(float, float)`, defaults to (0.9, 0.999)): The beta values are the decay rates of the first and second-order moment of the optimizer. eps (`float`, defaults to 1e-8): The epsilon value prevents division by zero in the optimizer. weight_decay (`float`, defaults to 1e-2): The weight decay value for the optimizer. amsgrad (`bool`, defaults to `False`): Whether to use the [AMSGrad](https://hf.co/papers/1904.09237) variant of Adam that uses the maximum of past squared gradients instead. optim_bits (`int`, defaults to 32): The number of bits of the optimizer state. args (`object`, defaults to `None`): An object with additional arguments. min_8bit_size (`int`, defaults to 4096): The minimum number of elements of the parameter tensors for 8-bit optimization. percentile_clipping (`int`, defaults to 100): Adapts clipping threshold automatically by tracking the last 100 gradient norms and clipping the gradient at a certain percentile to improve stability. block_wise (`bool`, defaults to `True`): Whether to independently quantize each block of tensors to reduce outlier effects and improve stability. is_paged (`bool`, defaults to `False`): Whether the optimizer is a paged optimizer or not. adamis_pagedNsuper__init__selfparamslrbetaseps weight_decayamsgrad optim_bitsargs min_8bit_sizepercentile_clipping block_wiser __class__s j/var/www/catia.catastroantioquia-mas.com/valormas/lib/python3.12/site-packages/bitsandbytes/optim/adamw.pyr zAdamW.__init__ s<X              MbP?g?g+?:0yE>{Gz?F NdTF__name__ __module__ __qualname__r __classcell__rs@rrr0   9 9 rrc6eZdZ dfd ZxZS) AdamW8bitc <t |d|||||d|| | | |  y)a 8-bit AdamW optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. lr (`float`, defaults to 1e-3): The learning rate. betas (`tuple(float, float)`, defaults to (0.9, 0.999)): The beta values are the decay rates of the first and second-order moment of the optimizer. eps (`float`, defaults to 1e-8): The epsilon value prevents division by zero in the optimizer. weight_decay (`float`, defaults to 1e-2): The weight decay value for the optimizer. amsgrad (`bool`, defaults to `False`): Whether to use the [AMSGrad](https://hf.co/papers/1904.09237) variant of Adam that uses the maximum of past squared gradients instead. optim_bits (`int`, defaults to 32): The number of bits of the optimizer state. args (`object`, defaults to `None`): An object with additional arguments. min_8bit_size (`int`, defaults to 4096): The minimum number of elements of the parameter tensors for 8-bit optimization. percentile_clipping (`int`, defaults to 100): Adapts clipping threshold automatically by tracking the last 100 gradient norms and clipping the gradient at a certain percentile to improve stability. block_wise (`bool`, defaults to `True`): Whether to independently quantize each block of tensors to reduce outlier effects and improve stability. is_paged (`bool`, defaults to `False`): Whether the optimizer is a paged optimizer or not. rrNr r s rr zAdamW8bit.__init__Fs<X             rrr%r*s@rr-r-Er+rr-c6eZdZ dfd ZxZS) AdamW32bitc <t |d|||||d|| | | |  y)a 32-bit AdamW optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. lr (`float`, defaults to 1e-3): The learning rate. betas (`tuple(float, float)`, defaults to (0.9, 0.999)): The beta values are the decay rates of the first and second-order moment of the optimizer. eps (`float`, defaults to 1e-8): The epsilon value prevents division by zero in the optimizer. weight_decay (`float`, defaults to 1e-2): The weight decay value for the optimizer. amsgrad (`bool`, defaults to `False`): Whether to use the [AMSGrad](https://hf.co/papers/1904.09237) variant of Adam that uses the maximum of past squared gradients instead. optim_bits (`int`, defaults to 32): The number of bits of the optimizer state. args (`object`, defaults to `None`): An object with additional arguments. min_8bit_size (`int`, defaults to 4096): The minimum number of elements of the parameter tensors for 8-bit optimization. percentile_clipping (`int`, defaults to 100): Adapts clipping threshold automatically by tracking the last 100 gradient norms and clipping the gradient at a certain percentile to improve stability. block_wise (`bool`, defaults to `True`): Whether to independently quantize each block of tensors to reduce outlier effects and improve stability. is_paged (`bool`, defaults to `False`): Whether the optimizer is a paged optimizer or not. rr"rNr r s rr zAdamW32bit.__init__s<X              rrr%r*s@rr1r1r+rr1c4eZdZ dfd ZxZS) PagedAdamWc <t |d|||||||| | | d y)a Paged AdamW optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. lr (`float`, defaults to 1e-3): The learning rate. betas (`tuple(float, float)`, defaults to (0.9, 0.999)): The beta values are the decay rates of the first and second-order moment of the optimizer. eps (`float`, defaults to 1e-8): The epsilon value prevents division by zero in the optimizer. weight_decay (`float`, defaults to 1e-2): The weight decay value for the optimizer. amsgrad (`bool`, defaults to `False`): Whether to use the [AMSGrad](https://hf.co/papers/1904.09237) variant of Adam that uses the maximum of past squared gradients instead. optim_bits (`int`, defaults to 32): The number of bits of the optimizer state. args (`object`, defaults to `None`): An object with additional arguments. min_8bit_size (`int`, defaults to 4096): The minimum number of elements of the parameter tensors for 8-bit optimization. percentile_clipping (`int`, defaults to 100): Adapts clipping threshold automatically by tracking the last 100 gradient norms and clipping the gradient at a certain percentile to improve stability. block_wise (`bool`, defaults to `True`): Whether to independently quantize each block of tensors to reduce outlier effects and improve stability. is_paged (`bool`, defaults to `False`): Whether the optimizer is a paged optimizer or not. rTrNr rrrrrrrrrrrrrs rr zPagedAdamW.__init__s<V              r rrr r!Fr"Nr#r$Tr%r*s@rr4r4-   8 8 rr4c4eZdZ dfd ZxZS)PagedAdamW8bitc <t |d|||||d|| | | d y)a Paged 8-bit AdamW optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. lr (`float`, defaults to 1e-3): The learning rate. betas (`tuple(float, float)`, defaults to (0.9, 0.999)): The beta values are the decay rates of the first and second-order moment of the optimizer. eps (`float`, defaults to 1e-8): The epsilon value prevents division by zero in the optimizer. weight_decay (`float`, defaults to 1e-2): The weight decay value for the optimizer. amsgrad (`bool`, defaults to `False`): Whether to use the [AMSGrad](https://hf.co/papers/1904.09237) variant of Adam that uses the maximum of past squared gradients instead. optim_bits (`int`, defaults to 32): The number of bits of the optimizer state. args (`object`, defaults to `None`): An object with additional arguments. min_8bit_size (`int`, defaults to 4096): The minimum number of elements of the parameter tensors for 8-bit optimization. percentile_clipping (`int`, defaults to 100): Adapts clipping threshold automatically by tracking the last 100 gradient norms and clipping the gradient at a certain percentile to improve stability. block_wise (`bool`, defaults to `True`): Whether to independently quantize each block of tensors to reduce outlier effects and improve stability. is_paged (`bool`, defaults to `False`): Whether the optimizer is a paged optimizer or not. rr/TrNr r6s rr zPagedAdamW8bit.__init__s<V             rr7r%r*s@rr:r:r8rr:c4eZdZ dfd ZxZS)PagedAdamW32bitc <t |d|||||d|| | | d y)a Paged 32-bit AdamW optimizer. Arguments: params (`torch.tensor`): The input parameters to optimize. lr (`float`, defaults to 1e-3): The learning rate. betas (`tuple(float, float)`, defaults to (0.9, 0.999)): The beta values are the decay rates of the first and second-order moment of the optimizer. eps (`float`, defaults to 1e-8): The epsilon value prevents division by zero in the optimizer. weight_decay (`float`, defaults to 1e-2): The weight decay value for the optimizer. amsgrad (`bool`, defaults to `False`): Whether to use the [AMSGrad](https://hf.co/papers/1904.09237) variant of Adam that uses the maximum of past squared gradients instead. optim_bits (`int`, defaults to 32): The number of bits of the optimizer state. args (`object`, defaults to `None`): An object with additional arguments. min_8bit_size (`int`, defaults to 4096): The minimum number of elements of the parameter tensors for 8-bit optimization. percentile_clipping (`int`, defaults to 100): Adapts clipping threshold automatically by tracking the last 100 gradient norms and clipping the gradient at a certain percentile to improve stability. block_wise (`bool`, defaults to `True`): Whether to independently quantize each block of tensors to reduce outlier effects and improve stability. is_paged (`bool`, defaults to `False`): Whether the optimizer is a paged optimizer or not. rr"TrNr r6s rr zPagedAdamW32bit.__init__8s<V              rr7r%r*s@rr=r=7r8rr=N)bitsandbytes.optim.optimizerrrr-r1r4r:r=rrrAsW 9: O: z: : z: : z9 9 x9 _9 x9 o9 r