Uh.AddlmZmZmZmZmZddlZddlZddlmZddl m Z m Z m Z ddl mZddlmZddlmZmZdd lmZdd lmZdd lmZmZd d lmZmZmZmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&ddl'm(Z(er ddl)m*Z*ddl+m,Z,ejZe.Z/dZ0Gdde"Z1GddeZ2GddeZ3Gdde$Z4Gdde#Z5GddeZ6Gdd e!Z7Gd!d"e Z8Gd#d$eZ9gd%Z:y)&)CallableListOptionalTupleUnionN)nn)CacheSlidingWindowCache StaticCache)AttentionMaskConverter)FlashAttentionKwargs)BaseModelOutputWithPastQuestionAnsweringModelOutput)ALL_ATTENTION_FUNCTIONS)Unpack)is_torch_flex_attn_availablelogging) LlamaAttentionLlamaDecoderLayerLlamaForCausalLMLlamaForQuestionAnsweringLlamaForSequenceClassificationLlamaForTokenClassificationLlamaMLP LlamaModelLlamaPreTrainedModelapply_rotary_pos_embeager_attention_forward) MistralConfig) BlockMask)make_flex_block_causal_maskzmistralai/Mistral-7B-v0.1ceZdZfdZxZS) MistralMLPcJt||tj|j|j d|_tj|j|j d|_tj|j |jd|_y)NFbias) super__init__rLinear hidden_sizeintermediate_size gate_projup_proj down_projselfconfig __class__s }/var/www/catia.catastroantioquia-mas.com/valormas/lib/python3.12/site-packages/transformers/models/mistral/modular_mistral.pyr+zMistralMLP.__init__*ss  4#3#3T5K5KRWXyy!1!143I3IPUV 4#9#94;K;KRWX)__name__ __module__ __qualname__r+ __classcell__r5s@r6r&r&)sYYr7r&c2eZdZdedeffd Z d dejdeejejfde ejde e de ejd e e d eeje eje eejffd ZxZS) MistralAttentionr4 layer_idxclt|t|ddxs|j|jz|_t j|j|j|j zd|_t j|j|j|j zd|_ t j|j|j|j zd|_ t j|j|j z|jd|_ y)Nhead_dimFr() r*r+getattrr-num_attention_headsrArr,q_projnum_key_value_headsk_projv_projo_projr3r4r?r5s r6r+zMistralAttention.__init__2s  D9mV=O=OSYSmSm=m ii 2 2F4N4NQUQ^Q^4^ejk ii 2 2F4N4NQUQ^Q^4^ejk ii 2 2F4N4NQUQ^Q^4^ejk ii : :T]] JFL^L^ejk r7 hidden_statesposition_embeddingsattention_maskpast_key_valuecache_positionkwargsreturnc |jdd}g|d|j}|j|j|j dd} |j |j|j dd} |j |j|j dd} |\} } t| | | | \} } |'| | |d}|j| | |j|\} } t}|jjdk7r^|jjdk(r(|jddrtjd nt |jj}||| | | |f|j"sd n |j$|j&t)|jd dd |\}}|j*g|dj-}|j/|}||fS) Nr!r)sincosrNeagersdpaoutput_attentionsFz`torch.nn.functional.scaled_dot_product_attention` does not support `output_attentions=True`. Falling back to eager attention. This warning can be removed using the argument `attn_implementation="eager"` when loading the model.sliding_window)dropoutscalingrY)shaperArDview transposerFrGrupdater?r r4_attn_implementationgetlogger warning_oncertrainingattention_dropoutr[rBreshape contiguousrH)r3rJrKrLrMrNrO input_shape hidden_shape query_states key_states value_statesrTrS cache_kwargsattention_interface attn_output attn_weightss r6forwardzMistralAttention.forward:s$))#2. 88b8$--8 {{=166|DNNqRST [[/44\BLLQPQR {{=166|DNNqRST &S#7 jRUWZ#[ j  %#&snUL'5'<'d##L '>dkk>^>^&_#$7      % $}}C$2H2HLL"4;;0@$G %  % ! \*k));;;;FFH kk+. L((r7)NN)r8r9r:r"intr+torchTensorrrr LongTensorrrrqr;r<s@r6r>r>1sl}ll+/59 0)||0)#5<<#=>0)!. 0) ! 0) !!1!12 0)-.0) u||Xell3XeELL>Q5RR S0)r7r>c(eZdZdedeffd ZxZS)MistralDecoderLayerr4r?cjt|||t|||_t ||_y)N)r4r?)r*r+r> self_attnr&mlprIs r6r+zMistralDecoderLayer.__init__ns, +)9Mf%r7)r8r9r:r"rrr+r;r<s@r6rwrwms&}&&&r7rwc eZdZy)MistralPreTrainedModelNr8r9r:r7r6r|r|tr7r|ceZdZdeffd Z ddeejdfdejdejdede f d Z e dejd e d e d ejdejd e dedefdZxZS) MistralModelr4c t||tjt |j Dcgc]}t ||c}|_ycc}wN)r*r+r ModuleListrangenum_hidden_layersrwlayersrIs r6r+zMistralModel.__init__ysD  mmEJ6KcKcEd e  3 e  esArLr# input_tensorrNpast_key_valuesrWc l|jjdk(rS|H|F|dddfjj|j dk7}|r t d|d|vr|Sy|jjdk(r't |tjr t|}|S||jnd}t |t}t |t} |jjdk(r?|s=| s;|s9tj||||jj|j ry|j"} tj$| j&} |j(d } | s|r|j+} n1t |tjr|j(dn|| zd z} |j-|| | | ||j(d|j| }|jjdk(r2|0|j.j0d vr|stj2|| }|S) Nflash_attention_2rRrzYou are attempting to perform batched generation with padding_side='right' this may lead to unexpected behaviour for Flash Attention version of Mistral. Make sure to call `tokenizer.padding_side = 'left'` before tokenizing the input. rXflex_attentionrV) inputs_embedspast_key_values_lengthrY is_trainingr!)sequence_length target_lengthdtyperN batch_sizer4r)cudaxpunpu)r4r`sumitemsize ValueError isinstancersrtr$get_seq_lengthr r r _ignore_causal_mask_sdparYrdrfinfominr\get_max_cache_shape5_prepare_4d_causal_attention_mask_with_cache_positiondevicetype_unmask_unattended)r3rLrrNrrWis_padding_rightpast_seen_tokensusing_static_cacheusing_sliding_window_cacher min_dtyperr causal_masks r6_update_causal_maskz MistralModel._update_causal_masks: ;; + +/B B)o.I#1!R%#8#<#<#>#C#C#EIZIZI\]^I_#_ #$a )c^.C%% ;; + +/? ?.%,,7!<^!L! ! @O?Z?99;`a'E%/AS%T" KK , , 6'+E%%>>*'7#{{99 MM ""KK&** &,,Q/ %);+??AM nell;$$R(%7!; PP +')#))!,;;+Q   KK , , 6*%%**.DD% 1CCKQZ[Kr7rrrrc||jdk(r|}|Stj|j} tj||f| ||j }tj ||j |jddkD} |j} t| ddrs| jgt|tr||kDrRtj ||j |jdd| jz k} | j| || z}|ddddddfj|ddd}||j}|j d|kDr |ddd|f}|j d} |ddddddd| f|ddddddfj#|j z}|d k(}|ddddddd| fj%|| |ddddddd| f<|S) a Creates a causal 4D mask of shape `(batch_size, 1, query_length, key_value_length)` from a 2D mask of shape `(batch_size, key_value_length)`, or if the input `attention_mask` is already 4D, do nothing. Args: attention_mask (`torch.Tensor`): A 2D attention mask of shape `(batch_size, key_value_length)` or a 4D attention mask of shape `(batch_size, 1, query_length, key_value_length)`. sequence_length (`int`): The sequence length being processed. target_length (`int`): The target length: when generating with static cache, the mask should be as long as the static cache, to account for the 0 padding, the part of the cache that is not filled yet. dtype (`torch.dtype`): The dtype to use for the 4D attention mask. cache_position (`torch.Tensor`): Indices depicting the position of the input sequence tokens in the sequence. batch_size (`torch.Tensor`): Batch size. config (`MistralConfig`): The model's configuration class past_key_values (`Cache`): The cache class that is being used currently to generate N) fill_valuerr)rrRr!use_sliding_windowTr)dimrsrrfullrarangerfget_text_configrBrYrr bitwise_or_expandcloner\to masked_fill)rLrrrrNrr4rrrdiagonal_attend_mask text_configsliding_attend_mask mask_length padding_masks r6rzBMistralModel._prepare_4d_causal_attention_mask_with_cache_positions%B  %.*<*<*>!*C(K@= E*..I** -0Ye\j\q\qK$)<< nF[F[#\_m_u_uA`$ !002K{$8$?KD^D^Dj"/3EF/\iJi*/,,}^MbMb*c&..r15 8R8RR+')445HI / /K%dD!Q&67>>z1bRTUK))//1 !''+m;%3A~ ~4E%FN,2226 *1aL[L+@ANSTVZ\`bcScDdDgDg&&E   ,q0 5@Aq,;,AV5W5c5c )6 Aq!\k\12r7)F)r8r9r:r"r+rrsrtr boolr staticmethodrrrrr;r<s@r6rrxs } #( TellK78TllT T  T  TlB BBB{{ B  B  BBBBr7rc eZdZy)MistralForCausalLMNr}r~r7r6rrrr7rc eZdZy)MistralForTokenClassificationNr}r~r7r6rrrr7rc eZdZy) MistralForSequenceClassificationNr}r~r7r6rr#rr7rcDeZdZdZfdZdZdZ ddeejdeejdeejdee e e ejfd eejd eejd eejd eed eedefdZxZS)MistralForQuestionAnsweringmodelcHt||t||_|`yr)r*r+rr transformerr2s r6r+z$MistralForQuestionAnswering.__init__*s"  !&)  r7c.|jjSrr embed_tokens)r3s r6get_input_embeddingsz0MistralForQuestionAnswering.get_input_embeddings/szz&&&r7c&||j_yrr)r3values r6set_input_embeddingsz0MistralForQuestionAnswering.set_input_embeddings2s"' r7 input_idsrL position_idsrrstart_positions end_positionsrWoutput_hidden_statesrPc |j||||||| } | j} |j| } | jdd\}}|j dj }|j dj }d}|||j ||||fi| }t|||| j| jS)a start_positions (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for position (index) of the start of the labelled span for computing the token classification loss. Positions are clamped to the length of the sequence (`sequence_length`). Position outside of the sequence are not taken into account for computing the loss. end_positions (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for position (index) of the end of the labelled span for computing the token classification loss. Positions are clamped to the length of the sequence (`sequence_length`). Position outside of the sequence are not taken into account for computing the loss. )rLrrrrWrr!rR)rN)loss start_logits end_logitsrJ attentions) rlast_hidden_state qa_outputssplitsqueezerg loss_functionrrJr)r3rrLrrrrrrWrrOoutputssequence_outputlogitsrrrs r6rqz#MistralForQuestionAnswering.forward5s0,0:: )%+'/!5,6, "331#)<<r<#: j#++B/::< ''+668   &=+D%4%%lJQ^ibhiD+%!!//))   r7) NNNNNNNNN)r8r9r:base_model_prefixr+rrrrsrurtrr r FloatTensorrrrqr;r<s@r6rr's '( 151537KO596:48,0/33 E,,-3 !.3 u//0 3 "%tE4E4E/F(F"GH 3   1 12 3 "%"2"233   0 013 $D>3 'tn3  &3 r7r)rrrr|rr);typingrrrrrrstorch.utils.checkpointr cache_utilsr r r modeling_attn_mask_utilsr modeling_flash_attention_utilsrmodeling_outputsrrmodeling_utilsrprocessing_utilsrutilsrrllama.modeling_llamarrrrrrrrrrr configuration_mistralr"!torch.nn.attention.flex_attentionr#integrations.flex_attentionr$ get_loggerr8rb_CHECKPOINT_FOR_DOCr&r>rwr|rrrrr__all__r~r7r6rs99 AA>BU5&:    1 !;J   H %1YY9)~9)x&+& 1 `:`F )  $?  'E A ";A H r7