UhGddlmZmZmZmZmZddlZddlmZddlm Z ddl m Z m Z m Z mZddlmZddlmZdd lmZdd lmZdd lmZdd lmZmZmZmZmZdd lm Z m!Z!ddl"m#Z#m$Z$ddl%m&Z&ddl'm(Z(m)Z)m*Z*m+Z+m,Z,ddl-m.Z.e+r ddl/m0Z0ddl1m2Z2e,jfe4Z5GddejlZ7dZ8d?dZ9dejtde;dejtfdZ< d@dejldejtdejtd ejtd!eejtd"e=d#e=fd$Z>Gd%d&ejlZ?ed'Gd(d)ejlZ@Gd*d+eZAe)Gd,d-e$ZBGd.d/ejlZCe)Gd0d1eBZDGd2d3ee(ZEe)Gd4d5eBeZFe)Gd6d7eBZGe)d89Gd:d;eBZHe)Gd<d=eBZIgd>ZJy)A)CallableListOptionalTupleUnionN)nn)ACT2FN)Cache DynamicCacheSlidingWindowCache StaticCache)GenerationMixin)use_kernel_forward_from_hub)AttentionMaskConverter)FlashAttentionKwargs)GradientCheckpointingLayer)BaseModelOutputWithPastCausalLMOutputWithPastQuestionAnsweringModelOutput SequenceClassifierOutputWithPastTokenClassifierOutput)ROPE_INIT_FUNCTIONSdynamic_rope_update)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack) LossKwargsauto_docstringcan_return_tupleis_torch_flex_attn_availablelogging) MistralConfig) BlockMask)make_flex_block_causal_maskc$eZdZfdZdZxZS) MistralMLPct|||_|j|_|j|_t j |j|jd|_t j |j|jd|_t j |j|jd|_ t|j|_ yNFbias) super__init__config hidden_sizeintermediate_sizerLinear gate_projup_proj down_projr hidden_actact_fnselfr/ __class__s ~/var/www/catia.catastroantioquia-mas.com/valormas/lib/python3.12/site-packages/transformers/models/mistral/modeling_mistral.pyr.zMistralMLP.__init__+s  !--!'!9!94#3#3T5K5KRWXyy!1!143I3IPUV 4#9#94;K;KRWXV../ c|j|j|j||j|z}|SN)r5r7r3r4)r9xr5s r;forwardzMistralMLP.forward5s6NN4;;t~~a/@#ADLLQRO#ST r<)__name__ __module__ __qualname__r.r@ __classcell__r:s@r;r(r(*s 0r<r(c|dd|jddzf}|d|jddzdf}tj| |fdS)z*Rotates half the hidden dims of the input..Ndim)shapetorchcat)r?x1x2s r; rotate_halfrP:sZ 3"!''"+"" " #B 3 q " " #B 99rc2YB ''r<c|j|}|j|}||zt||zz}||zt||zz}||fS)aApplies Rotary Position Embedding to the query and key tensors. Args: q (`torch.Tensor`): The query tensor. k (`torch.Tensor`): The key tensor. cos (`torch.Tensor`): The cosine part of the rotary embedding. sin (`torch.Tensor`): The sine part of the rotary embedding. position_ids (`torch.Tensor`, *optional*): Deprecated and unused. unsqueeze_dim (`int`, *optional*, defaults to 1): The 'unsqueeze_dim' argument specifies the dimension along which to unsqueeze cos[position_ids] and sin[position_ids] so that they can be properly broadcasted to the dimensions of q and k. For example, note that cos[position_ids] and sin[position_ids] have the shape [batch_size, seq_len, head_dim]. Then, if q and k have the shape [batch_size, heads, seq_len, head_dim], then setting unsqueeze_dim=1 makes cos[position_ids] and sin[position_ids] broadcastable to the shapes of q and k. Similarly, if q and k have the shape [batch_size, seq_len, heads, head_dim], then set unsqueeze_dim=2. Returns: `tuple(torch.Tensor)` comprising of the query and key tensors rotated using the Rotary Position Embedding. ) unsqueezerP)qkcossin position_ids unsqueeze_dimq_embedk_embeds r;apply_rotary_pos_embr[AsY( -- &C -- &C3w;q>C/0G3w;q>C/0G G r< hidden_statesn_repreturnc|j\}}}}|dk(r|S|dddddddddfj|||||}|j|||z||S)z This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch, num_key_value_heads, seqlen, head_dim) to (batch, num_attention_heads, seqlen, head_dim) r#N)rKexpandreshape)r\r]batchnum_key_value_headsslenhead_dims r; repeat_kvrf\so 2?1D1D.E h z!!Qa"23::5BUW\^bdlmM  (;e(CT8 TTr<modulequerykeyvalueattention_maskscalingdropoutc Tt||j}t||j} tj||j dd|z} |#|ddddddd|j df} | | z} t jj| dtjj|j} t jj| ||j} tj| | } | j ddj} | | fS)NrHr rG)rJdtype)ptrainingr#)rfnum_key_value_groupsrLmatmul transposerKr functionalsoftmaxfloat32torprmrr contiguous) rgrhrirjrkrlrmkwargs key_states value_states attn_weights causal_mask attn_outputs r;eager_attention_forwardrhs3 ; ;$>&B\B\$\!}}d* !'!9!9ii 2 2F4N4NQUQ^Q^4^ejk ii 2 2F4N4NQUQ^Q^4^ejk ii 2 2F4N4NQUQ^Q^4^ejk ii : :T]] JFL^L^ejk r<r\position_embeddingsrkpast_key_valuecache_positionr{r^c |jdd}g|d|j}|j|j|j dd} |j |j|j dd} |j |j|j dd} |\} } t| | | | \} } |'| | |d}|j| | |j|\} } t}|jjdk7r^|jjdk(r(|jddrtjd nt |jj}||| | | |f|j"sd n |j$|j&t)|jd dd |\}}|j*g|dj-}|j/|}||fS) NrGr#rH)rVrUreagersdpaoutput_attentionsFz`torch.nn.functional.scaled_dot_product_attention` does not support `output_attentions=True`. Falling back to eager attention. This warning can be removed using the argument `attn_implementation="eager"` when loading the model.sliding_window)rmrlr)rKrerviewrurrr[updaterrr/_attn_implementationgetlogger warning_oncerrrrrlrrarzr)r9r\rrkrrr{ input_shape hidden_shape query_statesr|r}rUrV cache_kwargsattention_interfacerr~s r;r@zMistralAttention.forwards$))#2. 88b8$--8 {{=166|DNNqRST [[/44\BLLQPQR {{=166|DNNqRST &S#7 jRUWZ#[ j  %#&snUL'5'<'d##L '>dkk>^>^&_#$7      % $}}C$2H2HLL"4;;0@$G %  % ! \*k));;;;FFH kk+. L((r<)NN)rArBrC__doc__r$intr.rLTensorrrr LongTensorrrr@rDrEs@r;rrsG l} l l&+/59 0)||0)#5<<#=>0)!. 0) ! 0) !!1!12 0)-.0) u||Xell3XeELL>Q5RR S0)r<rRMSNormc,eZdZdfd ZdZdZxZS)MistralRMSNormct|tjt j ||_||_y)z= MistralRMSNorm is equivalent to T5LayerNorm N)r-r.r ParameterrLonesweightvariance_epsilon)r9r0epsr:s r;r.zMistralRMSNorm.__init__s1 ll5::k#:; #r<c"|j}|jtj}|j dj dd}|tj ||jzz}|j|j|zS)NrHrGT)keepdim) rpryrLrxpowmeanrsqrtrr)r9r\ input_dtypevariances r;r@zMistralRMSNorm.forwardsy#)) %((7  $$Q',,R,>% Ht?T?T4T(UU {{]--k:::r<c^t|jjd|jS)Nz, eps=)tuplerrKrr9s r; extra_reprzMistralRMSNorm.extra_reprs* ))*+6$2G2G1HIIr<)gư>)rArBrCr.r@rrDrEs@r;rrs$;Jr<rcpeZdZdedeffd Z ddejdeejdeejdee dee d ee d eejd ee ejejfd e ed e ejee ejejfffdZxZS)MistralDecoderLayerr/rc t||j|_t|||_t ||_t|j|j|_ t|j|j|_ y)N)r/rr) r-r.r0r self_attnr(mlpr rms_norm_epsinput_layernormpost_attention_layernormrs r;r.zMistralDecoderLayer.__init__sl !--)9Mf%-f.@.@fFYFYZ(6v7I7IvObOb(c%r<r\rkrWrr use_cacherrr{r^c |} |j|}|jd||||||||d| \}} | |z}|} |j|}|j|}| |z}|f} |r| | fz } | S)N)r\rkrWrrrrr)rrrr) r9r\rkrWrrrrrr{residualself_attn_weightsoutputss r;r@zMistralDecoderLayer.forwards!,,]; ,:4>> , ')%)/) 3 ,  , ( (!=0 !55mD /  =0  "  )+ +Gr<)NNNFFNN)rArBrCr$rr.rLrrrr boolrrr FloatTensorr@rDrEs@r;rrsd}dd2637*.,1$)59KO'||'!.'u//0 ' ! ' $D> 'D>'!!1!12'&eELL%,,,F&GH'-.' u  (51B1BEDUDU1U+V"WW X'r<rcFeZdZeZdZdZdgZdgZdZ dZ dZ dZ dZ dZdZdZy)MistralPreTrainedModelmodelTrpast_key_valuesc|jj}t|tjrY|j j jd||j%|jj jyyt|tjrf|j j jd||j2|j j |jjyyt|tr&|j j jdyy)Nr)rstdg?)r/initializer_range isinstancerr2rdatanormal_r,zero_ Embedding padding_idxrfill_)r9rgrs r; _init_weightsz$MistralPreTrainedModel._init_weightsskk++ fbii ( MM   & &CS & 9{{&   &&('  - MM   & &CS & 9!!- ""6#5#56<<>.  / MM   $ $S )0r<N)rArBrCr$ config_classbase_model_prefixsupports_gradient_checkpointing_no_split_modules_skip_keys_device_placement_supports_flash_attn_2_supports_sdpa_supports_flex_attn_supports_cache_class_supports_quantized_cache_supports_static_cache_supports_attention_backendrrr<r;rrsS L&*#./#4"5!N  $!"& *r<rc^eZdZddeffd Zej edZxZ S)MistralRotaryEmbeddingr/ct|t|drG|j;|jj d|jj d|_nd|_|j |_|j |_||_ t|j |_ |j|j|\}|_ |jd|d|j|_y)N rope_scaling rope_typetypedefaultinv_freqF) persistent)r-r.hasattrrrrmax_position_embeddingsmax_seq_len_cachedoriginal_max_seq_lenr/r rope_init_fnattention_scalingregister_bufferroriginal_inv_freq)r9r/devicerr:s r;r.zMistralRotaryEmbedding.__init__,s  6> *v/B/B/N#0044[&BUBUBYBYZ`BabDN&DN"("@"@$*$B$B! /?+/+<+>@GG HZHZ[\H]_acdehhijiqiqr ,QaZ 8 > > @'1!((--'E!((--[`J`ahhmmfk ^^ U C 5&,,.1F1L1L1NNYYZ[]^_E))UEN3C'')d444C'')d444C  5 vvAGGv$cff177f&;;;  5 5s BF%%F.r>) rArBrCr$r.rLno_gradrr@rDrEs@r;rr+s3/}/"U]]_ < r rs r;get_input_embeddingsz!MistralModel.get_input_embeddings_s   r<c||_yr>rr9rjs r;set_input_embeddingsz!MistralModel.set_input_embeddingsbs !r< input_idsrkrWr inputs_embedsrroutput_hidden_statesrflash_attn_kwargsr^c ||n|jj}||n|jj}||n|jj}|du|duz r t d|j r%|j r|rtjdd}t|tdtfs t d||j|}|r | t}| F||jnd} tj | | |j"dz|j$} || j'd}|j)||| ||} |} |j+| |}|rdnd}|rdnd}|j,d|jj.D],}|r|| fz }|| f| ||||| |d | }|d} |s$||dfz }.|j1| } |r|| fz }t3| |r|nd|| S) Nz:You must specify exactly one of input_ids or inputs_embedszX`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`.FzBThe `past_key_values` should be either a `Cache` object or `None`.rr#rr)rkrWrrrrr)last_hidden_staterr\ attentions)r/rrr ValueErrorrrrrrrrr r r get_seq_lengthrLarangerKrrR_update_causal_maskrrrrr)r9rrkrWrrrrrrrpast_seen_tokensrr\rall_hidden_statesall_self_attns decoder_layer layer_outputss r;r@zMistralModel.forwardesT2C1N-TXT_T_TqTq$8$D $++JjJj "+!6IDKK?ab b   --i8M 0*nO  !CRC^==?de "\\ "2]5H5H5K"KTaThThN  )33A6L.. M>?L] & #oom\J#7BD0d![[)H4;;+H+HI 6M#!m%55!) *)."3#-$7 $ M*!,M =#3"55' 6* -0  -!1 1 &+/8Od+%   r<r% input_tensorc l|jjdk(rS|H|F|dddfjj|j dk7}|r t d|d|vr|Sy|jjdk(r't |tjr t|}|S||jnd}t |t}t |t} |jjdk(r?|s=| s;|s9tj||||jj|j ry|j"} tj$| j&} |j(d } | s|r|j+} n1t |tjr|j(dn|| zd z} |j-|| | | ||j(d|j| }|jjdk(r2|0|j.j0d vr|stj2|| }|S) Nflash_attention_2rGrzYou are attempting to perform batched generation with padding_side='right' this may lead to unexpected behaviour for Flash Attention version of Mistral. Make sure to call `tokenizer.padding_side = 'left'` before tokenizing the input. rflex_attentionr)rpast_key_values_lengthr is_trainingr#)sequence_length target_lengthrpr batch_sizer/r)cudaxpunpu)r/rsumitemsizer"rrLrr&r#rr r_ignore_causal_mask_sdparrrrpfinfominrKget_max_cache_shape5_prepare_4d_causal_attention_mask_with_cache_positionrr_unmask_unattended)r9rkr+rrris_padding_rightr&using_static_cacheusing_sliding_window_cacherp min_dtyper1r2rs r;r%z MistralModel._update_causal_masks: ;; + +/B B)o.I#1!R%#8#<#<#>#C#C#EIZIZI\]^I_#_ #$a )c^.C%% ;; + +/? ?.%,,7!<^!L! ! @O?Z?99;`a'E%/AS%T" KK , , 6'+E%%>>*'7#{{99 MM ""KK&** &,,Q/ %);+??AM nell;$$R(%7!; PP +')#))!,;;+Q   KK , , 6*%%**.DD% 1CCKQZ[Kr<r1r2rpr3c||jdk(r|}|Stj|j} tj||f| ||j }tj ||j |jddkD} |j} t| ddrs| jgt|tr||kDrRtj ||j |jdd| jz k} | j| || z}|ddddddfj|ddd}||j}|j d|kDr |ddd|f}|j d} |ddddddd| f|ddddddfj#|j z}|d k(}|ddddddd| fj%|| |ddddddd| f<|S) a Creates a causal 4D mask of shape `(batch_size, 1, query_length, key_value_length)` from a 2D mask of shape `(batch_size, key_value_length)`, or if the input `attention_mask` is already 4D, do nothing. Args: attention_mask (`torch.Tensor`): A 2D attention mask of shape `(batch_size, key_value_length)` or a 4D attention mask of shape `(batch_size, 1, query_length, key_value_length)`. sequence_length (`int`): The sequence length being processed. target_length (`int`): The target length: when generating with static cache, the mask should be as long as the static cache, to account for the 0 padding, the part of the cache that is not filled yet. dtype (`torch.dtype`): The dtype to use for the 4D attention mask. cache_position (`torch.Tensor`): Indices depicting the position of the input sequence tokens in the sequence. batch_size (`torch.Tensor`): Batch size. config (`MistralConfig`): The model's configuration class past_key_values (`Cache`): The cache class that is being used currently to generate N) fill_valuerprrrGr#use_sliding_windowTr)rJrLr;r<fullrr$raget_text_configrrrr bitwise_or_r`clonerKry masked_fill)rkr1r2rprr3r/rrrCdiagonal_attend_mask text_configsliding_attend_mask mask_length padding_masks r;r>zBMistralModel._prepare_4d_causal_attention_mask_with_cache_positions%B  %.*<*<*>!*C(K@= E*..I** -0Ye\j\q\qK$)<< nF[F[#\_m_u_uA`$ !002K{$8$?KD^D^Dj"/3EF/\iJi*/,,}^MbMb*c&..r15 8R8RR+')445HI / /K%dD!Q&67>>z1bRTUK))//1 !''+m;%3A~ ~4E%FN,2226 *1aL[L+@ANSTVZ\`bcScDdDgDg&&E   ,q0 5@Aq,;,AV5W5c5c )6 Aq!\k\12r< NNNNNNNNN)F)rArBrCr$r.rrr rrrLrrr rrrrrr@rr% staticmethodrrpr>rDrEs@r;rrMs} !"151537+/59$(,0/359\ E,,-\ !.\ u//0 \ "% \   1 12 \ D>\ $D>\ 'tn\ !!1!12\ $$89\  !\ \ H#( TellK78TllT T  T  TlB BBB{{ B  B  BBBBr<rc eZdZy)KwargsForCausalLMN)rArBrCrr<r;rUrUasr<rUceZdZdgZddiZddgdgfiZfdZdZdZd Z d Z d Z d Z e e dd eej"deej$deej"deedeej(deej"deedeedeedeej"deeej$fdeedefdZxZS)MistralForCausalLMzlm_head.weightlm_head colwise_repr\logitsct||t||_|j|_t j |j|jd|_|jyr*) r-r.rrr rr2r0rXrr8s r;r.zMistralForCausalLM.__init__jsU  !&)  ++yy!3!3V5F5FUS  r<c.|jjSr>rr rs r;rz'MistralForCausalLM.get_input_embeddingsszz&&&r<c&||j_yr>r]rs r;rz'MistralForCausalLM.set_input_embeddingsv"' r<c|jSr>rXrs r;get_output_embeddingsz(MistralForCausalLM.get_output_embeddingsys ||r<c||_yr>rb)r9new_embeddingss r;set_output_embeddingsz(MistralForCausalLM.set_output_embeddings|s % r<c||_yr>r)r9decoders r; set_decoderzMistralForCausalLM.set_decoders  r<c|jSr>rhrs r; get_decoderzMistralForCausalLM.get_decoders zzr<rrkrWrrlabelsrrrrlogits_to_keepr{r^c ||n|jj}| | n|jj} |jd|||||||| | d | } | j}t | t r t| dn| }|j|dd|ddf}d}|*|jd|||jjd| }t||| j| j| jS)a labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example: ```python >>> from transformers import AutoTokenizer, MistralForCausalLM >>> model = MistralForCausalLM.from_pretrained("meta-mistral/Mistral-2-7b-hf") >>> tokenizer = AutoTokenizer.from_pretrained("meta-mistral/Mistral-2-7b-hf") >>> prompt = "Hey, are you conscious? Can you talk to me?" >>> inputs = tokenizer(prompt, return_tensors="pt") >>> # Generate >>> generate_ids = model.generate(inputs.input_ids, max_length=30) >>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] "Hey, are you conscious? Can you talk to me?\nI'm not conscious, but I can talk to you." ```N) rrkrWrrrrrr)rZrmr lossrZrr\r!r)r/rrrr rrslicerX loss_functionr rrr\r!)r9rrkrWrrrmrrrrrnr{rr\ slice_indicesrZrqs r;r@zMistralForCausalLM.forwardsN2C1N-TXT_T_TqTq$8$D $++JjJj  ,64:: , )%+'/!5) ,  ,  11 8B>SV8W~ot4]k mA}a,?@A  %4%%pVFt{{OeOepiopD%#33!//))   r<) NNNNNNNNNNr)rArBrC_tied_weights_keys_tp_plan_pp_planr.rrrcrfrjrlr rrrLrrr rrrrrrUrr@rDrEs@r;rWrWds*+=)H_-z:;H'(&151537+/59-1$(,0/35934G E,,-G !.G u//0 G "% G   1 12 G ))*G D>G $D>G 'tnG !!1!12G c5<</0G *+G  G G r<rWceZdZfdZdZdZee ddee jdee jdee jdee dee jd ee jd eed eed eed efdZxZS)MistralForTokenClassificationct|||j|_t||_t |dd |j }nt |dd |j}nd}tj||_ tj|j|j|_ |jy)Nclassifier_dropouthidden_dropoutg?)r-r. num_labelsrrrr{r|rDropoutrmr2r0scorer)r9r/r{r:s r;r.z&MistralForTokenClassification.__init__s   ++!&) 6/ 6 B!'!:!:  V-t 4 @!'!6!6 !$ zz"45 YYv1163D3DE  r<c.|jjSr>r]rs r;rz2MistralForTokenClassification.get_input_embeddingsr^r<c&||j_yr>r]rs r;rz2MistralForTokenClassification.set_input_embeddingsr`r<rrkrWrrrmrrrr^c |j|||||||| } | j} |j| } |j| } d} ||j | ||j } t | | | j| jS) labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). rkrWrrrrrN)rqrZr\r!) rr rmrrsr/rr\r!)r9rrkrWrrrmrrrrsequence_outputrZrqs r;r@z%MistralForTokenClassification.forwards*,0:: )%+'/!5,6 , "33,,7O,  %%ffdkkBD$!//))   r<rR)rArBrCr.rrr rrrLrrr rrrr@rDrEs@r;ryrys '(151537+/59-1$(,0/3* E,,-* !.* u//0 * "% *   1 12 * ))** D>* $D>* 'tn*  * * r<rya The Mistral Model transformer with a sequence classification head on top (linear layer). [`MistralForSequenceClassification`] uses the last token in order to do the classification, as other causal models (e.g. GPT-2) do. Since it does classification on the last token, it requires to know the position of the last token. If a `pad_token_id` is defined in the configuration, it finds the last token that is not a padding token in each row. If no `pad_token_id` is defined, it simply takes the last value in each row of the batch. Since it cannot guess the padding tokens when `inputs_embeds` are passed instead of `input_ids`, it does the same (take the last value in each row of the batch). ) custom_introceZdZfdZdZdZee ddee jdee jdee jdee dee jd ee jd eed eed eed efdZxZS) MistralForSequenceClassificationct|||j|_t||_t j |j|jd|_|jyr*) r-r.r}rrrr2r0rrr8s r;r.z)MistralForSequenceClassification.__init__'sS   ++!&) YYv114??O  r<c.|jjSr>r]rs r;rz5MistralForSequenceClassification.get_input_embeddings0r^r<c&||j_yr>r]rs r;rz5MistralForSequenceClassification.set_input_embeddings3r`r<rrkrWrrrmrrrr^c |j|||||||| } | j} |j| } ||jd} n|jd} |jj | dk7r t d|jj d}n|||jj k7j| jtj}tj|jd| jtj}||zjd}n.d}tj|jj d| tj| | j |f}d}||j#| |||j }t%||| j&| j(| j* S) rrNrr#z=Cannot handle batch sizes > 1 if no padding token is defined.rG)rrpz will not detect padding tokens in `inputs_embeds`. Results may be unexpected if using padding tokens in conjunction with `inputs_embeds.`r)rZrm pooled_logitsr/rp)rr rrKr/r r"ryrrLint32r$argmaxrrr:rArsrrr\r!)r9rrkrWrrrmrrrtransformer_outputsr\rZr3last_non_pad_token non_pad_mask token_indicesrrqs r;r@z(MistralForSequenceClassification.forward6s*8!F!Fr!J !#    >>**+,ZZ  u||Jv}}MOaab   %%VFR_hlhshs%tD/ /??-;;*55   r<rR)rArBrCr.rrr rrrLrrr rrrr@rDrEs@r;rrs'(151537+/59-1$(,0/3A E,,-A !.A u//0 A "% A   1 12 A ))*A D>A $D>A 'tnA  *A A r<rcXeZdZdZfdZdZdZee dde e jde e jde e jde e eee j fd e e j d e e jd e e jd e ed e edefdZxZS)MistralForQuestionAnsweringrct||tj|jd|_t ||_|jy)NrH) r-r.rr2r0 qa_outputsrrrr8s r;r.z$MistralForQuestionAnswering.__init__s@  ))F$6$6:!&)  r<c.|jjSr>r]rs r;rz0MistralForQuestionAnswering.get_input_embeddingsr^r<c&||j_yr>r]rs r;rz0MistralForQuestionAnswering.set_input_embeddingsr`r<rrkrWrrstart_positions end_positionsrrr^c |j||||||| } | j} |j| } | jdd\}}|j dj }|j dj }d}|||j ||||fi| }t|||| j| jS)a start_positions (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for position (index) of the start of the labelled span for computing the token classification loss. Positions are clamped to the length of the sequence (`sequence_length`). Position outside of the sequence are not taken into account for computing the loss. end_positions (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for position (index) of the end of the labelled span for computing the token classification loss. Positions are clamped to the length of the sequence (`sequence_length`). Position outside of the sequence are not taken into account for computing the loss. )rkrWrrrrr#rGrIN)rq start_logits end_logitsr\r!) rr rsplitsqueezerzrsrr\r!)r9rrkrWrrrrrrr{rrrZrrrqs r;r@z#MistralForQuestionAnswering.forwards4,0:: )%+'/!5,6, "331#)<<r<#: j#++B/::< ''+668   &=+D%4%%lJQ^ibhiD+%!!//))   r<rR)rArBrCrr.rrr rrrLrrrr rrrrr@rDrEs@r;rr|s'(151537KO596:48,0/33 E,,-3 !.3 u//0 3 "%tE4E4E/F(F"GH 3   1 12 3 "%"2"233   0 013 $D>3 'tn3  &3 3 r<r)rWrrrrry)Nr#)r)KtypingrrrrrrLr activationsr cache_utilsr r r r generationr integrationsrmodeling_attn_mask_utilsrmodeling_flash_attention_utilsrmodeling_layersrmodeling_outputsrrrrrmodeling_rope_utilsrrmodeling_utilsrrprocessing_utilsrutilsrrr r!r"configuration_mistralr$!torch.nn.attention.flex_attentionr%integrations.flex_attentionr& get_loggerrArModuler(rPr[rrrfrrrrrrrrrUrWryrr__all__rr<r;rsE:9 !OO)7>B9LF&hh0 !;J   H %   (6 UU\\ U# U%,, U&% II% <<% % << % U\\* %  %%4A)ryyA)HY'JRYYJ(J(040f*_**8i /i i XC $:C C L  S '=S  S lF "8F F R r<