Uh:dZddlmZmZmZmZmZddlZddlZddlm Z ddl m Z m Z m Z ddlmZddlmZmZdd lmZdd lmZdd lmZmZdd lmZmZmZmZdd lm Z m!Z!ddl"m#Z#ddl$m%Z%m&Z&m'Z'm(Z(m)Z)ddl*m+Z+e(r ddl,m-Z-ddl.m/Z/er e)j`e1Z2Gdde jfZ4 d3de jjdejldejldejldeejlde7de7fdZ8Gdde jjZ9Gd d!e jjZ:e&Gd"d#e!Z;Gd$d%e;ZGd*d+e;eZ?e&d,-Gd.d/e;Z@e&Gd0d1e;ZAgd2ZBy)4zPyTorch OPT model.)CallableListOptionalTupleUnionN)nn)BCEWithLogitsLossCrossEntropyLossMSELoss)ACT2FN)Cache DynamicCache)GenerationMixin)AttentionMaskConverter)FlashAttentionKwargsis_flash_attn_available)BaseModelOutputWithPastCausalLMOutputWithPastQuestionAnsweringModelOutput SequenceClassifierOutputWithPast)ALL_ATTENTION_FUNCTIONSPreTrainedModel)Unpack) LossKwargsauto_docstringcan_return_tupleis_torch_flex_attn_availablelogging) OPTConfig) BlockMask)make_flex_block_causal_maskcxeZdZdZdedeffd Z d dejdedeejffd Z xZ S) OPTLearnedPositionalEmbeddingzN This module learns positional embeddings up to a fixed maximum size. num_embeddings embedding_dimcNd|_t| ||jz|yN)offsetsuper__init__)selfr&r' __class__s v/var/www/catia.catastroantioquia-mas.com/valormas/lib/python3.12/site-packages/transformers/models/opt/modeling_opt.pyr-z&OPTLearnedPositionalEmbedding.__init__;s$  $++5}Eattention_maskpast_key_values_length position_idsc|8tj|d}||zdz j}|dd|df}t|||j zS)z3`input_ids_shape` is expected to be [bsz x seqlen].Nr dim)torchcumsumlongr,forwardr+)r.r2r3r4r/s r0r;z%OPTLearnedPositionalEmbedding.forwardAs^   <<A>L(>9A=CCEL'+A+B(BCLw|dkk9::r1rN) __name__ __module__ __qualname____doc__intr-r8 LongTensorrr; __classcell__r/s@r0r%r%6s]FsF3F'(37 ;((;!$;u//0 ;;r1r%modulequerykeyvaluer2scalingdropoutc tj||jdd|z}|||z}tjj |dtj j|j}tjj|||j}tj||} | jddj} | |fS)N)r7dtypeptrainingr r*) r8matmul transposer functionalsoftmaxfloat32torNrJrQ contiguous) rErFrGrHr2rIrJkwargs attn_weights attn_outputs r0eager_attention_forwardr\Ss<<s}}R'<=GL!#n4 ==((2U]](SVVW\WbWbcL==((6??([L,,|U3K''1-88:K  $$r1c(eZdZdZ d dedeeffd Z ddejdee ejdeejdeejd e d eejd e ejeejee ffd Z xZS) OPTAttentionz=Multi-headed attention from 'Attention Is All You Need' paperconfig layer_idxc t|||_|j|_|j |_|j|_|j|_ ||_ |-tjd|jjd|j|j z|_d|_|j|j z|jk7r&t#d|jd|j d|jdz|_t'j(|j|j|j|_t'j(|j|j|j|_t'j(|j|j|j|_t'j(|j|j|j|_y) NzInstantiating z without passing a `layer_idx` is not recommended and will lead to errors during the forward call if caching is used. Please make sure to provide a `layer_idx` when creating this class.Tz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: z).g࿩bias)r,r-r_ hidden_size embed_dimnum_attention_heads num_headsattention_dropoutrJ enable_biasr`logger warning_oncer/r=head_dim is_causal ValueErrorrIrLineark_projv_projq_projout_proj)r.r_r`rYr/s r0r-zOPTAttention.__init__ms  ++33// !--"      !8!8 9:,,  $..8  MMDNN *t~~ =MdnnM]$T^^$4B8 }}d* iiTEUEUV iiTEUEUV iiTEUEUV  $..$..tGWGWX r1 hidden_statespast_key_valuer2layer_head_maskoutput_attentionscache_positionreturnc |j\}} } |j||jz} | j|d|j|j j dd} |j|} |j|} | j|d|j|j j dd} | j|d|j|j j dd} |#|j| | |jd|i\} } t}|jjdk7rN|jjdk(r|rtjdnt |jj}||| | | |f|j"sd n |j$d d |\}}|j'|| dj)}|j+|}|sd}|||fS) z#Input shape: Batch x Time x ChannelrLr r*Nrxeagersdpaz`torch.nn.functional.scaled_dot_product_attention` does not support `output_attentions=True`. Falling back to eager attention. This warning can be removed using the argument `attn_implementation="eager"` when loading the model.?)rJrI)sizerrrIviewrgrlrSrprqupdater`r\r__attn_implementationrjrkrrQrJreshaperXrs)r.rtrur2rvrwrxrYbsztgt_len_ query_states key_states value_statesattention_interfacer[rZs r0r;zOPTAttention.forwards(,,.Wa{{=1DLL@ #((b$..$--PZZ[\^_` [[/ {{=1 __S"dnndmmLVVWXZ[\ #((b$..$--PZZ[\^_`  %'5'<'<L$..;K^:\( $J )@ ;; + +w 6{{//69>O##L '>dkk>^>^&_#$7      % $}}C$,, %  % ! \"))#w;FFH mmK0  LL.88r1N)NNNFN)r=r>r?r@r!rrAr-r8Tensorrboolrr;rCrDs@r0r^r^jsG $(!Y!YC=!YL9=1526"'15<9||<9!u||!45<9!. <9 "%,,/ <9  <9!.<9 u||Xell3Xe_D E<9r1r^cteZdZddedeeffd Z ddejdeejdeejdee ejdee d ee d eejd eejd e e d e ejee ejejfffdZxZS)OPTDecoderLayerr_r`ct||j|_t |||_|j |_|j|_t|j|_ tj|j|j|_tj|j|j |j"|_tj|j |j|j"|_tj|j|j|_y)N)r_r`elementwise_affinerb)r,r-rdrer^ self_attndo_layer_norm_beforerJr activation_function activation_fnr LayerNormlayer_norm_elementwise_affineself_attn_layer_normroffn_dimrifc1fc2final_layer_norm)r.r_r`r/s r0r-zOPTDecoderLayer.__init__s ++%VyI$*$?$?!~~ #F$>$>?$&LL NNv/S/S% !99T^^V^^&BTBTU99V^^T^^&BTBTU " T^^PVPtPt ur1rtr2rvrurw use_cacher4rxrYryc "|} |jr|j|}|jd|||||||d| \}} } tjj ||j |j }| |z}|js|j|}|j} |jd|jd}|} |jr|j|}|j|}|j|}|j|}tjj ||j |j }| |zj| }|js|j|}|f}|r|| fz }|r|| fz }|S)a Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`, *optional*): attention mask of size `(batch, 1, tgt_len, src_len)` where padding elements are indicated by very large negative values. layer_head_mask (`torch.FloatTensor`, *optional*): mask for attention heads in a given layer of size `(encoder_attention_heads,)`. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. use_cache (`bool`, *optional*): If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding (see `past_key_values`). past_key_value (`Tuple(torch.FloatTensor)`, *optional*): cached past key and value projection states cache_position (`torch.LongTensor` of shape `(sequence_length)`, *optional*): Indices depicting the position of the input sequence tokens in the sequence.. )rtrur4r2rvrwrxrOrL)rrrrrTrJrQshaperrrrrrr)r.rtr2rvrurwrr4rxrYresidualself_attn_weightspresent_key_valuehidden_states_shapeoutputss r0r;zOPTDecoderLayer.forwards<!  $ $ 55mDM?Mdnn ? ')%)+/) ?  ? ; (*; --mt||VZVcVc-d  =0 (( 55mDM,11%--b-2D2DR2HI    $ $ 11-@M/ **=9 /  --mt||VZVcVc-d !M1778KL (( 11-@M "  )+ +G  )+ +Gr1r)NNNFFNN)r=r>r?r!rrAr-r8rrrrBrr FloatTensorr;rCrDs@r0rrsvyvXc]v(26268<,1$)3715S||S!.S"%,,/ S !u||!45 S $D> SD>Su//0S!.S-.S u  (51B1BEDUDU1U+V"WW XSr1rc@eZdZeZdZdZdgZdZdZ dZ dZ dZ dZ dZdZy)OPTPreTrainedModelmodelTrc|jj}t|tjrY|j j jd||j%|jj jyyt|tjrf|j j jd||j2|j j |jjyyt|tjrJ|j j jd|jj jyy)Nr})meanstdr~)r_init_std isinstancerroweightdatanormal_rczero_ Embedding padding_idxrfill_)r.rErs r0 _init_weightsz OPTPreTrainedModel._init_weightsEskk"" fbii ( MM   & &CS & 9{{&   &&('  - MM   & &CS & 9!!- ""6#5#56<<>.  - MM   $ $S ) KK   " " $.r1N)r=r>r?r! config_classbase_model_prefixsupports_gradient_checkpointing_no_split_modules_supports_attention_backend_supports_flash_attn_2_supports_sdpa_supports_flex_attn_supports_cache_class_supports_quantized_cache_supports_static_cacherrr1r0rr7sJL&*#*+"&!N  $! %r1rcBeZdZdZdeffd ZdZdZ ddee jdfde jd e jd e d e f d Z ede jd edede j d e jdef dZe ddee j(dee jdee jd eee j,dee j,dee d ee dee dee dee j(d ee jdeedeeeffdZxZS) OPTDecoderz Transformer decoder consisting of *config.num_hidden_layers* layers. Each layer is a [`OPTDecoderLayer`] Args: config: OPTConfig r_c 8t|||j|_|j|_|j|_|j |_|j|_tj|j|j|j |_ t|j |j|_|j|jk7r2tj |j|jd|_nd|_|j|jk7r2tj |j|jd|_nd|_|j&r=|j(s1tj*|j|j,|_nd|_tj0t3|j4Dcgc]}t7||c}|_d|_|j=ycc}w)NFrbr)r`)r,r-rJ layerdrop pad_token_idrmax_position_embeddingsmax_target_positions vocab_sizerrword_embed_proj_dim embed_tokensr%rdembed_positionsro project_out project_inr_remove_final_layer_normrrr ModuleListrangenum_hidden_layersrlayersgradient_checkpointing post_init)r.r_ir/s r0r-zOPTDecoder.__init__\s  ~~ ))!..$*$B$B! ++LL):):FT]n%>>*'7 MM  ""&,,Q/ !+??AMnell;$$R(%7!; PP +')#))!, Q  KK , , 6*%%**.DD%  E*..I0CCKQZ[Kr1rrrNrc ||jdk(r|}|Stj|j}tj||f|||j }|dk7rtj |d}|tj||j |jddkDz}|ddddddfj|ddd}||j}|jd} |ddddddd| f|ddddddfj|j z} | dk(} |ddddddd| fj| ||ddddddd| f<|S) aM Creates a causal 4D mask of shape `(batch_size, 1, query_length, key_value_length)` from a 2D mask of shape `(batch_size, key_value_length)`, or if the input `attention_mask` is already 4D, do nothing. Args: attention_mask (`torch.Tensor`): A 2D attention mask of shape `(batch_size, key_value_length)` or a 4D attention mask of shape `(batch_size, 1, query_length, key_value_length)`. sequence_length (`int`): The sequence length being processed. target_length (`int`): The target length: when generating with static cache, the mask should be as long as the static cache, to account for the 0 padding, the part of the cache that is not filled yet. dtype (`torch.dtype`): The dtype to use for the 4D attention mask. cache_position (`torch.Tensor`): Indices depicting the position of the input sequence tokens in the sequence. batch_size (`torch.Tensor`): Batch size. N) fill_valuerNrr )diagonalrrLr)r7r8rrfullrtriuarangerexpandclonerrW masked_fill) r2rrrNrxrrYrr mask_length padding_masks r0rz@OPTDecoder._prepare_4d_causal_attention_mask_with_cache_positions>  %.*<*<*>!*C(K*' E*..I** -0Ye\j\q\qK!##jjqA 5<< n>S>STWeWmWmnprsWtt tK%dD!Q&67>>z1bRTUK))//1 ,2226 *1aL[L+@ANSTVZ\`bcScDdDgDg&&E   ,q0 5@Aq,;,AV5W5c5c )6 Aq!\k\12r1 input_ids head_maskrroutput_hidden_states return_dictr4rYryc ||n|jj}||n|jj}||n|jj}| | n|jj} |du|duz r t d|j r%|jr|rtjdd}||jd|jd}||j|}d} |r>t|ts.d} tj |}|tjd||j#nd}| 2t%j&|||jd z|j( } |A||jd z}t%j*|jd||j( }|j-||| ||}| 8t%j.|d } | |zd z j1} | dd|df} |j3||| }|j4|j5|}||j7|j(z}|rd nd}|rd nd}d}t9|gdgD]j\}}| |j;dt=|j>k7s3t d|dt=|j>d|j;ddtA|j>D]\}}|r||fz }|jr%t%jBg}||jDkr?|j r7|jr+|jG|jH|||||ndd||| | }n||f|| |||nd|||| d| }|d}|r ||rdnd }|s||d fz }|jJ|jK|}|jL|jM|}|r||fz }|r|nd}| r|jO}tQ||||S)a Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Padding will be ignored by default should you provide it. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) head_mask (`torch.Tensor` of shape `(num_hidden_layers, num_attention_heads)`, *optional*): Mask to nullify selected heads of the attention modules. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. past_key_values (`tuple(tuple(torch.FloatTensor))`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`) and 2 additional tensors of Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. If `past_key_values` are used, the user can optionally input only the last `decoder_input_ids` (those that don't have their past key value states given to this model) of shape `(batch_size, 1)` instead of all `decoder_input_ids` of shape `(batch_size, sequence_length)`. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. position_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Indices of positions of each input sequence tokens in the position embeddings. Selected in the range `[0, config.n_positions - 1]`. for padding use -1. [What are position IDs?](../glossary#position-ids) cache_position (`torch.LongTensor` of shape `(sequence_length)`, *optional*): Indices depicting the position of the input sequence tokens in the sequence. Contrarily to `position_ids`, this tensor is not affected by padding. It is used to update the cache in the correct position and to infer the complete sequence length. Nz:You must specify exactly one of input_ids or inputs_embedszX`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`.FrLTzPassing a tuple of `past_key_values` is deprecated and will be removed in Transformers v4.53.0. You should pass an instance of `DynamicCache` instead, e.g. `past_key_values=DynamicCache.from_legacy_cache(past_key_values)`.rr rr6)r4rr zThe `z` should be specified for z layers, but it is for .)r2r4rvrurwrrxr*last_hidden_staterrt attentions))r_rwr ruse_return_dictrnrrQrjrkrrrrrrfrom_legacy_cacherr8rronesrr9r:rrrWziprlenr enumeraterandr_gradient_checkpointing_func__call__rrto_legacy_cacher)r.r r2r rrrrwr r r4rxrYreturn_legacy_cacher seq_lengthr pos_embedsrtall_hidden_statesall_self_attnsnext_decoder_cache attn_mask mask_nameidx decoder_layerdropout_probability layer_outputs next_caches r0r;zOPTDecoder.forwardsP2C1N-TXT_T_TqTq$8$D $++JjJj "+!6IDKK?I   --i8M# Z?"& *<<_MO&##Y @O?Z?99;`a  !"\\ "2]5H5H5K"KTaThThN  !)M,?,?,BBJ"ZZ (;(;A(> S`SgSghN.. M>?L]   <<A>L(>9A=CCEL'+;+<(<=L)).:JYe)f ?? & OOM:M% m6J6J(KK #7BD0d!%( k]$C  Iy$>>#A&3t{{+;<$ {*DSEUDVW%NN,Q/03 #,DKK"8) 6 C#!m%55!}}&+jjn#&7**t}} $ A A!**!&/&;IcN% " ! !.! !#.!-7@7LYs^RV#2&7'#1 ! ! *!,M%28I1q%Q" =#3"55S) 6V  , 11-@M    ' ,,];M -!1 1 +4'$ #335J&+&+%   r1)F NNNNNNNNNNN)r=r>r?r@r!r-rrrr8rrrr staticmethodrArNrrrrBrrrrrrr;rCrDs@r0rrTs#y#J!"#( BellK78BllB B  B  BH4 444{{ 4  4  44l1515,0=A59$(,0/3&*3715R E,,-R !.R ELL) R "$u'8'8"9: R   1 12 R D>R $D>R 'tnR d^R u//0R !.R -.R  u-- .R R r1rceZdZdeffd ZdZdZdZee dde e jde e jde e jd e eee j efd e e j d e ed e ed e ede ede e jde e jdeedeeeffdZxZS)OPTModelr_cdt||t||_|j yr)r,r-rdecoderrr.r_r/s r0r-zOPTModel.__init__s&  !&)  r1c.|jjSrr.rrs r0rzOPTModel.get_input_embeddingss||(((r1c&||j_yrr1rs r0rzOPTModel.set_input_embeddingss$) !r1c|jSr)r.rs r0 get_decoderzOPTModel.get_decoder ||r1r r2r rrrrwr r r4rxrYryc  |||n|jj}||n|jj}||n|jj}| | n|jj} |j d||| ||||||d| d | } t | j| j| j| jS)NT r r2r4r rrrrwr r rxrr) r_rwr rrr.rrrrtr)r.r r2r rrrrwr r r4rxrYdecoder_outputss r0r;zOPTModel.forwards"2C1N-TXT_T_TqTq$8$D $++JjJj "+!6IDKKr?r!r-rrr4rrrr8rBrrrrrrrrrrr;rCrDs@r0r,r,sTy )*1515,0KO59$(,0/3&*3715+ E,,-+ !.+ ELL) + "%U->->(?(F"GH +   1 12 + D>+ $D>+ 'tn+ d^+ u//0+ !.+ -.+  u-- .+ + r1r,c eZdZy)KwargsForCausalLMN)r=r>r?rr1r0r:r:sr1r:c!eZdZdgZfdZdZdZdZdZdZ dZ e e dd e ejd e ej d e ej d e eeej&efd e ej&de ejde ede ede ede ede ejde ej deedeeeffdZedZxZS)OPTForCausalLMzlm_head.weightct||t||_t j |j |jd|_|jyNFrb) r,r-r,rrrorrlm_headrr/s r0r-zOPTForCausalLM.__init__!sK  f% yy!;!;V=N=NUZ[  r1cB|jjjSrrr.rrs r0rz#OPTForCausalLM.get_input_embeddings+zz!!...r1c:||jj_yrrArs r0rz#OPTForCausalLM.set_input_embeddings.*/ 'r1c|jSrr?rs r0get_output_embeddingsz$OPTForCausalLM.get_output_embeddings1r5r1c||_yrrF)r.new_embeddingss r0set_output_embeddingsz$OPTForCausalLM.set_output_embeddings4s % r1c&||j_yrrr.)r.r.s r0 set_decoderzOPTForCausalLM.set_decoder7s$ r1c.|jjSrrLrs r0r4zOPTForCausalLM.get_decoder:szz!!!r1r r2r rrlabelsrrwr r r4rxrYryc  ||n|jj}| | n|jj} | | n|jj} |jj d||| |||||| d| d | }|j |dj}d}|E|j|j}|j||fd|jji| }t|||j|j|jS)an labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example: ```python >>> from transformers import AutoTokenizer, OPTForCausalLM >>> model = OPTForCausalLM.from_pretrained("facebook/opt-350m") >>> tokenizer = AutoTokenizer.from_pretrained("facebook/opt-350m") >>> prompt = "Hey, are you conscious? Can you talk to me?" >>> inputs = tokenizer(prompt, return_tensors="pt") >>> # Generate >>> generate_ids = model.generate(inputs.input_ids, max_length=30) >>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] "Hey, are you conscious? Can you talk to me?\nI'm not conscious. I'm just a little bit of a weirdo." ```NTr7rrlosslogitsrrtrr)r_rwr rrr.r?rXrWr loss_functionrrrrtr)r.r r2r rrrOrrwr r r4rxrYrrSrRs r0r;zOPTForCausalLM.forward=s;R2C1N-TXT_T_TqTq$8$D $++JjJj &1%z0OPTForCausalLM._reorder_cache..s.nU_j--aZ=N=N1OPns58)tuple)rrZreordered_past layer_pasts ` r0_reorder_cachezOPTForCausalLM._reorder_caches=) J ncmnn N r1 NNNNNNNNNNNN)r=r>r?_tied_weights_keysr-rrrGrJrMr4rrrr8rBrrrrrrrr:rrr;r*r_rCrDs@r0r<r<s*+/0&%"1515,0KO59-1$(,0/3&*3715P E,,-P !.P ELL) P "%U->->(?(F"GH P   1 12 P ))*P D>P $D>P 'tnP d^P u//0P !.P *+P  u,, -P P dr1r<a The OPT Model transformer with a sequence classification head on top (linear layer). [`OPTForSequenceClassification`] uses the last token in order to do the classification, as other causal models (e.g. GPT-2) do. Since it does classification on the last token, it requires to know the position of the last token. If a `pad_token_id` is defined in the configuration, it finds the last token that is not a padding token in each row. If no `pad_token_id` is defined, it simply takes the last value in each row of the batch. Since it cannot guess the padding tokens when `inputs_embeds` are passed instead of `input_ids`, it does the same (take the last value in each row of the batch). ) custom_introcreZdZdeffd Ze ddeejdeejdeejdee e eje fdeejdeejd ee d ee d ee d ee d eejde eeffdZdZdZxZS)OPTForSequenceClassificationr_ct|||j|_t||_t j |j|jd|_|jyr>) r,r- num_labelsr,rrrorscorerr/s r0r-z%OPTForSequenceClassification.__init__sT   ++f% YYv994??QVW  r1r r2r rrrOrrwr r r4ryc | | n|jj} |j|||| ||||| |  } | d} |j| }||jdd\}}n|jdd\}}|jj |dk7r t d|jj d}n|||jj k7j|jtj}tj|jd|jtj}||zjd}n.d}tj|jj d |tj||j |f}d}||jj"|j$dk(rd |j_nl|j$dkDrL|j&tj(k(s|j&tj*k(rd |j_nd |j_|jj"d k(rIt-}|j$dk(r&||j/|j/}n|||}n|jj"d k(r=t1}||j3d|j$|j3d}n,|jj"d k(rt5}|||}| s|f| ddz}||f|zS|St7||| j8| j:| j<S)a labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). N rr2r4r rrrwr r rr*r z=Cannot handle batch sizes > 1 if no padding token is defined.rL)rrNz will not detect padding tokens in `inputs_embeds`. Results may be unexpected if using padding tokens in conjunction with `inputs_embeds.`r regressionsingle_label_classificationmulti_label_classificationrQ)r_rrrgrrrnrWrr8int32rargmaxrjrkr/r= problem_typerfrNr:rAr squeezer rr rrrtr)r.r r2r rrrOrrwr r r4transformer_outputsrtrSrrlast_non_pad_token non_pad_mask token_indices pooled_logitsrRloss_fctoutputs r0r;z$OPTForSequenceClassification.forwards*&1%!F!Fr!J !#    >>**+,ZZ  u||Jv}}MOaab   {{''/??a'/;DKK,__q(fllejj.HFLL\a\e\eLe/LDKK,/KDKK,{{''<7"9??a'#M$9$9$;V^^=MND#M6:D))-JJ+- 2 22t GUWY))-II,. v6#%(;AB(??F)-)9TGf$ Ev E/ /??-;;*55   r1cB|jjjSrrArs r0rz1OPTForSequenceClassification.get_input_embeddingsrBr1c:||jj_yrrArs r0rz1OPTForSequenceClassification.set_input_embeddingsrDr1r))r=r>r?r!r-rrr8rBrrrrrrrr;rrrCrDs@r0rdrdsLy156:15KO59-1$(,0/3&*37\ E,,-\ !!2!23\ E--. \ "%U->->(?(F"GH \   1 12 \ ))*\ D>\ $D>\ 'tn\ d^\ u//0\  u66 7\ \ |/0r1rdceZdZdeffd Ze ddeejdeejdeejdee e eje fdeejdeejd eejd ee d ee d ee d ee deejde eeffdZdZdZxZS)OPTForQuestionAnsweringr_ct||t||_t j |j d|_|jyr)) r,r-r,rrror qa_outputsrr/s r0r-z OPTForQuestionAnswering.__init__s@  f% ))F$>$>B r1r r2r rrstart_positions end_positionsrrwr r r4ryc | | n|jj} |j|||| |||| | |  } | d}|j|}|j dd\}}|j dj }|j dj }d}||t|jdkDr|j d}t|jdkDr|j d}|jd}|jd|j|j}|jd|j|j}t|}|||}|||}||zdz }| s||f| ddz}||f|zS|St|||| j| j S) a Example: ```python >>> from transformers import AutoTokenizer, OPTForQuestionAnswering >>> import torch >>> torch.manual_seed(4) # doctest: +IGNORE_RESULT >>> tokenizer = AutoTokenizer.from_pretrained("facebook/opt-350m") >>> # note: we are loading a OPTForQuestionAnswering from the hub here, >>> # so the head will be randomly initialized, hence the predictions will be random >>> model = OPTForQuestionAnswering.from_pretrained("facebook/opt-350m") >>> question, text = "Who was Jim Henson?", "Jim Henson was a nice puppet" >>> inputs = tokenizer(question, text, return_tensors="pt") >>> with torch.no_grad(): ... outputs = model(**inputs) >>> answer_start_index = outputs.start_logits.argmax() >>> answer_end_index = outputs.end_logits.argmax() >>> answer_offset = len(tokenizer(question)[0]) >>> predict_answer_tokens = inputs.input_ids[ ... 0, answer_offset + answer_start_index : answer_offset + answer_end_index + 1 ... ] >>> predicted = tokenizer.decode(predict_answer_tokens) >>> predicted ' a nice puppet' ```Nrirr rLr6) ignore_indexr*)rR start_logits end_logitsrtr)r_rrr}splitrprXrrclamprWrr rrtr)r.r r2r rrr~rrrwr r r4rqrtrSrr total_loss ignored_indexrv start_lossend_lossrws r0r;zOPTForQuestionAnswering.forward#s`&1%r?r!r-rrr8rBrrrrrrrr;rrrCrDs@r0r{r{sey156:15KO596:48$(,0/3&*37_ E,,-_ !!2!23_ E--. _ "%U->->(?(F"GH _   1 12 _ "%"2"23_   0 01_ D>_ $D>_ 'tn_ d^_ u//0_  u22 3_ _ B/0r1r{)r<r,rrdr{)r})Cr@typingrrrrrr8torch.utils.checkpointrtorch.nnr r r activationsr cache_utilsrr generationrmodeling_attn_mask_utilsrmodeling_flash_attention_utilsrrmodeling_outputsrrrrmodeling_utilsrrprocessing_utilsrutilsrrrrrconfiguration_optr!!torch.nn.attention.flex_attentionr"integrations.flex_attentionr# get_loggerr=rjrr%Modulerfloatr\r^rrrr,r:r<rdr{__all__rr1r0rs99 AA!.)>[ G&hh( !;J   H %;BLL;H% II% <<% % << % U\\* %  %%.b9299b9JebiieP%%%8C #C L = != = @?,j>z'zz  m0#5m0 m0`o00o0o0d r1