UhJ<dZddlZddlZddlmZmZmZmZddlZddl Zddlm Z ddl m Z m Z mZddlmZddlmZdd lmZmZdd lmZmZmZmZmZmZmZdd lmZdd l m!Z!m"Z"d dl#m$Z$e"jJe&Z'dejPde)de)fdZ*Gdde jVZ,Gdde jZZ.Gdde jZZ/Gdde jZZ0Gdde jZZ1Gdde jZZ2e!Gdd eZ3Gd!d"e3Z4Gd#d$e3Z5e!Gd%d&e3Z6e!d'(Gd)d*e3eZ7e!d+(Gd,d-e3Z8e!Gd.d/e3Z9Gd0d1e3Z:Gd2d3e3eZ;gd4Z(HPUPZPZcgcncncucufS"o (11!4Lw|dkk9::r)rN) __name__ __module__ __qualname____doc__intr3r;TensorrA __classcell__r5s@r'r+r+CsHFsF3F ; ;s ;^c^j^j ; ;r)r+ceZdZdZ ddededededef fd Zdejd ed efd Z dd ejd e ejde e ejde ejde ejde ejdede eje eje e ejffdZ xZS) MvpAttentionz=Multi-headed attention from 'Attention Is All You Need' paper embed_dim num_headsdropout is_decoderbiasct|||_||_||_||z|_|j |z|jk7rt d|jd|d|j dz|_||_tj||||_ tj||||_ tj||||_ tj||||_y)Nz;embed_dim must be divisible by num_heads (got `embed_dim`: z and `num_heads`: z).g࿩rS)r2r3rOrPrQhead_dimr$scalingrRrLineark_projv_projq_projout_proj)r4rOrPrQrRrSr5s r'r3zMvpAttention.__init___s "" !Y. MMI %$.. 8MdnnM]$YKr3 }}d* $ii 94@ ii 94@ ii 94@  )YTB r)tensorrCrBc|j|||j|jjddj S)Nrr0)viewrPrV transpose contiguous)r4r]rCrBs r'_shapezMvpAttention._shapezs7{{3GQQRSUVWbbddr) hidden_stateskey_value_statespast_key_valueattention_masklayer_head_mask attn_promptoutput_attentionsreturnc6 |du}|j\} } } |j||jz} |r| |d} |d}n |rE|j|j |d| } |j|j |d| }n|}|j|j |d| } |j|j |d| }t j|d| gd} t j|d|gd}nD|j|j |d| } |j|j |d| }|jr| |f}|t j|dj| ddd| gd} t j|dj| ddd|gd}|\t j| d| |djdj|j}t j||gd}| |jzd|jf}|j| | | j|} | j|} |j|}| jd}t j | | j#dd}|j| |jz| |fk7r/t%d| |jz| |fd|j|{|j| d| |fk7r#t%d | d| |fd|j|j| |j| ||z}|j| |jz| |}t&j(j+|d}||j|jfk7r*t%d |jfd|j|jdddd|j| |j| |z}|j| |jz| |}|r?|j| |j| |}|j| |jz| |}nd}t&j(j-||j,|j. }t j ||}|j| |jz| |jfk7r7t%d | |j| |jfd|j|j| |j| |j}|j#dd}|j1| | |j2}|j5|}|||fS) z#Input shape: Batch x Time x ChannelNrrr r0dimz$Attention weights should be of size z , but is z!Attention mask should be of size z/Head mask for a single layer should be of size ptrainingz `attn_output` should be of size )sizer[rWrbrYrZr;catrRr?zerostor:rPrVr_bmmr`r$r functionalsoftmaxrQrpreshaperOr\)r4rcrdrerfrgrhriis_cross_attentionrBtgt_len_ query_states key_states value_states prompt_mask proj_shapesrc_len attn_weightsattn_weights_reshaped attn_probs attn_outputs r'rAzMvpAttention.forward}s.T9',,.Wa{{=1DLL@ ."<'*J)!,L T[[1A%BBLJ;;t{{3C'Db#NL  'T[[%?SIJ;;t{{='A2sKLN1$5z#BJJ 99nQ&7%FANLT[[%?SIJ;;t{{='A2sKL ??),7N  "KN$9$9#r2r$JJ#W]^_J 99k!n&;&;CR&Ll%[abcL)#kk#q';q>;N;Nq;QRUUVdVkVkl !&K+Hr!SDNN*B > Ct{{<#>CCZP $Z__j1 (|((*5 //!$yyz/C/CAq/IJ    3#7'"J J6dnn8LgW^7_6`a %%'(*   %""$a'(BB 7a'8R7SS\]k]p]p]r\st(,,S$..'7SVddL',,S4>>-A7GTL}},,\r,B  &##%$..):: Et~~FWEXY',,./1+//2q!<|?P?PQTVZVdVdfmov?wwL',,S4>>-A7GTL  %1$5$5c4>>7T[$\ !055cDNN6JGU\]L$( !]]**<4<AAr))FT)NNNNNF)rErFrGrHrIfloatboolr3r;rJrbrrrArKrLs@r'rNrN\sDG   CCC C  C  C6eU\\eCece 488<1526.2"'wB||wB#5<<0wB!u||!45 wB !. wB "%,,/ wBell+wB wB u||Xell3XeELL>Q5RR SwBr)rNceZdZdeffd Z d dej dej dej dej deede ej eej ff d Z xZ S) MvpEncoderLayerconfigcft||j|_t |j|j |j |_tj|j|_ |j|_ t|j|_|j|_tj |j|j"|_tj |j"|j|_tj|j|_y)N)rOrPrQ)r2r3d_modelrOrNencoder_attention_headsattention_dropout self_attnr LayerNormself_attn_layer_normrQr activation_function activation_fnactivation_dropoutrXencoder_ffn_dimfc1fc2final_layer_normr4rr5s r'r3zMvpEncoderLayer.__init__s %nn44,,  %'LL$@!~~ #F$>$>?"(";";99T^^V-C-CD99V33T^^D " T^^ <r)rcrfrgself_attn_promptrirjc|}|j|||||\}}}tjj||j|j}||z}|j |}|}|j |j|}tjj||j|j}|j|}tjj||j|j}||z}|j|}|jtjk(rtj|js#tj |jrEtj"|jj$dz } tj&|| | }|f} |r| |fz } | S)a@ Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`): attention mask of size `(batch, 1, tgt_len, src_len)` where padding elements are indicated by very large negative values. layer_head_mask (`torch.FloatTensor`): mask for attention heads in a given layer of size `(encoder_attention_heads,)`. self_attn_prompt (`torch.FloatTensor`): prompt of self attention of shape `(2, encoder_attention_heads, pro_len, head_dim)`. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. )rcrfrgrhrirni)minmax)rrrvrQrprrrrrrr9r;float16isinfanyisnanfinforclamp) r4rcrfrgrriresidualrr{ clamp_valueoutputss r'rAzMvpEncoderLayer.forwards*!)-')+(/ *8* & |Q --mt||VZVcVc-d  =0 11-@  **488M+BC  --mt?V?Vaeanan-o /  --mt||VZVcVc-d  =0 --m<   %-- / KK & * * , M0J0N0N0P++m&9&9:>>EK!KK K<[YM "    &Gr))F) rErFrGrr3r; FloatTensorrrrrArKrLs@r'rrs=y=,-2 4((4))4** 4  ++ 4 $D> 4 u  (5+<+<"== >4r)rceZdZdeffd Z ddej deej deej deej deej deej d eej d eej d eeej d ee d ee deejeeejejfffdZ xZ S)MvpDecoderLayerrct||j|_t |j|j |j d|_|j|_t|j|_ |j|_ tj|j|_t |j|j |j d|_tj|j|_tj$|j|j&|_tj$|j&|j|_tj|j|_y)NT)rOrPrQrR)rQrR)r2r3rrOrNdecoder_attention_headsrrrQr rrrrrr encoder_attnencoder_attn_layer_normrXdecoder_ffn_dimrrrrs r'r3zMvpDecoderLayer.__init__@s %nn44,,   ~~ #F$>$>?"(";";$&LL$@!( NN  * *,,   (*||DNN'C$99T^^V-C-CD99V33T^^D " T^^ <r)rcrfencoder_hidden_statesencoder_attention_maskrgcross_attn_layer_head_maskrcross_attn_promptreri use_cacherjc x|} | | ddnd} |j|| |||| \}}}tjj||j|j}| |z}|j |}d}d}|x|} | | ddnd}|j ||||||| \}}}tjj||j|j}| |z}|j|}||z}|} |j|j|}tjj||j|j}|j|}tjj||j|j}| |z}|j|}|f}| r|||fz }| r||fz }|S)aC Args: hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)` attention_mask (`torch.FloatTensor`): attention mask of size `(batch, 1, tgt_len, src_len)` where padding elements are indicated by very large negative values. encoder_hidden_states (`torch.FloatTensor`): cross attention input to the layer of shape `(batch, seq_len, embed_dim)` encoder_attention_mask (`torch.FloatTensor`): encoder attention mask of size `(batch, 1, tgt_len, src_len)` where padding elements are indicated by very large negative values. layer_head_mask (`torch.FloatTensor`): mask for attention heads in a given layer of size `(encoder_attention_heads,)`. cross_attn_layer_head_mask (`torch.FloatTensor`): mask for cross-attention heads in a given layer of size `(decoder_attention_heads,)`. self_attn_prompt (`torch.FloatTensor`): prompt of self attention of shape `(2, decoder_attention_heads, pro_len, head_dim)`. cross_attn_prompt (`torch.FloatTensor`): prompt of cross attention of shape `(2, decoder_attention_heads, pro_len, head_dim)`. past_key_value (`Tuple(torch.FloatTensor)`): cached past key and value projection states output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. Nr0)rcrerfrgrhrirn)rcrdrfrgrhreri) rrrvrQrprrrrrrrr)r4rcrfrrrgrrrrerirrself_attn_past_key_valueself_attn_weightspresent_key_valuecross_attn_present_key_valuecross_attn_weightscross_attn_past_key_valuers r'rAzMvpDecoderLayer.forwardZsH!:H9S>"1#5Y] >Bnn'3)+(/ ?M? ; (*; --mt||VZVcVc-d  =0 11-@ (,$! ,$H@N?Yrs(;_c %NRN_N_+!65 :-8"3O`O KM-/KMM11-4< >G  )+ +Gr)) NNNNNNNNFT) rErFrGrr3r;rJrrrrrArKrLs@r'rr?s;=y=:268<9=26=A37488<,1$(_||_!._ ( 5 _ !) 6 _ "%,,/ _%-U\\$:_#5<<0_$ELL1_!u||!45_$D>_D>_ u  (51B1BEDUDU1U+V"WW X_r)rcleZdZdZdedededeffd Zdejdejfd Z xZ S) MvpClassificationHeadz-Head for sentence-level classification tasks. input_dim inner_dim num_classespooler_dropoutct|tj|||_tj ||_tj|||_y)Nro)r2r3rrXdenseDropoutrQr\)r4rrrrr5s r'r3zMvpClassificationHead.__init__sD YYy)4 zzN3  )[9 r)rcrjc|j|}|j|}tj|}|j|}|j |}|SN)rQrr;tanhr\)r4rcs r'rAzMvpClassificationHead.forwardsN ]3  =1  =1  ]3  m4 r)) rErFrGrHrIrr3r;rJrArKrLs@r'rrsL7 : : : :  :U\\ellr)rc`eZdZdZfdZdej deej fdZxZ S) MvpPromptz)Layer-wise prompt for encoder or decoder.c 8t||j|_||_||_|j |z|_tj|j|_ tj|j|j |_ tjtj|j |jtjtj|j|dz|j z|_y)Nrr0)r2r3 prompt_length num_layersrPrrVrrrQ Embeddingprompt_embedding SequentialrXprompt_mid_dimGELU prompt_trans)r4rrrPr5s r'r3zMvpPrompt.__init__s #11$")3 zzFNN3 " V-A-A6>> RMM IIfnnf&;&; < GGI IIf++Z!^fnn-L M r) prompt_idsrjc*|j|j|}|j|j|jdz|j |j }|j|}|jgdjd}|S)Nr0)rr0rr ) rrr_rrrPrVrQpermutesplit)r4rprompts r'rAzMvpPrompt.forwardsw""4#8#8#DET//11DdnnVZVcVcdf% -33A6 r)) rErFrGrHr3r;rJrrArKrLs@r'rrs+3  %,,53Fr)rc.eZdZeZdZdZdZedZ y)MvpPreTrainedModelmodelTc|jj}t|tjrY|j j jd||j%|jj jyyt|tjrf|j j jd||j2|j j |jjyyy)Nr)meanstd) rinit_std isinstancerrXr>datanormal_rSzero_r padding_idx)r4modulers r' _init_weightsz MvpPreTrainedModel._init_weightsskk"" fbii ( MM   & &CS & 9{{&   &&('  - MM   & &CS & 9!!- ""6#5#56<<>..r)c|jj}tjgddddd|gg|j}|j ||d}|S)N)r r0r r0r:)rfr)rrr;r]r:ne)r4 pad_tokenr dummy_inputss r'rzMvpPreTrainedModel.dummy_inputssWKK,, LL"2Q2q)4L!MVZVaVab 'll95" r)N) rErFrGr config_classbase_model_prefixsupports_gradient_checkpointingrpropertyrr)r'rrs,L&*# ?r)rceZdZdZ ddedeejdeeffd Z dZ dZ ddee jd ee jd ee jd ee jd eed eedeedeeeffdZxZS) MvpEncodera Transformer encoder consisting of *config.encoder_layers* self attention layers. Each layer is a [`MvpEncoderLayer`]. Args: config: MvpConfig embed_tokens (nn.Embedding): output embedding use_prompt (bool): whether to use prompt r embed_tokens use_promptc2t|||j|_|j|_|j }|j |_|j|_ |jrtj|nd|_ |||_n0tj |j"||j|_t%|j||_tj(t+|j,Dcgc] }t/|c}|_tj2||_||_|r7|j8|_t;||j,|j<|_d|_ |jCycc}wNg?F)"r2r3rQencoder_layerdrop layerdroprrrmax_position_embeddingsmax_source_positionsscale_embeddingmathsqrt embed_scalerrr vocab_sizer+embed_positions ModuleListrangeencoder_layersrlayersrlayernorm_embeddingrrrrrgradient_checkpointing post_init)r4rrrrOr{r5s r'r3zMvpEncoder.__init__sD  ~~ 11NN !..$*$B$B!393I3I499Y/s  # ,D  " V->-> 4K[K[ \D <  * *  mmeFLaLaFb$c_V%<$cd #%<< #: $ !'!5!5D $-%%..%D ! ',# %dsFc|jSrrr4s r'get_input_embeddingszMvpEncoder.get_input_embeddings;   r)c||_yrrr4values r'set_input_embeddingszMvpEncoder.set_input_embeddings> !r)rrf head_mask inputs_embedsrioutput_hidden_states return_dictrjc 4||n|jj}||n|jj}||n|jj}| | t d|$|}|j } |j d| d}n-| |jdd} |dddddf}n t d||j||jz}|j|} || z} |j| } tjj| |j|j} |j rIt#j$|j&j)|j*} |j-| } |t/||j0}|rdnd}|rdnd}|_|jdt3|j4k7r6t dt3|j4d |jdd t7|j4D]\}}|r|| fz}d }|jr&t#j8g}||j:krd }|rd }n{|j<rE|jr9|j?|j@| ||||nd|j r |nd|}n%|| ||||nd|j r |nd|}|d} |s||dfz}|r|| fz}|stCd| ||fDStE| ||S)a~ Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Padding will be ignored by default should you provide it. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) head_mask (`torch.Tensor` of shape `(encoder_layers, encoder_attention_heads)`, *optional*): Mask to nullify selected heads of the attention modules. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. NzDYou cannot specify both input_ids and inputs_embeds at the same timer z5You have to specify either input_ids or inputs_embedsrnrrz&The head_mask should be specified for  layers, but it is for .FT)NN)rgrrirc3&K|] }|| ywrr.0vs r' z%MvpEncoder.forward..seqWXWdeslast_hidden_staterc attentions)#rriruse_return_dictr$r"r_rqrrrr rrvrQrprr;r<rrtr:rrr9lenr  enumeraterandrr _gradient_checkpointing_func__call__tupler)r4rrfrrrirrinput input_shape embed_posrcrrencoder_statesall_attentionsidx encoder_layerto_dropdropout_probability layer_outputss r'rAzMvpEncoder.forwardAsV\2C1N-TXT_T_TqTq$8$D $++JjJj &1%cd d  "E++K!r;r?;I  &',,.s3K!!Q(+ETU U   --i84;K;KKM((/ % 1 00?  --mt||VZVcVc-d  ??d&8&89<>Ne]NN$See e+>Vd  r)NF)NNNNNNN)rErFrGrHrrrrrr3rrr; LongTensorrJrrrrrArKrLs@r'rr slq$$/7 /E$ZbcgZh$L!" 1515,059,0/3&*J E,,-J !.J ELL) J   1 12 J $D> J 'tnJ d^J  uo% &J r)rceZdZdZ ddedeejdeeffd Z dZ dZ ddee jd ee jd ee jd ee jd ee jd ee jdeee jdee jdeedeedeedeedeeeffdZxZS) MvpDecoderz Transformer decoder consisting of *config.decoder_layers* layers. Each layer is a [`MvpDecoderLayer`] Args: config: MvpConfig embed_tokens (nn.Embedding): output embedding use_prompt (bool): whether to use prompt rrrct|||j|_|j|_|j |_|j|_|jrtj|jnd|_ |||_n:tj |j"|j|j |_t%|j|j|_tj(t+|j,Dcgc] }t/|c}|_tj2|j|_||_|r]|j8|_t;||j,|j<|_t;||j,|j<|_ d|_!|jEycc}wr)#r2r3rQdecoder_layerdroprrrrmax_target_positionsrrrrrrrrrr+rrrdecoder_layersrr rr rrrrrrr r )r4rrrr{r5s r'r3zMvpDecoder.__init__sq  ~~ 11!..$*$B$B!8>8N8N499V^^4TW  # ,D  " V->->PTP`P` aD <  * * NN mmeFLaLaFb$c_V%<$cd #%<<#? $ !'!5!5D $-%%..%D ! &/%%..&D " ',# '%dsGc|jSrrrs r'rzMvpDecoder.get_input_embeddingsrr)c||_yrrrs r'rzMvpDecoder.set_input_embeddingsrr)rrfrrrcross_attn_head_maskpast_key_valuesrrrirrrjc j| | n|jj} | | n|jj} | | n|jj} | | n|jj} | | t d|$|} |j }|jd|d}n-| |jdd}|dddddf} n t d||ddj dnd}||j||jz}t||||}||t||j|d}|j| |}||z}|j|}t j"j%||j$|j&}|j(rZt+j,|j.j1|j2}|j5|}|j7|}|j8r%|j&r| rt:j=d d } | rd nd}| rd nd}| r|d nd}| rd nd}t?||gd d gD]j\}}| |jdtA|jBk7s3t d|dtA|jBd|jddtE|jBD](\}}| r||fz }|j&r%t+jFg}||jHkr@|||nd}|j8rc|j&rW|jK|jL|||||||nd|||nd|j(r|nd|j(r|ndd| | }nC||||||||nd|||nd|j(r|nd|j(r|nd|| |  }|d}| r ||| rdndfz }| s||dfz }| ||dfz }+| r||fz }| r|nd} | stOd|| |||fDStQ|| |||S)a  Args: input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. Padding will be ignored by default should you provide it. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) attention_mask (`torch.Tensor` of shape `(batch_size, sequence_length)`, *optional*): Mask to avoid performing attention on padding token indices. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) encoder_hidden_states (`torch.FloatTensor` of shape `(batch_size, encoder_sequence_length, hidden_size)`, *optional*): Sequence of hidden-states at the output of the last layer of the encoder. Used in the cross-attention of the decoder. encoder_attention_mask (`torch.LongTensor` of shape `(batch_size, encoder_sequence_length)`, *optional*): Mask to avoid performing cross-attention on padding tokens indices of encoder input_ids. Mask values selected in `[0, 1]`: - 1 for tokens that are **not masked**, - 0 for tokens that are **masked**. [What are attention masks?](../glossary#attention-mask) head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*): Mask to nullify selected heads of the attention modules. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. cross_attn_head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*): Mask to nullify selected heads of the cross-attention modules in the decoder to avoid performing cross-attention on hidden heads. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. past_key_values (`tuple(tuple(torch.FloatTensor))`, *optional*, returned when `use_cache=True` is passed or when `config.use_cache=True`): Tuple of `tuple(torch.FloatTensor)` of length `config.n_layers`, with each tuple having 2 tensors of shape `(batch_size, num_heads, sequence_length, embed_size_per_head)`) and 2 additional tensors of shape `(batch_size, num_heads, encoder_sequence_length, embed_size_per_head)`. Contains pre-computed hidden-states (key and values in the self-attention blocks and in the cross-attention blocks) that can be used (see `past_key_values` input) to speed up sequential decoding. If `past_key_values` are used, the user can optionally input only the last `decoder_input_ids` (those that don't have their past key value states given to this model) of shape `(batch_size, 1)` instead of all `decoder_input_ids` of shape `(batch_size, sequence_length)`. inputs_embeds (`torch.FloatTensor` of shape `(batch_size, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. output_attentions (`bool`, *optional*): Whether or not to return the attentions tensors of all attention layers. See `attentions` under returned tensors for more detail. output_hidden_states (`bool`, *optional*): Whether or not to return the hidden states of all layers. See `hidden_states` under returned tensors for more detail. return_dict (`bool`, *optional*): Whether or not to return a [`~utils.ModelOutput`] instead of a plain tuple. NzTYou cannot specify both decoder_input_ids and decoder_inputs_embeds at the same timer zEYou have to specify either decoder_input_ids or decoder_inputs_embedsrr0)rzrnzZ`use_cache=True` is incompatible with gradient checkpointing. Setting `use_cache=False`...FrrrBzThe `z` should be specified for rr) rfrrrgrrrrerirr rc3$K|]}|| ywrrr s r'r#z%MvpDecoder.forward..s=s)r%rCrcr&cross_attentions))rrirrr'r$r"r_rqrrrrr9rr rrvrQrprr;r<rrtr:rrr logger warning_oncezipr(r r)r*rr+r,r-r)!r4rrfrrrrBrCrrrirrr.r/r6 positionsrcrrrall_hidden_statesall_self_attnsall_cross_attentionsnext_decoder_cache attn_mask mask_namer3 decoder_layerr6rer7 next_caches! r'rAzMvpDecoder.forwards`2C1N-TXT_T_TqTq$8$D $++JjJj "+!6IDKKst t  "E#//K!r;r?;I  &',,.s3K!!Q(+Ede eDSC^!3A!6!>#A&3t{{+;<$ {*DSEUDVW%NN,Q/03 #,DKK"83 @ C#!m%55!}}&+jjn#&75D5P_S1VZN**t}} $ A A!**!")*&/&;IcN1E1Q(-W[-1__$S)$.2oo%c*4% ! !.!#1*?+A7@7LYs^RV5I5U,S1[_?C&6s&;TXAE'8'=VZ#1&7'! *!,M"}:KQQR'S&UU" =#3"55(4(]1-=,??(g3 @l -!1 1 +4'$ '5FXlm  9+&+%1   r)r8) NNNNNNNNNNNN)rErFrGrHrrrrrr3rrr;r9rJrrrrrrArKrLs@r'r;r;sqlq&&/7 /E&ZbcgZh&P!" 1515=A=A,07;=A59$(,0/3&*_ E,,-_ !._  ((9(9: _ !))9)9 : _ ELL) _ 'u||4_ "$u'8'8"9:_   1 12_ D>_ $D>_ 'tn_ d^_  u?? @_ r)r;c$eZdZdgZddgZdeffd ZdZdZdZ d Z d Z e dd e ejd e ej d e ejde ejde ej de ej de ej de eej$de eej$de ej$de ej$de ede ede ede edeeeff dZxZS)MvpModelfinal_logits_biasencoder.embed_tokens.weightdecoder.embed_tokens.weightrczt|||j|j}}|j|_t j ||j||_t||j|j|_ t||j|j|_ |jyr)r2r3rrrrrrsharedrencoderr;decoderr )r4rrrr5s r'r3zMvpModel.__init__s  "("5"5v7H7HZ  ++ll:v~~{K !&$++v7H7HI !&$++v7H7HI  r)c|jSr)rYrs r'rzMvpModel.get_input_embeddingss {{r)c~||_|j|j_|j|j_yr)rYrZrr[rs r'rzMvpModel.set_input_embeddingss) $(KK !$(KK !r)c|jSr)rZrs r' get_encoderzMvpModel.get_encoder ||r)c|jSrr[rs r' get_decoderzMvpModel.get_decoderr`r)c*|jsJd|jd|jjjd|jjjd|jj jdy)NzHIf you want to use lightweight tuning, make sure that `use_prompt=True`.FT)rrequires_grad_rZrr[rrs r'set_lightweight_tuningzMvpModel.set_lightweight_tuningsjj jj E" %%44T: %%44T: &&55d;r)rrfdecoder_input_idsdecoder_attention_maskrdecoder_head_maskrBencoder_outputsrCrdecoder_inputs_embedsrrirrrjc:|D| B| tdt||jj|jj}| | n|jj } ||n|jj }| | n|jj} ||n|jj}||j|||| | ||}nI|rGt|ts7t|dt|dkDr|dndt|dkDr|dnd}|j|||d|||| | | | || }|s||zSt|j|j |j"|j$|j&|j|j"|j$ S) a" decoder_input_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Indices of decoder input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are decoder input IDs?](../glossary#decoder-input-ids) Mvp uses the `eos_token_id` as the starting token for `decoder_input_ids` generation. If `past_key_values` is used, optionally only the last `decoder_input_ids` have to be input (see `past_key_values`). For translation and summarization training, `decoder_input_ids` should be provided. If no `decoder_input_ids` is provided, the model will create this tensor by shifting the `input_ids` to the right for denoising pre-training following the paper. decoder_attention_mask (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Default behavior: generate a tensor that ignores pad tokens in `decoder_input_ids`. Causal mask will also be used by default. If you want to change padding behavior, you should read [`modeling_mvp._prepare_decoder_attention_mask`] and modify to your needs. See diagram 1 in [the paper](https://arxiv.org/abs/1910.13461) for more information on the default strategy. cross_attn_head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*): Mask to nullify selected heads of the cross-attention modules in the decoder. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. NzIf no `decoder_input_ids` or `decoder_inputs_embeds` are passed, `input_ids` cannot be `None`. Please pass either `input_ids` or `decoder_input_ids` or `decoder_inputs_embeds`.)rrfrrrirrrrr0r$ rrfrrrrBrCrrrirr)r%rCdecoder_hidden_statesdecoder_attentionsrFencoder_last_hidden_staterencoder_attentions)r$r(rrrrirrr'rZrrr(r[rr%rCrcr&rF)r4rrfrgrhrrirBrjrCrrkrrirrdecoder_outputss r'rAzMvpModel.forwardsd  $)>)F  U !34;;33T[[5W5W! 2C1N-TXT_T_TqTq$8$D $++JjJj "+!6IDKK&79VW y 0 <15158<=A,0487;=A=A59=A$(,0/3&*!r E,,-r !.r $E$4$45 r !))9)9 : r ELL) r $ELL1r 'u||4r "$u'8'8"9:r "$u'8'8"9:r   1 12r  ((9(9:r D>r $D>r 'tnr d^!r " u(( )#r r r)rTze The MVP Model with a language modeling head. Can be used for various text generation tasks. ) custom_introc&eZdZgdZdeffd ZdZdZ d#dede ed e d e jffd Z ded dfd Zd ZdZdZe d$de ej(de ej*de ej(de ej(de ej*de ej*de ej*de eej.de eej.de ej.de ej.de ej(de e de e de e de e d eeeff"d Zdej*fd!Zed"ZxZS)%MvpForConditionalGeneration)rVrWlm_head.weightrcxt||t||_|j dt j d|jjjftj|j|jjjd|_ |jy)NrUrFrU)r2r3rTrregister_bufferr;rsrYr,rrXrlm_headr rs r'r3z$MvpForConditionalGeneration.__init__s  f%  0%++q$**BSBSBbBb>c2deyy1B1B1Q1QX]^  r)c6|jjSr)rr_rs r'r_z'MvpForConditionalGeneration.get_encoderzz%%''r)c6|jjSr)rrcrs r'rcz'MvpForConditionalGeneration.get_decoderr~r)Nnew_num_tokenspad_to_multiple_of mean_resizingrjcLt||||}|j||Sr)r2resize_token_embeddings_resize_final_logits_bias)r4rrrnew_embeddingsr5s r'rz3MvpForConditionalGeneration.resize_token_embeddingss.8I[]jk &&~6r)c6|jjd}||kr|jddd|f}nStjd||z f|jj}tj |j|gd}|j d|y)Nr rrrlrU)rUr"r;rsr:rrr{)r4rold_num_tokensnew_bias extra_biass r'rz5MvpForConditionalGeneration._resize_final_logits_biass//55b9 ^ +--a..@AHa.)H%IRVRhRhRoRopJyy$"8"8*!E1MH 0(;r)c|jSrr|rs r'get_output_embeddingsz1MvpForConditionalGeneration.get_output_embeddingsr`r)c||_yrrr4rs r'set_output_embeddingsz1MvpForConditionalGeneration.set_output_embeddings % r)cn|jj|jjdyr8rrfr|rers r'rfz2MvpForConditionalGeneration.set_lightweight_tuning$ ))+ ##E*r)rrfrgrhrrirBrjrCrrklabelsrrirrc||n|jj}| R| rtjdd} |7| 5t | |jj |jj }|j||||||||| | | | |||}|j|d|jz}d}| Ft}||jd|jj| jd}|s|f|ddz}||f|zS|St|||j|j|j |j"|j$|j&|j( S) a decoder_input_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Indices of decoder input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are decoder input IDs?](../glossary#decoder-input-ids) Mvp uses the `eos_token_id` as the starting token for `decoder_input_ids` generation. If `past_key_values` is used, optionally only the last `decoder_input_ids` have to be input (see `past_key_values`). For translation and summarization training, `decoder_input_ids` should be provided. If no `decoder_input_ids` is provided, the model will create this tensor by shifting the `input_ids` to the right for denoising pre-training following the paper. decoder_attention_mask (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Default behavior: generate a tensor that ignores pad tokens in `decoder_input_ids`. Causal mask will also be used by default. If you want to change padding behavior, you should read [`modeling_mvp._prepare_decoder_attention_mask`] and modify to your needs. See diagram 1 in [the paper](https://arxiv.org/abs/1910.13461) for more information on the default strategy. cross_attn_head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*): Mask to nullify selected heads of the cross-attention modules in the decoder. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example of summarization: Fine-tuning a model ```python >>> import torch >>> from transformers import AutoTokenizer, MvpForConditionalGeneration >>> tokenizer = AutoTokenizer.from_pretrained("RUCAIBox/mvp") >>> model = MvpForConditionalGeneration.from_pretrained("RUCAIBox/mvp") >>> inputs = tokenizer( ... "Summarize: You may want to stick it to your boss and leave your job, but don't do it if these are your reasons.", ... return_tensors="pt", ... ) >>> labels = tokenizer("Bad Reasons To Quit Your Job", return_tensors="pt")["input_ids"] >>> loss = model(**inputs, labels=labels).loss >>> loss.backward() ``` Inference after the model fine-tuned ```python >>> with torch.no_grad(): ... generated_ids = model.generate(**inputs) >>> generated_text = tokenizer.batch_decode(generated_ids, skip_special_tokens=True) ``` NzJThe `use_cache` argument is changed to `False` since `labels` is provided.F)rfrgrjrhrrirBrCrrkrrirrrr r losslogitsrCrnrorFrprrq)rr'rGwarningr(rrrr|rUr r_rrrCrnrorFrprrq)r4rrfrgrhrrirBrjrCrrkrrrirrr lm_logitsmasked_lm_lossloss_fctoutputs r'rAz#MvpForConditionalGeneration.forwardsb&1%.Ds.rU_j--aZ=N=N1OPr58r0r-rCrreordered_past layer_pasts ` r'_reorder_cachez*MvpForConditionalGeneration._reorder_cache>sT) J rcmnpopcqrrQR.! N  r))NTNNNNNNNNNNNNNNNN) rErFrGrurr3r_rcrIrrrrrrrrrfrr;r9rJrrrrrrAr staticmethodrrKrLs@r'rxrxs@ jy((dh!7?}\` <<<&+15158<=A,0487;=A=A59=A-1$(,0/3&*#A E,,-A !.A $E$4$45 A !))9)9 : A ELL) A $ELL1A 'u||4A "$u'8'8"9:A "$u'8'8"9:A   1 12A  ((9(9:A ))*A D>A $D>A 'tn!A "d^#A $ uo% &%A A FhELLhr)rxz Mvp model with a sequence classification/head on top (a linear layer on top of the pooled output) e.g. for GLUE tasks. c$eZdZddgZdeffd ZdZe ddee jdee jdee jd ee jd ee jd ee jd ee jd ee e jdee jdee jdee jdeedeedeedeedeeeff dZxZS)MvpForSequenceClassificationrVrWrc t||fi|t||_t |j |j |j |j|_|jyr) r2r3rTrrr num_labelsclassifier_dropoutclassification_headr )r4rkwargsr5s r'r3z%MvpForSequenceClassification.__init__SsZ *6*f% #8 NN NN     % % $   r)cn|jj|jjdyr8)rrfrrers r'rfz3MvpForSequenceClassification.set_lightweight_tuning`s& ))+   //6r)rrfrgrhrrirBrjrrkrrrirrrjc||n|jj}| d} |$| "td|jj|j ||||||||| | | | ||}|d}|j |jjj|j}ttj|jddkDr td||ddfj|j!dd|j!ddddddf}|j#|}d}| |jj$|jj&dk(rd |j_nv|jj&dkDrL| j(tj*k(s| j(tj,k(rd |j_nd |j_|jj$d k(rSt/}|jj&dk(r&||j1| j1}n||| }n|jj$d k(rGt3}||jd|jj&| jd}n,|jj$d k(rt5}||| }|s|f|ddz}||f|zS|St7|||j8|j:|j<|j>|j@|jB|jD S) a decoder_input_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Indices of decoder input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are decoder input IDs?](../glossary#decoder-input-ids) Mvp uses the `eos_token_id` as the starting token for `decoder_input_ids` generation. If `past_key_values` is used, optionally only the last `decoder_input_ids` have to be input (see `past_key_values`). For translation and summarization training, `decoder_input_ids` should be provided. If no `decoder_input_ids` is provided, the model will create this tensor by shifting the `input_ids` to the right for denoising pre-training following the paper. decoder_attention_mask (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Default behavior: generate a tensor that ignores pad tokens in `decoder_input_ids`. Causal mask will also be used by default. If you want to change padding behavior, you should read [`modeling_mvp._prepare_decoder_attention_mask`] and modify to your needs. See diagram 1 in [the paper](https://arxiv.org/abs/1910.13461) for more information on the default strategy. cross_attn_head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*): Mask to nullify selected heads of the cross-attention modules in the decoder. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*): Labels for computing the sequence classification/regression loss. Indices should be in `[0, ..., config.num_labels - 1]`. If `config.num_labels > 1` a classification loss is computed (Cross-Entropy). Example of single-label classification: Fine-tuning a model on `num_labels` classes ```python >>> import torch >>> from transformers import AutoTokenizer, MvpForSequenceClassification >>> num_labels = 2 # for example, this is a binary classification task >>> tokenizer = AutoTokenizer.from_pretrained("RUCAIBox/mvp") >>> model = MvpForSequenceClassification.from_pretrained("RUCAIBox/mvp", num_labels=num_labels) >>> inputs = tokenizer("Classify: Hello, my dog is cute", return_tensors="pt") >>> labels = torch.tensor(1) # the real label for inputs >>> loss = model(**inputs, labels=labels).loss >>> loss.backward() ``` Inference after the model fine-tuned ```python >>> with torch.no_grad(): ... logits = model(**inputs).logits >>> predicted_class_id = logits.argmax() ``` NFz8Passing input embeddings is currently not supported for rfrgrhrrirBrjrrkrrirrrrz7All examples must have the same number of tokens.r regressionsingle_label_classificationmulti_label_classificationr)#rr'NotImplementedErrorr5rEreq eos_token_idrtr:r(r;unique_consecutivesumr$r_rqr problem_typerr9r=rIr squeezer rrrCrnrorFrprrq)r4rrfrgrhrrirBrjrrkrrrirrrrceos_masksentence_representationrrrrs r'rAz$MvpForSequenceClassification.forwardds Z&1%>KbKbJcd ** )/#9/!5+'"7/!5#    << 8 89<<]=Q=QR u'' Q8 9A =VW W"/! "<"A"A-BTBTUVBWY[]j]o]opr]s"t r1H# ))*AB  {{''/;;))Q./;DKK,[[++a/V\\UZZ5OSYS_S_chclclSl/LDKK,/KDKK,{{''<7"9;;))Q.#FNN$4fnn6FGD#FF3D))-JJ+- B 0F0F GUWY))-II,./Y,F)-)9TGf$ Ev E.#33")"?"?&99$55&-&G&G")"?"?&99  r)rs)rErFrGrurr3rfrrr;r9rJrrrrrrrArKrLs@r'rrJs89VW y 715158<=A,0487;=A59=A-1$(,0/3&*!T E,,-T !.T $E$4$45 T !))9)9 : T ELL) T $ELL1T 'u||4T "$u'8'8"9:T   1 12T  ((9(9:T ))*T D>T $D>T 'tnT d^!T " u55 6#T T r)rc&eZdZddgZfdZdZe ddeejdeejdeejdeejd eejd eejd eejd ee ejd eejdeejdeejdeejdee dee dee dee deeeff"dZxZS)MvpForQuestionAnsweringrVrWct||d|_|j|_t||_t j |j|j|_|jyr/) r2r3rrTrrrX hidden_size qa_outputsr rs r'r3z MvpForQuestionAnswering.__init__s[   ++f% ))F$6$68I8IJ r)cn|jj|jjdyr8)rrfrrers r'rfz.MvpForQuestionAnswering.set_lightweight_tuning s$ ))+ &&u-r)rrfrgrhrrirBrjstart_positions end_positionsrrkrrirrrjc||n|jj}| | d} |j||||||||| | | |||}|d}|j|}|j dd\}}|j dj }|j dj }d}| | t| jdkDr| j d} t| jdkDr| j d} |jd}| jd|} | jd|} t|}||| }||| }||zd z }|s||f|ddz}||f|zS|St||||j|j|j|j|j |j"|j$ S) aX decoder_input_ids (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Indices of decoder input sequence tokens in the vocabulary. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are decoder input IDs?](../glossary#decoder-input-ids) Mvp uses the `eos_token_id` as the starting token for `decoder_input_ids` generation. If `past_key_values` is used, optionally only the last `decoder_input_ids` have to be input (see `past_key_values`). For translation and summarization training, `decoder_input_ids` should be provided. If no `decoder_input_ids` is provided, the model will create this tensor by shifting the `input_ids` to the right for denoising pre-training following the paper. decoder_attention_mask (`torch.LongTensor` of shape `(batch_size, target_sequence_length)`, *optional*): Default behavior: generate a tensor that ignores pad tokens in `decoder_input_ids`. Causal mask will also be used by default. If you want to change padding behavior, you should read [`modeling_mvp._prepare_decoder_attention_mask`] and modify to your needs. See diagram 1 in [the paper](https://arxiv.org/abs/1910.13461) for more information on the default strategy. cross_attn_head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*): Mask to nullify selected heads of the cross-attention modules in the decoder. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. Example: Fine-tuning a model for extrative question answering, and our model also supports generative question answering using `BartForConditionalGeneration` ```python >>> import torch >>> from transformers import AutoTokenizer, MvpForQuestionAnswering >>> tokenizer = AutoTokenizer.from_pretrained("RUCAIBox/mvp") >>> model = MvpForQuestionAnswering.from_pretrained("RUCAIBox/mvp") >>> inputs = tokenizer( ... "Answer the following question: Who was Jim Henson? [SEP] Jim Henson was a nice puppet", ... return_tensors="pt", ... ) >>> target_start_index = torch.tensor([18]) >>> target_end_index = torch.tensor([19]) >>> loss = model(**inputs, start_positions=target_start_index, end_positions=target_end_index).loss >>> loss.backward() ``` Inference after the model fine-tuned ```python >>> with torch.no_grad(): ... outputs = model(**inputs) >>> answer_start_index = outputs.start_logits.argmax() >>> answer_end_index = outputs.end_logits.argmax() >>> predict_answer_tokens = inputs.input_ids[0, answer_start_index : answer_end_index + 1] >>> predict_answer = tokenizer.decode(predict_answer_tokens) ``` NFrrrr rl) ignore_indexr0) r start_logits end_logitsrCrnrorFrprrq)rr'rrrrrar(rqrr rrCrnrorFrprrq)r4rrfrgrhrrirBrjrrrrkrrirrrsequence_outputrrr total_loss ignored_indexr start_lossend_lossrs r'rAzMvpForQuestionAnswering.forwardsf&1%Q $D>Q 'tn!Q "d^#Q $ u99 :%Q Q r)rc(eZdZdZfdZdZxZS)MvpDecoderWrapperz This wrapper class is a helper class to correctly load pretrained checkpoints when the causal language model is used in combination with the [`EncoderDecoderModel`] framework. cDt||t||_yr)r2r3r;r[rs r'r3zMvpDecoderWrapper.__init__s  !&) r)c&|j|i|Srrb)r4argsrs r'rAzMvpDecoderWrapper.forwardst||T,V,,r))rErFrGrHr3rArKrLs@r'rrs *-r)rc eZdZdgZfdZdZdZdZdZdZ dZ d Z e dd e ejd e ej d e ej"d e ej"de ej de ej de eej"de ej"de ejde ede ede ede edeeeffdZedZxZS)MvpForCausalLMryc tj|}d|_d|_t||t ||_tj|j|jd|_ |jy)NTFrU)copydeepcopyrRis_encoder_decoderr2r3rrrrXrrr|r rs r'r3zMvpForCausalLM.__init__sfv& $)!  &v. yy!3!3V5F5FUS  r)cB|jjjSrrr[rrs r'rz#MvpForCausalLM.get_input_embeddingsszz!!...r)c:||jj_yrrrs r'rz#MvpForCausalLM.set_input_embeddingss*/ 'r)c|jSrrrs r'rz$MvpForCausalLM.get_output_embeddingsr`r)c||_yrrrs r'rz$MvpForCausalLM.set_output_embeddingsrr)c&||j_yrrr[)r4r[s r' set_decoderzMvpForCausalLM.set_decoders$ r)c.|jjSrrrs r'rczMvpForCausalLM.get_decoderszz!!!r)cn|jj|jjdyr8rrs r'rfz%MvpForCausalLM.set_lightweight_tuningrr)rrfrrrrBrCrrrrirrrjcD| | n|jj} | | n|jj} | | n|jj} |jj ||||||||| | | |  }|j |d}d}| Ft}||jd|jj| jd}| s|f|ddz}||f|zS|St|||j|j|j|jS)a cross_attn_head_mask (`torch.Tensor` of shape `(decoder_layers, decoder_attention_heads)`, *optional*): Mask to nullify selected heads of the cross-attention modules. Mask values selected in `[0, 1]`: - 1 indicates the head is **not masked**, - 0 indicates the head is **masked**. labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*): Labels for computing the masked language modeling loss. Indices should either be in `[0, ..., config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`. Example: ```python >>> from transformers import AutoTokenizer, MvpForCausalLM >>> tokenizer = AutoTokenizer.from_pretrained("RUCAIBox/mvp") >>> model = MvpForCausalLM.from_pretrained("RUCAIBox/mvp", add_cross_attention=False) >>> inputs = tokenizer("Hello, my dog is cute", return_tensors="pt") >>> outputs = model(**inputs) >>> logits = outputs.logits >>> list(logits.shape) [1, 8, 50267] ```Nrmrr r)rrrCrcr&rF)rrirr'rr[r|r r_rrrCrcr&rF)r4rrfrrrrBrCrrrrirrrrrrrs r'rAzMvpForCausalLM.forwardsFX2C1N-TXT_T_TqTq$8$D $++JjJj &1%.4s.nU_j--aZ=N=N1OPnrrrs ` r'rzMvpForCausalLM._reorder_cache/s=) J ncmnn N r)) NNNNNNNNNNNNN)rErFrGrur3rrrrrrcrfrrr;r9rJrrrrrrrArrrKrLs@r'rrs*+ /0&%"+1515=A>B,07;=A59-1$(,0/3&*S E,,-S !.S  ((9(9: S !)):): ; S ELL) S 'u||4S "$u'8'8"9:S   1 12S ))*S D>S $D>S 'tnS d^S  u77 8S S jr)r)rrxrrrTr)=rHrrtypingrrrrr;torch.utils.checkpointrtorch.nnrr r activationsr generationr modeling_attn_mask_utilsrrmodeling_outputsrrrrrrrmodeling_utilsrutilsrrconfiguration_mvpr get_loggerrErGrJrIr(rr+ModulerNrrrrrrr;rTrxrrrr__all__rr)r'rs // AA!).,(   H % %,, c [^ ";BLL;2XB299XBvEbiiEPzbiiz|BII0 26A #A HW #W tZ !Z Z z |"4o|  |~ i #5i  i Xe 0e e R -* -B'BJ r)