Uh`c(dZddlmZddlmZmZmZddlZddlmZm Z ddl m Z ddl m Z dd lmZmZmZd d lmZd d lmZej,eZeGddeZeGddeZeGddeZeGdde ZGddeZGddeZGddeZGddeZ GddeZ!ed !Gd"d#eZ"ed$!Gd%d&e Z#ed'!Gd(d)e!Z$gd*Z%y)+z5PyTorch DPR model for Open Domain Question Answering.) dataclass)OptionalTupleUnionN)Tensornn)BaseModelOutputWithPooling)PreTrainedModel) ModelOutputauto_docstringlogging) BertModel) DPRConfigceZdZUdZej ed<dZee ej dfed<dZ ee ej dfed<y)DPRContextEncoderOutputac Class for outputs of [`DPRQuestionEncoder`]. Args: pooler_output (`torch.FloatTensor` of shape `(batch_size, embeddings_size)`): The DPR encoder outputs the *pooler_output* that corresponds to the context representation. Last layer hidden-state of the first token of the sequence (classification token) further processed by a Linear layer. This output is to be used to embed contexts for nearest neighbors queries with questions embeddings. hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the initial embedding outputs. attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. pooler_outputN. hidden_states attentions __name__ __module__ __qualname____doc__torch FloatTensor__annotations__rrrrv/var/www/catia.catastroantioquia-mas.com/valormas/lib/python3.12/site-packages/transformers/models/dpr/modeling_dpr.pyrr*S*$$$=AM8E%"3"3S"89:A:>Ju00#567>r!rceZdZUdZej ed<dZee ej dfed<dZ ee ej dfed<y)DPRQuestionEncoderOutputac Class for outputs of [`DPRQuestionEncoder`]. Args: pooler_output (`torch.FloatTensor` of shape `(batch_size, embeddings_size)`): The DPR encoder outputs the *pooler_output* that corresponds to the question representation. Last layer hidden-state of the first token of the sequence (classification token) further processed by a Linear layer. This output is to be used to embed questions for nearest neighbors queries with context embeddings. hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the initial embedding outputs. attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. rN.rrrr r!r"r%r%Fr#r!r%ceZdZUdZej ed<dZeej ed<dZ eej ed<dZ ee ej dfed<dZ ee ej dfed<y) DPRReaderOutputa Class for outputs of [`DPRQuestionEncoder`]. Args: start_logits (`torch.FloatTensor` of shape `(n_passages, sequence_length)`): Logits of the start index of the span for each passage. end_logits (`torch.FloatTensor` of shape `(n_passages, sequence_length)`): Logits of the end index of the span for each passage. relevance_logits (`torch.FloatTensor` of shape `(n_passages, )`): Outputs of the QA classifier of the DPRReader that corresponds to the scores of each passage to answer the question, compared to all the other passages. hidden_states (`tuple(torch.FloatTensor)`, *optional*, returned when `output_hidden_states=True` is passed or when `config.output_hidden_states=True`): Tuple of `torch.FloatTensor` (one for the output of the embeddings + one for the output of each layer) of shape `(batch_size, sequence_length, hidden_size)`. Hidden-states of the model at the output of each layer plus the initial embedding outputs. attentions (`tuple(torch.FloatTensor)`, *optional*, returned when `output_attentions=True` is passed or when `config.output_attentions=True`): Tuple of `torch.FloatTensor` (one for each layer) of shape `(batch_size, num_heads, sequence_length, sequence_length)`. Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. start_logitsN end_logitsrelevance_logits.rr) rrrrrrrr)rr*rrrr r!r"r'r'bs~0###.2J**+248hu0018=AM8E%"3"3S"89:A:>Ju00#567>r!r'ceZdZdZdZy)DPRPreTrainedModelTct|tjrm|jjj d|j j|j%|jjjyyt|tjrz|jjj d|j j|j2|jj|jjyyt|tjrJ|jjj|jjjdyy)zInitialize the weightsg)meanstdNg?) isinstancerLinearweightdatanormal_configinitializer_rangebiaszero_ Embedding padding_idx LayerNormfill_)selfmodules r" _init_weightsz DPRPreTrainedModel._init_weightss fbii ( MM   & &CT[[5R5R & S{{&   &&('  - MM   & &CT[[5R5R & S!!- ""6#5#56<<>.  - KK   " " $ MM   $ $S ).r!N)rrr_supports_sdpar?r r!r"r,r,s N*r!r,ceZdZdZdeffd Z ddedeedeedeeded ed ed e e e ed fffd Z e d efdZxZS) DPREncoder bert_modelr5ct||t|d|_|jjj dkr t d|j|_|jdkDrCtj|jjj |j|_ |jy)NF)add_pooling_layerrz!Encoder hidden_size can't be zero) super__init__rrCr5 hidden_size ValueErrorprojection_dimrr1 encode_proj post_initr=r5 __class__s r"rGzDPREncoder.__init__s  #FeD ?? ! ! - - 2@A A$33    "!yy)?)?)K)KVMbMbcD  r! input_idsattention_masktoken_type_ids inputs_embedsoutput_attentionsoutput_hidden_states return_dictreturn.c |j|||||||}|d} | dddddf} |jdkDr|j| } |s | | f|ddzSt| | |j|j S)NrOrPrQrRrSrTrUrr)last_hidden_staterrr)rCrJrKr rr) r=rOrPrQrRrSrTrUoutputssequence_output pooled_outputs r"forwardzDPREncoder.forwards//))'/!5#" "!*'1a0    " ,,];M#]3gabkA A)-'!//))   r!c|jdkDr|jjS|jjj S)Nr)rJrK out_featuresrCr5rH)r=s r"embeddings_sizezDPREncoder.embeddings_sizes8    "##00 0%%111r!)NNNFFF)rrrbase_model_prefixrrGrrboolrr rr]propertyintr` __classcell__rNs@r"rBrBs$ y ,0+/*."'%*!! ! !(! !( !  ' !  ! #! !  )5+== >! F222r!rBcpeZdZdZdeffd Z d dededeededed ed e e e ed fffd Z xZ S)DPRSpanPredictorencoderr5c t||t||_t j |jj d|_t j |jj d|_|jy)Nrr) rFrGrBrirr1r` qa_outputs qa_classifierrLrMs r"rGzDPRSpanPredictor.__init__s_  !&) ))DLL$@$@!DYYt||'C'CQG r!rOrPrRrSrTrUrV.cV||jn|jdd\}}|j||||||} | d} |j| } | jdd\} } | j dj } | j dj } |j | dddddf}| j||} | j||} |j|}|s | | |f| ddzSt| | || j| jS)Nr)rPrRrSrTrUrr)dim)r(r)r*rr) sizerirksplitsqueeze contiguousrlviewr'rr)r=rOrPrRrSrTrU n_passagessequence_lengthrZr[logitsr(r)r*s r"r]zDPRSpanPredictor.forwardsN;D:Oinn&6UbUgUgUijlklUm# O,, )'/!5#  "!*1#)<<r<#: j#++B/::< ''+668 --oaAg.FG$((_E __ZA +00< *.>?'!"+M M%!-!//))   r!)NFFF)rrrrarrGrrrbrr'rr]rerfs@r"rhrhs!y+/"'%*!+ + +  ' +  + # + +  fck 22 3+ r!rhceZdZdZeZdZdZy)DPRPretrainedContextEncoder An abstract class to handle weights initialization and a simple interface for downloading and loading pretrained models. N ctx_encoderrrrrr config_classload_tf_weightsrar r!r"ryrys LO%r!ryceZdZdZeZdZdZy)DPRPretrainedQuestionEncoderrzNquestion_encoderr|r r!r"rrs LO*r!rceZdZdZeZdZdZy)DPRPretrainedReaderrzNspan_predictorr|r r!r"rr$s LO(r!rzf The bare DPRContextEncoder transformer outputting pooler outputs as context representations. ) custom_introceZdZdeffd Ze d deedeedeedeedeedeed eed e e e ed fffd Z xZ S)DPRContextEncoderr5crt||||_t||_|j yN)rFrGr5rBr{rLrMs r"rGzDPRContextEncoder.__init__:s.   %f- r!rOrPrQrRrSrTrUrV.c ||n|jj}||n|jj}||n|jj}| | t d||j }n!||j dd}n t d| |j n |j } |2|tj|| n||jjk7}|&tj|tj| }|j|||||||} |s| ddSt| j| j| j S) aS input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. To match pretraining, DPR input sequence should be formatted with [CLS] and [SEP] tokens as follows: (a) For sequence pairs (for a pair title+text for example): ``` tokens: [CLS] is this jack ##son ##ville ? [SEP] no it is not . [SEP] token_type_ids: 0 0 0 0 0 0 0 0 1 1 1 1 1 1 ``` (b) For single sequences (for a question for example): ``` tokens: [CLS] the dog is hairy . [SEP] token_type_ids: 0 0 0 0 0 0 0 ``` DPR is a model with absolute position embeddings so it's usually advised to pad the inputs on the right rather than the left. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) Examples: ```python >>> from transformers import DPRContextEncoder, DPRContextEncoderTokenizer >>> tokenizer = DPRContextEncoderTokenizer.from_pretrained("facebook/dpr-ctx_encoder-single-nq-base") >>> model = DPRContextEncoder.from_pretrained("facebook/dpr-ctx_encoder-single-nq-base") >>> input_ids = tokenizer("Hello, is my dog cute ?", return_tensors="pt")["input_ids"] >>> embeddings = model(input_ids).pooler_output ```NDYou cannot specify both input_ids and inputs_embeds at the same timern5You have to specify either input_ids or inputs_embedsdevicedtyperrXrrrr)r5rSrTuse_return_dictrIrprrones pad_token_idzeroslongr{rrrr r=rOrPrQrRrSrTrU input_shaperrZs r"r]zDPRContextEncoder.forwardAspb2C1N-TXT_T_TqTq$8$D $++JjJj &1%cd d  "#..*K  &',,.s3KTU U%.%:!! @T@T  !$ ;v64;;#;#;;   !"[[EJJvVN""))'/!5## 12; &!//w?T?Tahasas  r!NNNNNNN)rrrrrGr rrrbrrrr]rerfs@r"rr4s y'++/+/*.,0/3&*X F#X !(X !( X  ' X $D> X 'tnX d^X  &fck(:: ;X X r!rzh The bare DPRQuestionEncoder transformer outputting pooler outputs as question representations. ceZdZdeffd Ze d deedeedeedeedeedeed eed e e e ed fffd Z xZ S)DPRQuestionEncoderr5crt||||_t||_|j yr)rFrGr5rBrrLrMs r"rGzDPRQuestionEncoder.__init__s.   *6 2 r!rOrPrQrRrSrTrUrV.c ||n|jj}||n|jj}||n|jj}| | t d|#|j |||j }n!||j dd}n t d| |jn |j} |2|tj|| n||jjk7}|&tj|tj| }|j|||||||} |s| ddSt| j| j | j" S) aj input_ids (`torch.LongTensor` of shape `(batch_size, sequence_length)`): Indices of input sequence tokens in the vocabulary. To match pretraining, DPR input sequence should be formatted with [CLS] and [SEP] tokens as follows: (a) For sequence pairs (for a pair title+text for example): ``` tokens: [CLS] is this jack ##son ##ville ? [SEP] no it is not . [SEP] token_type_ids: 0 0 0 0 0 0 0 0 1 1 1 1 1 1 ``` (b) For single sequences (for a question for example): ``` tokens: [CLS] the dog is hairy . [SEP] token_type_ids: 0 0 0 0 0 0 0 ``` DPR is a model with absolute position embeddings so it's usually advised to pad the inputs on the right rather than the left. Indices can be obtained using [`AutoTokenizer`]. See [`PreTrainedTokenizer.encode`] and [`PreTrainedTokenizer.__call__`] for details. [What are input IDs?](../glossary#input-ids) Examples: ```python >>> from transformers import DPRQuestionEncoder, DPRQuestionEncoderTokenizer >>> tokenizer = DPRQuestionEncoderTokenizer.from_pretrained("facebook/dpr-question_encoder-single-nq-base") >>> model = DPRQuestionEncoder.from_pretrained("facebook/dpr-question_encoder-single-nq-base") >>> input_ids = tokenizer("Hello, is my dog cute ?", return_tensors="pt")["input_ids"] >>> embeddings = model(input_ids).pooler_output ``` NrrnrrrrXrr)r5rSrTrrI%warn_if_padding_and_no_attention_maskrprrrrrrrr%rrrrs r"r]zDPRQuestionEncoder.forwardsb2C1N-TXT_T_TqTq$8$D $++JjJj &1%cd d  "  6 6y. Q#..*K  &',,.s3KTU U%.%:!! @T@T  !$ ;v64;;#;#;;   !"[[EJJvVN''))'/!5#( 12; '!//w?T?Tahasas  r!r)rrrrrGr rrrbrr%rr]rerfs@r"rrs y'++/+/*.,0/3&*Y F#Y !(Y !( Y  ' Y $D> Y 'tnY d^Y  'vs{);; <Y Y r!rzE The bare DPRReader transformer outputting span predictions. ceZdZdeffd Ze d deedeedeedeedeedeed e e e ed fffd Z xZ S) DPRReaderr5crt||||_t||_|j yr)rFrGr5rhrrLrMs r"rGzDPRReader.__init__ s.   .v6 r!rOrPrRrSrTrUrV.c||n|jj}||n|jj}||n|jj}| | t d|#|j |||j }n!||j dd}n t d| |jn |j}|tj||}|j||||||S)a input_ids (`Tuple[torch.LongTensor]` of shapes `(n_passages, sequence_length)`): Indices of input sequence tokens in the vocabulary. It has to be a sequence triplet with 1) the question and 2) the passages titles and 3) the passages texts To match pretraining, DPR `input_ids` sequence should be formatted with [CLS] and [SEP] with the format: `[CLS] [SEP] [SEP] ` DPR is a model with absolute position embeddings so it's usually advised to pad the inputs on the right rather than the left. Indices can be obtained using [`DPRReaderTokenizer`]. See this class documentation for more details. [What are input IDs?](../glossary#input-ids) inputs_embeds (`torch.FloatTensor` of shape `(n_passages, sequence_length, hidden_size)`, *optional*): Optionally, instead of passing `input_ids` you can choose to directly pass an embedded representation. This is useful if you want more control over how to convert `input_ids` indices into associated vectors than the model's internal embedding lookup matrix. Examples: ```python >>> from transformers import DPRReader, DPRReaderTokenizer >>> tokenizer = DPRReaderTokenizer.from_pretrained("facebook/dpr-reader-single-nq-base") >>> model = DPRReader.from_pretrained("facebook/dpr-reader-single-nq-base") >>> encoded_inputs = tokenizer( ... questions=["What is love ?"], ... titles=["Haddaway"], ... texts=["'What Is Love' is a song recorded by the artist Haddaway"], ... return_tensors="pt", ... ) >>> outputs = model(**encoded_inputs) >>> start_logits = outputs.start_logits >>> end_logits = outputs.end_logits >>> relevance_logits = outputs.relevance_logits ``` Nrrnrr)rRrSrTrU) r5rSrTrrIrrprrrr) r=rOrPrRrSrTrUrrs r"r]zDPRReader.forwards `2C1N-TXT_T_TqTq$8$D $++JjJj &1%cd d  "  6 6y. Q#..*K  &',,.s3KTU U%.%:!! @T@T  !"ZZ FCN""  '/!5# #  r!)NNNNNN)rrrrrGr rrrbrr'rr]rerfs@r"rrs y'++/*.,0/3&*K F#K !(K  ' K $D> K 'tn K d^K  fck 22 3K K r!r)rryr,rrrr)&r dataclassesrtypingrrrrrrmodeling_outputsr modeling_utilsr utilsr r rbert.modeling_bertrconfiguration_dprr get_loggerrloggerrr%r'r,rBrhryrrrrr__all__r r!r"rsz<!)) :- +(   H % ?k? ?6 ?{? ?6 ?k? ?@***(52#52p6 )6 |&"4&+#5+),)  a 3a  a H b 5b  b J T #T  T n r!