Uh <dZddlZddlZddlmZmZmZmZm Z ddl Z ddl Z ddl m Z ddl mZddlmZmZddlmZdd lmZmZdd lmZd d lmZej6eZGd de j<Ze j@jBdZ"dZ#Gdde j<Z$d1de jJde&de'de jJfdZ(Gdde j<Z)Gdde j<Z*Gdde j<Z+Gdde j<Z,d Z-d!Z.Gd"d#e j<Z/Gd$d%e j<Z0d&e j<ddfd'Z1eGd(d)eZ2eGd*d+e2Z3ed,-Gd.d/e2eZ4gd0Z5y)2zPyTorch ViTDet backbone.N)DictListOptionalTupleUnion)nn)ACT2FN)BackboneOutputBaseModelOutput)PreTrainedModel)auto_docstringlogging) BackboneMixin) VitDetConfigc`eZdZdZfdZdZdejdejfdZxZ S)VitDetEmbeddingsz This class turns `pixel_values` of shape `(batch_size, num_channels, height, width)` into the initial `hidden_states` (patch embeddings) to be consumed by a Transformer. cpt||j|j}}|j|j }}t |tjjr|n||f}t |tjjr|n||f}|d|dz|d|dzz}||_ ||_||_||_ |jr?|dz}tjtj d||j |_nd|_tj$|||||_y)Nrr) kernel_sizestride)super__init__pretrain_image_size patch_size num_channels hidden_size isinstance collectionsabcIterable image_size num_patches use_absolute_position_embeddingsr Parametertorchzerosposition_embeddingsConv2d projection) selfconfigr"rrrr# num_positions __class__s |/var/www/catia.catastroantioquia-mas.com/valormas/lib/python3.12/site-packages/transformers/models/vitdet/modeling_vitdet.pyrzVitDetEmbeddings.__init__*s !'!;!;V=N=NJ $*$7$79K9Kk #-j+//:R:R#SZZdfpYq #-j+//:R:R#SZZdfpYq !!} 1 5*Q-:VW=:XY $$(&  2 2'!OM')||EKK=RXRdRd4e'fD $'+D $))L+:^hic|r |ddddf}|jd}ttj|}||z|k7r t dt j js ||k7s||k7r]tjj|jd||djdddd||fdd }|jddddS|jd||dS) a Calculate absolute positional embeddings. If needed, resize embeddings and remove cls_token dimension for the original embeddings. Args: abs_pos_embeddings (`torch.Tensor`): Absolute positional embeddings with (1, num_position, num_channels). has_cls_token (`bool`): If true, has 1 embedding in abs_pos_embeddings for cls token. height (`int`): Height of input image tokens. width (`int`): Width of input image tokens. Returns: Absolute positional embeddings after processing with shape (1, height, width, num_channels) Nrz5Absolute position embeddings must be a square number.rr bicubicF)sizemode align_corners) shapeintmathsqrt ValueErrorr&jit is_tracingr functional interpolatereshapepermute)r+abs_pos_embeddings has_cls_tokenheightwidth num_positionr5new_abs_pos_embeddingss r/get_absolute_positionsz'VitDetEmbeddings.get_absolute_positions@s$ !3AqrE!: )//2 499\*+ $;, &TU U 99   !dfn %']]%>%>"**1dD"=EEaAqQe_# &?& "*11!Q1= =%--aC Cr0 pixel_valuesreturncz|jd}||jk7rtd|jd|d|j|}|jc|j dddd}||j |jd|jd|jdz}|j dddd}|S) NrzoMake sure that the channel dimension of the pixel values match with the one set in the configuration. Expected z but got .rr3r T)r8rr<r*r(rBrI)r+rJr embeddingss r/forwardzVitDetEmbeddings.forwardfs#))!, 4,, ,!../yaI __\2  # # /#++Aq!Q7J#d&A&A(($ 0@0@0CZEUEUVWEX'J$++Aq!Q7Jr0) __name__ __module__ __qualname____doc__rrIr&TensorrO __classcell__r.s@r/rr$s0 j,$DLELLU\\r0rcTtdt||zdz }|jd|k7rttjj |j d|jddjddd|d}|j d|jdd}n|}tj|dddft||z dz}tj|dddft||z dz}||z |dz t||z dzz}||jS) a Get relative positional embeddings according to the relative positions of query and key sizes. Args: q_size (`int`): Size of query q. k_size (`int`): Size of key k. rel_pos (`torch.Tensor`): Relative position embeddings (num_embeddings, num_channels). Returns: Extracted positional embeddings according to relative positions. r3rrr2linear)r5r6N?) r9maxr8rr?r@rArBr&arangelong)q_sizek_sizerel_pos max_rel_distrel_pos_resizedq_coordsk_coordsrelative_coordss r/ get_rel_posre|s% q3vv..23L}}Q<'--33 OOAw}}Q/ 4 < bhwkzbhwc,wkc->bhwkN)rer8rAr&einsumview)attnqueries rel_pos_h rel_pos_wr]r^queries_height queries_width keys_height keys_widthrelative_heightrelative_width batch_size_dimr_qrelative_weights r/!add_decomposed_relative_positionsrxs,%+!NM$K!.+yIO  IFN J3 //*nmS ICll#3S/JOll#3S.IO *nm[*U !Q1d* + , !Q4* + , d:~ 5{Z7OP  Kr0c,eZdZdZdfd ZddZxZS)VitDetAttentionz=Multi-head Attention block with relative position embeddings.c t||j}|j}||_||z}|dz|_t j||dz|j|_ t j|||_ |j|_ |jrot jtjd|dzdz ||_t jtjd|dzdz ||_yy)z Args: config (`VitDetConfig`): Model configuration. input_size (`Tuple[int]`, *optional*): Input resolution, only required in case relative position embeddings are added. gr biasr3rrN)rrrnum_attention_heads num_headsscalerLinearqkv_biasqkvproj use_relative_position_embeddingsr%r&r'rkrl)r+r, input_sizerurhead_dimr.s r/rzVitDetAttention.__init__s   .. ")#t^ 99S#'@IIc3' 060W0W-  0 0\\%++a*Q-6G!6KX*VWDN\\%++a*Q-6G!6KX*VWDN 1r0c 0|j\}}}}|j|j|||zd|jdj ddddd}|jd||jz||zdj d\}} } ||j z| jddz} |jr(t| ||j|j||f||f} | jd} | | z}|j||j||d}|j ddddd}|j|||d}|j|}|r>| j||j| jd| jd} || f} | S|f} | S) Nr r2r3rr)ru)r8rrArrBunbindr transposerrxrkrlsoftmaxrhr)r+ hidden_stateoutput_attentionsrsrErFrtrrjkeysvaluesattention_scoresattention_probsoutputss r/rOzVitDetAttention.forwards'3'9'9$ FE1hh|$,,Z%DNN\^_gghiklnoqrtuv # AzDNN/JFUZN\^ _ f fgh iv#djj0DNN2r4JJ  0 0@ '4>>4>>FTY?]cej\k  +22r2:&/ #((T^^VUTVW #++Aq!Q: #++JrJ yy. -55DNNO,A,A",EG\G\]_G`O$_5G$oGr0N)FrPrQrRrSrrOrUrVs@r/rzrzsGX4r0rzinput drop_probtrainingrKc|dk(s|s|Sd|z }|jdfd|jdz zz}|tj||j|j z}|j |j||z}|S)aF Drop paths (Stochastic Depth) per sample (when applied in main path of residual blocks). Comment by Ross Wightman: This is the same as the DropConnect impl I created for EfficientNet, etc networks, however, the original name is misleading as 'Drop Connect' is a different form of dropout in a separate paper... See discussion: https://github.com/tensorflow/tpu/issues/494#issuecomment-532968956 ... I've opted for changing the layer and argument names to 'drop path' rather than mix DropConnect as a layer name and use 'survival rate' as the argument. rr)r)dtypedevice)r8ndimr&randrrfloor_div)rrr keep_probr8 random_tensoroutputs r/ drop_pathr sCx I I [[^  Q 7 7E 5 ELL YYM YYy !M 1F Mr0cxeZdZdZd deeddffd ZdejdejfdZ de fdZ xZ S) VitDetDropPathzXDrop paths (Stochastic Depth) per sample (when applied in main path of residual blocks).NrrKc0t|||_yr)rrr)r+rr.s r/rzVitDetDropPath.__init__!s "r0 hidden_statescDt||j|jSr)rrr)r+rs r/rOzVitDetDropPath.forward%s FFr0c8dj|jS)Nzp={})formatrr+s r/ extra_reprzVitDetDropPath.extra_repr(s}}T^^,,r0r) rPrQrRrSrfloatrr&rTrOstrrrUrVs@r/rrsGb#(5/#T#GU\\GellG-C-r0rc*eZdZdZdfd ZdZxZS)VitDetLayerNormaL A LayerNorm variant, popularized by Transformers, that performs point-wise mean and variance normalization over the channel dimension for inputs that have shape (batch_size, channels, height, width). https://github.com/facebookresearch/ConvNeXt/blob/d1fa8f6fef0a165b27399986cc2bdacc92777e40/models/convnext.py#L119 ct|tjt j ||_tjt j||_||_ |f|_ yr) rrrr%r&onesweightr'r}epsnormalized_shape)r+rrr.s r/rzVitDetLayerNorm.__init__3sT ll5::.>#?@ LL-=!>? !1 3r0c|jdd}||z jdjdd}||z tj||jzz }|j ddddf|z|j ddddfz}|S)NrT)keepdimr3)meanpowr&r;rrr})r+xuss r/rOzVitDetLayerNorm.forward:s FF1dF # UKKN  4  0 UejjTXX. . KK4 & *TYYq$}-E Er0)gư>rrVs@r/rr,s 4r0rc(eZdZdZfdZdZxZS)VitDetResBottleneckBlockz The standard bottleneck residual block without the last activation layer. It contains 3 conv layers with kernels 1x1, 3x3, 1x1. ct|tj||dd|_t ||_t|j|_ tj||ddd|_ t ||_ t|j|_ tj||dd|_ t ||_y)ar Args: config (`VitDetConfig`): Model configuration. in_channels (`int`): Number of input channels. out_channels (`int`): Number of output channels. bottleneck_channels (`int`): Number of output channels for the 3x3 "bottleneck" conv layers. rFr|r )paddingr}N)rrrr)conv1rnorm1r hidden_actact1conv2norm2act2conv3norm3)r+r, in_channels out_channelsbottleneck_channelsr.s r/rz!VitDetResBottleneckBlock.__init__Hs YY{,?O $%89 6,,- YY24GTU\ab $%89 6,,- YY2L!%P $\2 r0cN|}|jD] }||} ||z}|Sr)children)r+routlayers r/rOz VitDetResBottleneckBlock.forward`s5]]_ E*C #g r0rrVs@r/rrBs 30r0rcdeZdZdededdffd Zdej dej fdZxZS) VitDetMlp in_featureshidden_featuresrKNct|tj|||_t |j |_tj|||_tj|j|_ yr) rrrrfc1r ractfc2Dropout dropout_probdrop)r+r,rrr.s r/rzVitDetMlp.__init__jsZ 99[/:&++,99_k:JJv223 r0rc|j|}|j|}|j|}|j|}|j|}|Sr)rrrr)r+rs r/rOzVitDetMlp.forwardqsH HHQK HHQK IIaL HHQK IIaLr0) rPrQrRr9rr&rTrOrUrVs@r/rris84C4#4$4%,,r0rc `|j\}}}}|||zz |z}|||zz |z}tjj|ddd|d|f}||z||z} }|j |||z|| |z||}|j ddddddj j d|||} | || ffS)a  Partition into non-overlapping windows with padding if needed. Args: hidden_state (`torch.Tensor`): Input tokens with [batch_size, height, width, num_channels]. window_size (`int`): Window size. Returns: `tuple(torch.FloatTensor)` comprising various elements: - windows: windows after partition with [batch_size * num_windows, window_size, window_size, num_channels]. - (padded_height, padded_width): padded height and width before partition rrr r3rr2)r8rr?padrhrB contiguous) r window_sizersrErFr pad_height pad_width padded_height padded_widthwindowss r/window_partitionr{s/;.@.@+J| 44 CJu{22kAI==$$\Aq!Y:3VWL"(:"5uy7H}dN^N^._ +M+!% /"0" /q1 (,   a .}d>N>NP`cikpbqrM!4>>-#@@ %txx =@Y7Z([[ %--aAq9  " " MM-8M "W,r0)rrF)NF)rPrQrRrSrrr9boolrr&rTrrrrOrUrVs@r/rrsMqv!"!49!LO!im! !L-1"' (||(ELL)( ( uU\\5<</0% 2EE F (r0rceZdZdeddffd Z d dej deej deded ede e e ff d Z xZ S) VitDetEncoderr,rKNc t|||_|j}t j d|j |dDcgc]}|j}}g}t|D]I}|jt|||||jvr |jnd||jvKtj||_d|_ycc}w)Nrcpu)r)rrrF)rrr,num_hidden_layersr&linspaceritemrangeappendrwindow_block_indicesrresidual_block_indicesr ModuleListrgradient_checkpointing)r+r,depthrrlayersir.s r/rzVitDetEncoder.__init__ s  ((-2NN1f>S>SUZch,ijq!&&(jju A MM#1!#4676;V;V6V 2 2\]'(F,I,I'I   ]]6* &+#ksCrrroutput_hidden_states return_dictct|rdnd}|rdnd}t|jD]h\}} |r||fz}|||nd} |jr+|jr|j | j || |} n | || |} | d}|s`|| dfz}j|r||fz}|st d|||fDSt|||S)Nrrc3&K|] }|| ywrr).0vs r/ z(VitDetEncoder.forward..Fsmq_`_lmslast_hidden_stater attentions) enumeraterr r_gradient_checkpointing_func__call__rr ) r+rrrrrall_hidden_statesall_self_attentionsr layer_modulelayer_head_mask layer_outputss r/rOzVitDetEncoder.forward"s#7BD$5b4(4 POA|#$58H$H!.7.CilO**t}} $ A A ))!#% ! !-]OM^ _ )!,M &9]1=Mt|tjtjfrtjj |j jjtjd|jjj|j j|j _|j%|jjjyyt|tj rJ|jjj|j jj#dyt|t$rtjj |j&jjtjd|jjj|j&j|j&_yt|t(r|jj*rtjj |j,jjtjd|jj|j,_tjj |j.jjtjd|jj|j._yt|t0r|j2|j4|j6fD] }t9||j:|j<fD]K}|j jj#d|jjjM|j>j jj|j>jjjyy)zInitialize the weightsr)rstdNrY) rrrr)r) trunc_normal_rdatator&float32r,initializer_rangerr}zero_rfill_rr(rzrrkrlrrrrr,rrr)r+r$rs r/ _init_weightsz#VitDetPreTrainedModel._init_weightsds fryy"))4 5"$!6!6 ""%%emm43DKKDaDa"7"b$$% MM {{&   &&('  - KK   " " $ MM   $ $S )  0 1.0gg.C.C**//225==AKK11/D/b++112  & & +  0T[[5a5a$&GG$9$9  %%((7KK11%:%F   ! %'GG$9$9  %%((7KK11%:%F   !  8 9 ,, fllC ( ' ( ,, 5 ( !!'', %%' ( LL   $ $ * * , LL   " " ( ( *:r0)rPrQrRr config_classbase_model_prefixmain_input_namesupports_gradient_checkpointing_no_split_modulesrrrr)rr9rr0r/r.r.\sJL $O&*#)+E"))RYY *L$M)+RV)+r0r.ceZdZdeffd ZdefdZdeee efddfdZ e dde e jd e e jd e ed e ed e edeeeff d ZxZS) VitDetModelr,ct||||_t||_t ||_|jyr)rrr,rrNrencoder post_init)r+r,r.s r/rzVitDetModel.__init__s;   *62$V,  r0rKc.|jjSrrNr*rs r/get_input_embeddingsz VitDetModel.get_input_embeddings)))r0heads_to_pruneNc|jD]7\}}|jj|jj |9y)z Prunes heads of the model. heads_to_prune: dict of {layer_num: list of heads to prune in this layer} See base class PreTrainedModel N)itemsrBrr prune_heads)r+rHrheadss r/ _prune_headszVitDetModel._prune_headssE +002 CLE5 LL  u % / / ; ;E B Cr0rJrrrrc||n|jj}||n|jj}||n|jj}| t d|j ||jj }|j|}|j|||||}|d}|s |f|ddzSt||j|jS)a Examples: ```python >>> from transformers import VitDetConfig, VitDetModel >>> import torch >>> config = VitDetConfig() >>> model = VitDetModel(config) >>> pixel_values = torch.randn(1, 3, 224, 224) >>> with torch.no_grad(): ... outputs = model(pixel_values) >>> last_hidden_states = outputs.last_hidden_state >>> list(last_hidden_states.shape) [1, 768, 14, 14] ```Nz You have to specify pixel_values)rrrrrrr) r,rruse_return_dictr< get_head_maskrrNrBr rr) r+rJrrrrembedding_outputencoder_outputssequence_outputs r/rOzVitDetModel.forwards82C1N-TXT_T_TqTq$8$D $++JjJj &1% = 'tn = d^ =  uo% &= = r0r@zF ViTDet backbone, to be used with frameworks like Mask R-CNN. ) custom_introc ~eZdZfdZdefdZe d dejde e de e de e de f dZ xZ S) VitDetBackbonect||t| |t||_t ||_t|jdzDcgc]}|jc}|_ |jycc}w)Nr) rr_init_backbonerrNrrBrrr num_featuresrC)r+r,rtr.s r/rzVitDetBackbone.__init__sq   v&*62$V, 9>v?W?WZ[?[9\]AV//] ^sBrKc.|jjSrrErs r/rFz#VitDetBackbone.get_input_embeddingsrGr0rJrrrc||n|jj}||n|jj}||n|jj}|j |}|j |d||}|r |j n|d}d}t|j|D]\} } | |jvs|| fz }|s|r |f|ddz} | S|f|ddz} | St||r |j nd|jS)a Examples: ```python >>> from transformers import VitDetConfig, VitDetBackbone >>> import torch >>> config = VitDetConfig() >>> model = VitDetBackbone(config) >>> pixel_values = torch.randn(1, 3, 224, 224) >>> with torch.no_grad(): ... outputs = model(pixel_values) >>> feature_maps = outputs.feature_maps >>> list(feature_maps[-1].shape) [1, 768, 14, 14] ```NT)rrrrrr3) feature_mapsrr) r,rOrrrNrBrzip stage_names out_featuresr r) r+rJrrrrQrrr\stagerrs r/rOzVitDetBackbone.forwards56&1% ; d^ ;  ; ; r0rV)r@r.rV)rF)6rScollections.abcrr:typingrrrrrr&torch.utils.checkpointr activationsr modeling_outputsr r modeling_utilsr utilsrrutils.backbone_utilsrconfiguration_vitdetr get_loggerrPloggerModulerr=script_if_tracingrerxrzrTrrrrrrrrrrrr,r.r@rV__all__rr0r/ros 55 !?-,1.   H %UryyUp!3!3H&R;bii;~U\\eTV[VbVb* -RYY -bii,$ryy$N $2@>N"))Nb@ BII@ F *RYY *4 *0+O0+0+fT 'T T n K *MK  K \ Er0