§ ˜’MgÊ ãóî—ddlZddlmZddlmZ ddejdeded ejfd „Z ddejdede dededej dzd ejfd„Z ddede de fd„Z Gd„dej ¦«ZdS)éN)Únn)Ú functionalçéÚ logits_BlVÚtop_kÚtop_pÚreturncóÚ—|j\}}}|dkrY|| |ddd¬¦«d dd¬¦«k}| |tj ¦«|dkr’| dd¬¦«\} } |  d¬¦« d¬¦«d |z k} d| d dd…f<| |   | j d z | | ¦«tj ¦«|dk} t|¦«}t j | d¬¦«  d|¦«|| |¬ ¦«  |||¦«S) NrTFéÿÿÿÿ)ÚlargestÚsortedÚdim)rÚkeepdim)rÚ descending)rr.)Ú num_samplesÚ replacementÚ generator)ÚshapeÚtopkÚaminÚ masked_fill_ÚtorchÚinfÚsortÚsoftmaxÚcumsum_ÚscatterÚndimÚabsÚ multinomialÚview) rrr ÚrngrÚBÚlÚVÚ idx_to_removeÚ sorted_logitsÚ sorted_idxÚsorted_idx_to_removers ú1/home/notantonvoron/switti_demo/models/helpers.pyÚsample_with_top_k_top_p_r,sž€ðÔG€A€qˆ!Ø ˆq‚yð;Ø" Z§_¢_Ø ˜4¨°2ð&5ñ& ô& à ô& ç ŠTb $ˆTÑ 'Ô 'ò(ˆ ð ×Ò  µ´ ¨zÑ:Ô:Ð:Ø ˆq‚yð  Ø$.§O¢O¸Àu OÑ$MÔ$MÑ!ˆ zØ,×4Ò4¸Ð4Ñ<Ô<×DÒDÈÐDÑLÔLÐQRÐUZÑQZÒ[ÐØ).ИS " # #˜XÑ&Ø×ÒØ × (Ò (Ø” !Ñ# ZÐ1Eñ ô õŒYˆJñ  ô ð ð Ò"€KÝkÑ"Ô"€KÝ Ô Ø×Ò˜rÐÑ"Ô"×'Ò'¨¨AÑ.Ô.ØØØð  ñ ô ÷  ‚dˆ1ˆaÑÔð óF绽×Ùß|Û=r ÚlogitsÚtauÚhardÚepsrr#cóæ—|€tj|||||¬¦«Stj|tj¬¦« |¬¦« ¦« }||z|z }| |¦«}|rm| |d¬¦«d}tj |tj¬¦«  ||d¦«} | |  ¦«z |z} n|} | S)N)r/r0r1r2r)Ú memory_format)rT)rrgð?) ÚFÚgumbel_softmaxrÚ empty_likeÚlegacy_contiguous_formatÚ exponential_ÚlogrÚmaxÚ zeros_likeÚscatter_Údetach) r/r0r1r2rr#ÚgumbelsÚy_softÚindexÚy_hardÚrets r+Úgumbel_softmax_with_rngrD(s€ð ðUÝÔ v°3¸TÀsÐPSÐTÑTÔTÐTõ Ô ˜&µÔ0NÐ OÑ OÔ Oß Š ˆÑ $Ô $ß Š‰Œð ð ð ˜Ñ 3Ñ&€GØ _Š_˜SÑ !Ô !€Fà ðØ— ’ ˜3¨ Ñ-Ô-¨aÔ0ˆÝÔ!Ø ¥%Ô"@ð ñ ô ç Š(3˜˜sÑ #Ô #ð ðv—}’}‘”Ñ&¨Ñ/ˆˆàˆØ €Jr-TÚ drop_probÚtrainingÚ scale_by_keepcóì—|dks|s|Sd|z }|jdfd|jdz zz}| |¦« |¦«}|dkr|r| |¦«||zS)Nrrr)r)rrÚ new_emptyÚ bernoulli_Údiv_)ÚxrErFrGÚ keep_probrÚ random_tensors r+Ú drop_pathrOFs”€ðCÒð˜xðØˆØI‘ €IØ ŒWQŒZˆM˜D A¤F¨Q¡JÑ/Ñ /€EØ—K’K Ñ&Ô&×1Ò1°)Ñ<Ô<€MØ3‚ð&˜=ð&Ø×Ò˜9Ñ%Ô%Ð%Ø ˆ}Ñ Ðr-có6‡—eZdZddedefˆfd„ Zd„Zd„ZˆxZS) ÚDropPathrTrErGcór•—tt|¦« ¦«||_||_dS©N)ÚsuperrQÚ__init__rErG)ÚselfrErGÚ __class__s €r+rUzDropPath.__init__Ts4ø€Ý h˜ÑÔ×&Ò&Ñ(Ô(Ð(Ø"ˆŒØ*ˆÔÐÐr-cóD—t||j|j|j¦«SrS)rOrErFrG)rVrLs r+ÚforwardzDropPath.forwardYs€Ý˜˜DœN¨D¬M¸4Ô;MÑNÔNÐNr-có—dS)Nz(drop_prob=...)©)rVs r+Ú extra_reprzDropPath.extra_repr\s€Ø!Ð!r-)rT) Ú__name__Ú __module__Ú __qualname__ÚfloatÚboolrUrYr\Ú __classcell__)rWs@r+rQrQSsqø€€€€€ð+ð+ %ð+¸dð+ð+ð+ð+ð+ð+ð OðOðOð"ð"ð"ð"ð"ð"ð"r-rQ)rrNr)rFr.r N)rFT)rrÚtorch.nnrr5ÚTensorÚintr`r,raÚ GeneratorrDrOÚModulerQr[r-r+úrhsgðØ € € € ØÐÐÐÐÐØ$Ð$Ð$Ð$Ð$Ð$ð ØØ Øð ðØ” ðà ðð ðð  „\ð ððððHØØØØ"&ð ðØ ŒLðà ðð ðð ð ð ð 𠌘4Ñ ð ð „\ððððð>NRð ð Øð Ø)-ð ØFJð ð ð ð ð "ð "ð "ð "ð "ˆrŒyñ "ô "ð "ð "ð "r-