eEg 3 d dl Z d dlZd dlT d dlZd dlZd dlmZ d dl
m Z d dl
Zd dlmZ d dlmZ d&ded fdZd'd ed
efd Zd(d efd
ZdedefdZd)dedededefdZd*defdZ d'de de de fdZ d+d Z d,d Z d-d!ed"efd#Z!d.d%Z"dS )/ N)*)LogisticRegression)Navbarcenteredlayout)wider c t j d| t j }d}t j |d dt j vrdt j d< |S )Nz VARCO Arena)
page_titler z
T)unsafe_allow_htmlkoreanF)stset_page_configsidebaremptymarkdown
session_state)r sidebar_placeholdercsss ?/home/deftson/nfs-deftson/2024/public_varco_arena/view_utils.pydefault_page_settingr sm -???? ***,, C Kt,,,,r'''%*"
is_runningtoggle_hashstrc F | t j d< t || d S )Nr )r )r
r r )r r r s r set_nav_barr ' s) %/B\"
~>>>>>>r did_select_promptc $ | t j d< d S )Nr )r
r )r expander_placeholders r set_prompt_previewr - s ,=B()))r textreturnc \ t | t r| dd ndS )N
z
z(Empty))
isinstancestrreplace)r s r show_linebreak_in_mdr' 1 s* +5dC+@+@ O4<<h ' ' 'iOr version entity_typec t | dk rd}n5t | dk r|dv rd}n|dk rd}nd}nt d t j d
t j | d d | S )
aM
Helper function to escape telegram markup symbols.
Args:
text (:obj:`str`): The text.
version (:obj:`int` | :obj:`str`): Use to specify the version of telegrams Markdown.
Either ``1`` or ``2``. Defaults to ``1``.
entity_type (:obj:`str`, optional): For the entity types ``PRE``, ``CODE`` and the link
part of ``TEXT_LINKS``, only certain characters need to be escaped in ``MarkdownV2``.
See the official API documentation for details. Only valid in combination with
``version=2``, will be ignored else.
z_*`[r( )precodez\` text_linkz\)z_*[]()~`>#+-=|{}.!:z'Markdown version must be either 1 or 2!z([z])z\\\1)int
ValueErrorresubescape)r r) r* escape_charss r escape_markdownr6 5 s 7||q W / ) ) LL
K
'
' LL1LLBCCC
62ry ..222GT B BBr float_ptsc ddl m} | d } |t | }| d | | d< | S )z
Post-processing utility for saving elo table to an excel file. Possibly work as a absolute measure for quality.
elo_df:
columns: Model, Elo rating
add:
column: relative_winrate_to_1st
r )partial
Elo rating)rating_bwinrate_vs_1st) functoolsr: maxelo_to_winrateapply)elo_dfr8 r: rating1stwin_rate_to_1sts r compute_relative_winrate_to_1strE R sc "!!!!!|$((**I gnyAAAO %l 3 9 9/ J JF Mr rating_ar< c . | |z
}ddd| dz z z z }|S )Nr,
)rF r< rate_diffwin_rates r r@ r@ e s, 8#IA zC /001H Or rI rH c t | t rt j | } | g d } t j | d | d g }t j t j t | | }t j | | gd } t |j }| j d }t j
||g }t j |
|t j | || d f< t j | |t j | || d f< t j
| }d || d
d k < d } |dk r)t# | d
d |d< n@|d k r(t# | d d|d< t' d }
|
|| ||
j d z |z t j |j d t j fd D ddg dd d } | j dz | _ | S )N)winnermodel_amodel_b)subsetrP rQ )indexT) ignore_indexr g ?rO Aa elo.py:L{L} compute_mle_elo() // Warning: Seeing this message indicates the regression result for elo is unreliable. You should be test-running the Varco Arena or something odd (perfect one-sided wins) is happening
to avoid logistic regressor error, manually putting other class )L# g F)
fit_intercept ascendingc @ g | ]}|t | d gS )r( )round).0n
elo_scoress r
z#compute_mle_elo.. s, E E Eaaz!}a((
) E E Er Modelr; columns)dropr, )r$ listpd DataFramedropnaconcatuniqueSeriesnparangelenrS shapezerosmathlogallprintformatr fitcoef_ sort_valueskeys reset_index) dfSCALEBASE INIT_RATINGmodelspr` XYWARNINGlrra s @r compute_mle_elor l s "d
\"
::: ; ;B
Y9
r)}5
6
6
=
=
?
?F
YryV--V
<
<
._win_to_loss_wc_ratio s
:$$ 5 5 7 7 8 83s?T?T?V?V;W;W WEE 5 5 7 7 8 83s?T?T?V?V;W;W WE
Ls B&B) )
B?3B::B?r, r r zwin/loss wc ratiocategoryc g | ]}|d v| S ))r r rJ )r_ r s r rb z%_plot_length_bias.. s# KKKS#5J*J*JS*J*J*Jr rd z
Length bias ())xytitlewonrO rU )r
wordcountslostzwon/lost ratior outcome wordcount)r r )r r r labels)r$ rh ri from_dictrA rf pxvioliniterrowsappendrp r r r )
resultsr r r r} plot_dffigdata_r s
r _plot_length_biasr s gr| , , 2,((11 E
hh 41h==7 ,: ''KKBKKK
i .)...
&&(( FAs KK %8}++ #&co&;&;&=&=">">"> S_ 2 2 4 455
KK &8}++ #&co&;&;&=&=">">"> S_ 2 2 4 455
KK 0 8}++ #&co&;&;&=&=">">#///1122#3 #3 S_ 2 2 4 455#///11223
,t$$ i .)... ) DD
<r Fc H t t j s t j i } j d d }t | \ }}||d< ||d< t
j d d| dd d
}| d d
d ||d< t
j t j
d d g dd }| ddd
d ||d< t j dddd }||j z }|
d j }t
j |j ||f d d }| d d d d d d!d t% d"# $ | d%& ||d'< t j d d(k dddd } t) | } t j d d)k dddd }
t) |
}
t j dddd } | |
j z | | j z z } | d*+ d }
t- |
}| j ||f }t
j |d,d-d./ }| d d d!d d d d 0 | d1& ||d2< t1 }t3 | }|d3 t6 |d3< |d4 d5 |d4< d6|j _ ||d7< |r d8 }t j
fd9|D }t
j |d3d8d d8|idd t j! j" j# d:; }||d<< ||d=< |S )>uv
varco_arena/visualization.py 로부터 가져온 함수이나 업데이트가 많이 되었으므로 조심!
r evaluation_model)r length_biaslength_bias_dfrO zPosition A/B bias
(r TrI )r text_autoheightz Match WinnerCountF) xaxis_title yaxis_title
showlegendcounts_of_match_winnersrP rQ zMatch Count per Model)r r rc z Match Count)r r r r match_count_for_each_modelsize)rS re aggfunc
fill_valuer[ z#Number of Matches (model vs. model)zModel BzModel Atopi leftrH )r )r r
xaxis_sider width
title_xanchor
title_yanchorfontzModel B: %{x}
Count: %{z})
hovertemplate)match_count_of_each_combination_of_modelsrU Br, r RdBuz.2fz P(A wins B))color_continuous_scaler r )r r r r r r r zBModel A: %{y}
Model B: %{x}
P(A wins B): %{z}/fraction_of_model_a_wins_for_all_a_vs_b_matchesr; r= r7 Rank
elo_ratingtaskc v g | ]5}t d |k | 6S )r )r )r assign)r_ r r s r rb z!visualization.. sP
4(? @AAHHdHSS
r zElo Rating by Task)rtheta
line_closecategory_orderscolormarkerscolor_discrete_sequencer elo_rating_by_taskr )$r$ rh ri r ilocr r bar value_counts
update_layoutrk pivot_tableTsumrz rS imshowr dict
update_tracesr meanrg r{ r rE astyper0 r^ namerl tolist
line_polarcolors qualitativePastel)r
is_overall figure_dictr r r ptbl match_countsordering
a_win_ptbl
b_win_ptblnum_results_ptblrow_beats_col_freq prop_wins model_names
row_beats_coleloelo_wrtasks elo_by_tasks ` r
visualizationr
s gr| , , 2,((11K Q 23I %W BBBLC!$K
$+K !
&&&((1Y111 C .gRWXXX-0K)*
&
79%wy'9:;;HHJJ% C
su 14K,- > D $&=L !!--->>DH
)8+,3 C
r]]] T @CK;< !S() J %Z00J !S() J %Z00J~y)VPQ %z|3+-- #''Q'//;;e;LLIy~~''((K&*; +CDM
) % C Z FIKAB '
"
"C
,S
1
1F!,/66s;;F< %&6 7 = =a @ @F FL &K 0&&((//11i
!
m #UO$&I$9$@&
-0 () (K r )r )NN)N)r( N)r7 )rI rH rM )r )NT)F)#rs r2 typingnumpyrn pandasrh plotly.expressexpressr plotly.graph_objects
graph_objectsgo streamlitr
sklearn.linear_modelr modules.navr Literalr boolr% r r r' r0 r6 rE floatr@ r r r r rJ r r r s2 ! ! ! ! ! ! 3 3 3 3 3 3
+5 & ' .? ?D ?C ? ? ? ? > >$ > > > >Ps Ps P P P PC C# C Cc CS C C C C: s & U U e . . . .b >K K# KT K K K K\S S S S S Sr