a ̴xd- @svddlZddlZddlZddlZddlZddlZeejejj ddZ ddZ ddZ dd Z d d Zd d ZdS)NcCs tdtjd|||d}|S)Ng333333?ZPOST)headersdata)timesleeprequestsZrequest)urlrpayloadresponser ./mnt/chromeos/MyFiles/work/nlp_tsa/TSA/util.pypost_url s r cCstd|}|S)Nu(?<=[。!?])\s*)resplit)text sentencesr r r segs rcCs|dd}tdd|}tdd|}tdd|}tdd|}d}|D]}|s`|dkrL||7}qLddd |D}|S) N  -z \d+/\d+/\d+z[0-2]?[0-9]:[0-6][0-9]z>/[a-zA-Z]*[:\//\]*[A-Za-z0-9\-_]+\.+[A-Za-z0-9\.\/%&=\?\-_]+/icss|]}t|dkr|VqdS)N)len).0Zwordr r r !zclean_text..)replacer subisalphajoinr)rZ pure_textZletterr r r clean_texts  rcCsLi}|D]>}|d|vr*|d||d<q||d|d||d<q|S)Nrrr )groupsZtopicsparair r r article_to_group%s  r#csddfdd}d|}i}g}t|D]`\}\}}|} t||d} ddd} t|| | } t| j} ||f|| d <|| d q4||fS) NZIZt1uK9PAI0LiqleqT0cE30bZ Xv5kHB8eyhNuI1B1G7fRgm2SIPdlxGxscs,d}dd}ttj||ddS)Nz(https://aip.baidubce.com/oauth/2.0/tokenZclient_credentials)Z grant_typeZ client_idZ client_secret)paramsZ access_token)strrZpostjsonget)rr$ZAPI_KEYZ SECRET_KEYr r get_access_token3s z$generation..get_access_tokenzPhttps://aip.baidubce.com/rpc/2.0/nlp/v1/news_summary?charset=UTF-8&access_token=)ZcontentZmax_summary_lenzapplication/json)z Content-TypeZAcceptZsummary) enumerateitemsr&dumpsr loadsrappend)r! max_lengthr)rZtopicZ Ai_abstractr"jkZ input_textrrr Z text_dictr r(r generation/s(   r2cCs\g}|D]N}||vr(|d|||vr|d||||dq|S)Nz# z## r)keysr.) title_dict outline_listZformatedZeachr r r formate_textOs  r6)r&Zjiebar rZbackoffrZ on_exceptionZexpo exceptionsZRequestExceptionr rrr#r2r6r r r r s