U g@sddlmZmZddlZddlZddlmZdZe eZ e eZ dZ ddl mZee ddZd ZeZeed ed d ZdddZdS)) AutoTokenizer"AutoModelForSequenceClassificationN) preprocessingz%ahmedheakl/bert-resume-classificationzahmedheakl/resume-atlas) load_datasetT)Ztrust_remote_codeCategorytraincCsPt|dddd}tf|}tjjj|jdd}t|}t |gd}|S)NptTZreturn_tensors truncationpadding)dimr) tokenizermodeltorchnn functionalsoftmaxlogitsargmaxitemleinverse_transform)textinputsoutputs probabilitiesZpredicted_class_indexZpredicted_categoryr$/home/manasa/App/modules/classify.py classify_texts  rffffff?cCs^t|dddd}tf|}tjj|j}||kd}ddt |D}|sZdgS|S)NrTr rcSs(g|] \}}|dkrt|gdqS)r)rr).0idxvalrrr (sz'classify_text_multi..zUncertain Prediction) rrrrrsigmoidrinttolist enumerate)r thresholdrrrZpredicted_classesZ job_titlesrrrclassify_text_multi"s r+)r ) transformersrrrnumpynpsklearnr model_nameZfrom_pretrainedrr dataset_iddatasetsrdsZ label_column LabelEncoderrfitrr+rrrrs