rh9ddlZddlZddlmZmZmZmZddlmZddl m Z ddl m Z ddl mZddlmZeejeZded ed efd Zd ed ed efdZd ed efdZd eded efdZd ed efdZd eded efdZd ed efdZejeddZejeddZejeddZejeddZ ejedd Z!d e fd!Z"ed"krej#ej$d#$e%d%d&gZ&d'e&DZ'e'r$e(d(d))e'd*dSej*d+d,Z+d-e+,vr%d.e+,vre(d/e+d0d1Z- dd2l.m/Z/m0Z0m1Z1e/j2d3d4d56Z3e0j4e3Z5 e1j6d7d8Z7n%#e8$re1j9Z7e(d9YnwxYwe5:d:d;dd?@e5:dAdBdee-e>Z?e(dLe?e(dMdNZ@ee-e@ZAe(dOeAejBe-dS#eC$rGe(dP e"ZDe(dQYdS#eE$rZFe(dReFYdZF[FYdSdZF[FwwxYweE$rIZFe(dSeFejGHe-rejBe-YdZF[FdSYdZF[FdSdZF[FwwxYwdS)TN)ListDictOptionalUnion) load_dotenv) ReActAgent) ImageDocument) FunctionTool) GoogleGenAIfilenamedefault_promptreturnc tjt}tj||}t |d5}|}td||cdddS#1swxYwYdS#t$r&t d|d|d|cYSt$r.}t d|d|d |cYd}~Sd}~wwxYw) z Loads a prompt from a text file.rz Successfully loaded prompt from Nz Prompt file z not found at z. Using default.zError loading prompt file z: Texc_info) ospathdirname__file__joinopenreadloggerinfoFileNotFoundErrorwarning Exceptionerror)r r script_dir prompt_pathfpromptes I/Users/aedelon/Workspace/GAIA_Agent/agents/figure_interpretation_agent.pyload_prompt_from_filer&sl W__X.. gll:x88 +s # # qVVXXF KKH;HH I I I                   [h[[k[[[\\\  A(AAaAAD QQQsHAB#3B B#BB#BB##-D  D #D>D D  image_pathrequestctd|d|tj|s"td|d|Stjdd}tjd}|stdd S t|| }td |d |d }ddlm }||g}| }|rt|dts#td|d|dS| ||} | j} td| S#t $r%td|d|cYSt"$r-} td| d| dcYd} ~ Sd} ~ wt$$rz} td| ddt'| vr)td|d d!|d"cYd} ~ Sd#| cYd} ~ Sd} ~ wwxYw)$aInterprets a figure in an image based on a specific request using a multi-modal LLM. Args: image_path (str): Path to the image file containing the figure. request (str): The specific question or interpretation task (e.g., "Describe this chart", "Extract sales for Q3", "Identify the main trend"). Returns: str: The interpretation result or an error message. zInterpreting figure in image: z with request: zImage file not found: zError: Image file not found at FIGURE_INTERPRETATION_LLM_MODELmodels/gemini-1.5-proGEMINI_API_KEYz7GEMINI_API_KEY not found for figure interpretation LLM.zError: GEMINI_API_KEY not set.api_keymodelz!Using figure interpretation LLM: zYou are an expert figure interpreter. Analyze the provided image containing a chart, graph, diagram, or table. Focus *only* on the visual information present in the image. Fulfill the following request accurately and concisely: REQUEST: z0 Based *only* on the image, provide the answer:r)SimpleDirectoryReader) input_filesz'Failed to load image as ImageDocument: z!Error: Could not load image file z for analysis.)r#image_documentsz!Figure interpretation successful.z&Image file not found during LLM call: z,Missing library for multi-modal processing: z6Error: Missing required library for image processing (z).Nz.LLM call failed during figure interpretation: Trzdoes not support image inputzThe configured model z does not support image input.zError: The configured LLM (zE) does not support image input. Please configure a multi-modal model.z$Error during figure interpretation: )rrrrexistsrgetenvr llama_index.corer0 load_data isinstancer completetextstripr ImportErrorrstrlower) r'r(llm_model_namegemini_api_keyllmr#r0readerr2responseinterpretationier$s r%interpret_figure_with_llmrE$s: KKUUUGUUVVV 7>>* % %> :j::;;;====Y@BYZZNY/00N 0 NOOO//::.GGG HHHIII  >  > > >  ;:::::'&J<@@@ **,, Sj1C]&S&S S \\PJPP Q Q QR RRR R<<+    ",,.. 7888 ??? KzKK L L L>*>> > > > PPP IRII J J JOOOO O O O O O O ::: IaIITX YYY )SVV\\^^ ; ; \\`.``` a a aH.HHHHHHHHH9a99999999:sK*BFA F,I7? I7"G0*I70 I7=A%I2"I7(I2,I72I7c"t|dS)zRProvides a general description of the figure in the image (type, elements, topic).zbDescribe this figure, including its type, main elements (axes, labels, legend), and overall topic.rEr's r%describe_figure_tool_fnrI{s $Z2V W WW data_requestc*t|d|dS)zEExtracts specific data points or values from the figure in the image.z:Extract the following data points/values from the figure: zR. If exact values are not clear, provide the closest estimate based on the visual.rG)r'rKs r%extract_data_points_tool_fnrMs4 $Z2Onz2O2O2O P PPrJc"t|dS)zMIdentifies and describes trends or patterns shown in the figure in the image.zFAnalyze and describe the main trends or patterns shown in this figure.rGrHs r%identify_trends_tool_fnrOs $Z1y z zzrJcomparison_requestc*t|d|dS)z;Compares different elements within the figure in the image.z2Compare the following elements within the figure: z<. Be specific about the comparison based on the visual data.rG)r'rPs r%compare_elements_tool_fnrRs4 $Z2wfx2w2w2w x xxrJc"t|dS)zPSummarizes the key insights or main message conveyed by the figure in the image.zGSummarize the key insights or the main message conveyed by this figure.rGrHs r%!summarize_figure_insights_tool_fnrTs $Z1z { {{rJdescribe_figurezkProvides a general description of the figure in the image (type, elements, topic). Input: image_path (str).)fnname descriptionextract_data_pointszbExtracts specific data points/values from the figure. Input: image_path (str), data_request (str).identify_trendszPIdentifies and describes trends/patterns in the figure. Input: image_path (str).compare_elementszaCompares different elements within the figure. Input: image_path (str), comparison_request (str).summarize_figure_insightszPSummarizes the key insights/main message of the figure. Input: image_path (str).ctdtjdd}tjd}|s)tdt d t ||}td|d }td |}||krtd tttttg}gd }td d||||}td|S#t$r%}td|dd}~wwxYw)z,Initializes the Figure Interpretation Agent.z)Initializing FigureInterpretationAgent...%FIGURE_INTERPRETATION_AGENT_LLM_MODELr+r,z7GEMINI_API_KEY not found for FigureInterpretationAgent.z8GEMINI_API_KEY must be set for FigureInterpretationAgentr-zUsing agent LLM: zSYou are FigureInterpretationAgent... [Default prompt content - replace with actual]z1../prompts/figure_interpretation_agent_prompt.txtzCUsing default/fallback system prompt for FigureInterpretationAgent.) planner_agentresearch_agentreasoning_agentfigure_interpretation_agentzAnalyzes and interprets visual data representations (charts, graphs, tables) from image files. Can describe figures, extract data, identify trends, compare elements, and summarize insights.)rWrXtoolsr@ system_promptcan_handoff_toz3FigureInterpretationAgent initialized successfully.z7Error during FigureInterpretationAgent initialization: TrN)rrrr4r ValueErrorr r&rdescribe_figure_toolextract_data_points_toolidentify_trends_toolcompare_elements_toolsummarize_figure_insights_toolrr) agent_llm_modelr?r@default_system_promptrdrcvalid_handoffsagentr$s r%&initialize_figure_interpretation_agentrps KK;<<<i GI`aaOY/00N U NOOOSTTT0.HHH 999:::"w-.acxyy 1 1 1 ^^a b b b ! $ ! *      .q')      IJJJ  RqRR]a bbb s0B4D%% E/ EE__main__z4%(asctime)s - %(name)s - %(levelname)s - %(message)s)levelformatz>Running figure_interpretation_agent.py directly for testing...r,c:g|]}tj||S)rr4).0keys r% rxs%GGGC #GCGGGrJz1Error: Required environment variable(s) not set: z, z. Cannot run test.r*r+provisionzWarning: Configured LLM z/ might not support image input. Tests may fail.zdummy_figure.png)Image ImageDraw ImageFontRGB)i)rr)colorz arial.ttfz-Arial font not found, using default PIL font.) rzSimple Bar Chart)rrr)fillfont)2rd)rrr)r)<A)rPrr)rrr)rrBzCreated dummy image file: z Testing describe_figure...z Description: z- Testing extract_data_points (qualitative)...zHeight of bar A vs Bar BzExtraction Result: z Testing compare_elements...zCompare bar A and bar BzComparison Result: zUPillow library not installed. Skipping direct tool tests that require image creation.zK Figure Interpretation Agent initialized successfully (tool tests skipped).zError initializing agent: zError during testing: )Irloggingtypingrrrrdotenvrllama_index.core.agent.workflowrllama_index.core.schemar llama_index.core.toolsr llama_index.llms.google_genair getLogger__name__rr<r&rErIrMrOrRrT from_defaultsrgrhrirjrkrp basicConfigINFOr required_keys missing_keysprintrr4 model_namer=dummy_image_pathPILr{r|r}newimgDrawdtruetyperIOError load_defaultr9 rectanglesavedesc extract_req extract_res compare_req compare_resremover; test_agentrr$rr3rurJr%rs- ............666666111111//////555555   8 $ $C$R:#R:R:R:R:R:R:nWWWWWWPCPsPsPPPP{{{{{{xx#x#xxxx|#|#|||| 2|1 } 6<5" t 2|1 b 3 2 s "<!;( $b"""< <<<<~ zGgl3ijjjj KKPQQQ&&MGG=GGGL9- m$))LBYBYmmmnnnnnRY@BYZZ  ((** * *xz?O?O?Q?Q/Q/Q Uijiii j j j./ - 7 7 7 7 7 7 7 7 7 7%)E:GGGC s##A H* *;;; H H H. .00FGGGGG H FF7.W4F H H H KK***K ; ; ; FF9cdF ; ; ; KK+++)K < < < FF:stF < < < HH% & & & EA/?AA B B B E0 1 1 1**+;<