First Commit

First Commit

First Commit

.gradio/certificate.pem

@@ -0,0 +1,31 @@
+-----BEGIN CERTIFICATE-----
+MIIFazCCA1OgAwIBAgIRAIIQz7DSQONZRGPgu2OCiwAwDQYJKoZIhvcNAQELBQAw
+TzELMAkGA1UEBhMCVVMxKTAnBgNVBAoTIEludGVybmV0IFNlY3VyaXR5IFJlc2Vh
+cmNoIEdyb3VwMRUwEwYDVQQDEwxJU1JHIFJvb3QgWDEwHhcNMTUwNjA0MTEwNDM4
+WhcNMzUwNjA0MTEwNDM4WjBPMQswCQYDVQQGEwJVUzEpMCcGA1UEChMgSW50ZXJu
+ZXQgU2VjdXJpdHkgUmVzZWFyY2ggR3JvdXAxFTATBgNVBAMTDElTUkcgUm9vdCBY
+MTCCAiIwDQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBAK3oJHP0FDfzm54rVygc
+h77ct984kIxuPOZXoHj3dcKi/vVqbvYATyjb3miGbESTtrFj/RQSa78f0uoxmyF+
+0TM8ukj13Xnfs7j/EvEhmkvBioZxaUpmZmyPfjxwv60pIgbz5MDmgK7iS4+3mX6U
+A5/TR5d8mUgjU+g4rk8Kb4Mu0UlXjIB0ttov0DiNewNwIRt18jA8+o+u3dpjq+sW
+T8KOEUt+zwvo/7V3LvSye0rgTBIlDHCNAymg4VMk7BPZ7hm/ELNKjD+Jo2FR3qyH
+B5T0Y3HsLuJvW5iB4YlcNHlsdu87kGJ55tukmi8mxdAQ4Q7e2RCOFvu396j3x+UC
+B5iPNgiV5+I3lg02dZ77DnKxHZu8A/lJBdiB3QW0KtZB6awBdpUKD9jf1b0SHzUv
+KBds0pjBqAlkd25HN7rOrFleaJ1/ctaJxQZBKT5ZPt0m9STJEadao0xAH0ahmbWn
+OlFuhjuefXKnEgV4We0+UXgVCwOPjdAvBbI+e0ocS3MFEvzG6uBQE3xDk3SzynTn
+jh8BCNAw1FtxNrQHusEwMFxIt4I7mKZ9YIqioymCzLq9gwQbooMDQaHWBfEbwrbw
+qHyGO0aoSCqI3Haadr8faqU9GY/rOPNk3sgrDQoo//fb4hVC1CLQJ13hef4Y53CI
+rU7m2Ys6xt0nUW7/vGT1M0NPAgMBAAGjQjBAMA4GA1UdDwEB/wQEAwIBBjAPBgNV
+HRMBAf8EBTADAQH/MB0GA1UdDgQWBBR5tFnme7bl5AFzgAiIyBpY9umbbjANBgkq
+hkiG9w0BAQsFAAOCAgEAVR9YqbyyqFDQDLHYGmkgJykIrGF1XIpu+ILlaS/V9lZL
+ubhzEFnTIZd+50xx+7LSYK05qAvqFyFWhfFQDlnrzuBZ6brJFe+GnY+EgPbk6ZGQ
+3BebYhtF8GaV0nxvwuo77x/Py9auJ/GpsMiu/X1+mvoiBOv/2X/qkSsisRcOj/KK
+NFtY2PwByVS5uCbMiogziUwthDyC3+6WVwW6LLv3xLfHTjuCvjHIInNzktHCgKQ5
+ORAzI4JMPJ+GslWYHb4phowim57iaztXOoJwTdwJx4nLCgdNbOhdjsnvzqvHu7Ur
+TkXWStAmzOVyyghqpZXjFaH3pO3JLF+l+/+sKAIuvtd7u+Nxe5AW0wdeRlN8NwdC
+jNPElpzVmbUq4JUagEiuTDkHzsxHpFKVK7q4+63SM1N95R1NbdWhscdCb+ZAJzVc
+oyi3B43njTOQ5yOf+1CceWxG1bQVs5ZufpsMljq4Ui0/1lvh+wjChP4kqKOJ2qxq
+4RgqsahDYVvTH9w7jXbyLeiNdd8XM2w9U/t7y0Ff/9yi0GE44Za4rF2LN9d11TPA
+mRGunUHBcnWEvgJBQl9nJEiU0Zsnvgc/ubhPgXRR4Xq37Z0j4r7g1SgEEzwxA57d
+emyPxgcYxn/eR44/KJ4EBs+lVDR3veyJm+kXQ99b21/+jh5Xos1AnX5iItreGCc=
+-----END CERTIFICATE-----

README.md

@@ -1,5 +1,5 @@
 ---
-title: 视频Pose Estimation处理平台
+title: AI Sports Coaching
 emoji: 🎬
 colorFrom: blue
 colorTo: purple
@@ -9,61 +9,66 @@ app_file: app.py
 pinned: false
 ---
 
-# 🎬 视频Pose Estimation处理平台
+# 🎬 AI Sports Coaching System
 
-一个基于Gradio的视频姿态估计和AI分析平台，支持视频上传、姿态检测、时序下采样和智能分析。
+A video-based pose estimation and AI analysis platform powered by Vision-Language Models (VLMs). Users can upload videos, perform pose detection, temporal downsampling, and get intelligent feedback.
 
-## 功能特性
+## Features
 
-- 📹 **视频上传**: 支持多种视频格式
-- 🤖 **Pose Estimation**: 人体姿态关键点检测
-- ⏱️ **时序下采样**: 可配置的帧率降采样
-- 🧠 **AI分析**: 集成ChatGPT进行智能分析
-- 🐍 **Python处理**: 后续数据处理和分析
-- 📊 **结果展示**: 多维度结果可视化
+- 📹 **Video Upload**: Support for multiple video formats
+- 🤖 **Pose Estimation**: Human keypoint detection
+- ⏱️ **Temporal Downsampling**: Configurable frame rate reduction
+- 🧠 **AI Analysis**: Integrate LLMs/VLMs for intelligent insights
+- 🐍 **Python Processing**: Custom data processing and analysis pipelines
+- 📊 **Result Visualization**: Multi-dimensional result display
+- ⭐ **Scoring Mechanism**: User feedback scoring for outputs
 
-## 处理流程
+## Workflow
 
-1. **视频上传** → 上传待分析的视频文件
-2. **Pose Estimation** → 提取人体关键点数据
-3. **时序下采样** → 根据设定比率降采样
-4. **ChatGPT分析** → AI智能分析姿态特征
-5. **Python处理** → 运行自定义分析程序
-6. **结果生成** → 输出综合分析报告
+1. **Video Upload** → Upload one or more videos for analysis
+2. **Pose Estimation** → Extract human keypoint data
+3. **Temporal Downsampling** → Reduce frame rate according to settings
+4. **AI Analysis** → Use VLM/LLM to analyze pose features
+5. **Python Processing** → Run custom analysis scripts
+6. **Result Generation** → Produce a comprehensive analysis report
+7. **Scoring** → User rates output quality (e.g., 1–5)
 
-## 使用方法
+## Usage
 
-1. 上传视频文件
-2. 设置下采样率 (1-10)
-3. 输入ChatGPT分析提示词
-4. 点击"开始处理"
-5. 在不同标签页查看结果
+1. Upload video file(s)
+2. Configure downsampling rate (e.g., 1–10)
+3. Click “Start Processing”
+4. View results in different tabs (pose visualization, analysis report, charts, etc.)
+5. Provide feedback score for the outputs
 
-## 技术栈
+## TODO
 
-- **前端**: Gradio Web UI
-- **视频处理**: OpenCV
-- **数据处理**: NumPy
-- **AI集成**: ChatGPT API (待集成)
-- **部署平台**: Hugging Face Spaces
+- **Accelerate Pose Estimation**  
+  - Optimize model inference (e.g., model pruning/quantization, GPU/CPU parallelism)  
+  - Batch processing for multiple videos or frames  
+  - Investigate lightweight architectures or delegate to hardware accelerators  
 
-## 开发说明
+- **Local Deployment of VLMs**  
+  - Documentation for downloading and setting up VLM weights locally  
+  - Instructions for environment configuration (dependencies, hardware requirements)  
+  - Offline inference capabilities and fallback strategies  
+  - Security considerations for storing API keys or model files  
 
-当前版本使用模拟数据进行演示。实际部署时需要：
+- **Support Multiple Video Formats**  
+  - Automatic compatibility check and conversion (e.g., mp4, avi, mov, webm)  
+  - Integrate ffmpeg (or similar) for on-the-fly format handling  
+  - Graceful fallback or user guidance when format is unsupported  
 
-1. 集成真实的Pose Estimation模型 (如MediaPipe, OpenPose等)
-2. 配置ChatGPT API密钥
-3. 实现具体的Python分析程序
-4. 添加错误处理和日志记录
+- **Extend Scoring & Feedback Loop**  
+  - Store user scores along with video metadata  
+  - Use scores to fine-tune or adjust analysis parameters over time  
 
-## 部署到Hugging Face
+- **Support Different Language**  
+  - Use different language prompts for different language
+  - Update prompts for stable language-control 
 
-1. 创建新的Space
-2. 选择Gradio SDK
-3. 上传所有文件
-4. 配置环境变量 (如API密钥)
-5. 启动应用
 
-## 许可证
 
-MIT License
+## License
+
+MIT License

WHAM

@@ -0,0 +1 @@
+Subproject commit 2b54f7797391c94876848b905ed875b154c4a295

app.py

@@ -2,288 +2,560 @@ import gradio as gr
 import numpy as np
 import cv2
 import json
-import time
-from typing import Tuple, Dict, Any
-import tempfile
+import subprocess
 import os
+from typing import Tuple, List, Dict, Any
+import tempfile
+from transformers import AutoTokenizer, AutoModelForCausalLM, AutoProcessor
+import torch
+from PIL import Image
+import datetime
+import uuid
+from PIL import Image
+import io, base64
+
+# Use HuggingFace remote inference
+try:
+    from huggingface_hub import InferenceClient
+except ImportError:
+    InferenceClient = None
 
 class PoseEstimationApp:
-    def __init__(self):
+    def __init__(self, model_name: str = "meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8", use_remote: bool = True, max_history_turns: int = 50):
         self.processing_steps = [
-            "视频上传完成",
-            "开始Pose Estimation...",
-            "执行时序下采样...",
-            "调用ChatGPT分析...",
-            "运行Python程序...",
-            "生成最终结果"
+            "Video upload completed",
+            "Starting video downsampling...",
+            "Executing Pose Estimation...",
+            "Running Stage1 prompt...",
+            "Running Stage2 prompt...",
+            "Running Evaluator...",
+            "Running Stage3 prompt...",
+            "Generating final result"
         ]
-    
-    def extract_video_frames(self, video_path: str, downsample_rate: int = 2):
-        """提取视频帧并进行下采样"""
-        cap = cv2.VideoCapture(video_path)
-        frames = []
+        self.use_remote = use_remote
+        self.model_name = model_name
+        self.device = "cuda" if torch.cuda.is_available() else "cpu"
+        if not use_remote:
+            raise RuntimeError("Remote inference only supported, please set use_remote=True and provide HF_TOKEN environment variable.")
+        if InferenceClient is None:
+            raise RuntimeError("huggingface_hub not installed, please install to use remote inference.")
+        token = os.getenv("HF_TOKEN")
+        if not token:
+            raise RuntimeError("HF_TOKEN environment variable not set, please set the access token in deployment environment.")
+        try:
+            self.client = InferenceClient(model=model_name, token=token)
+        except Exception as e:
+            raise RuntimeError(f"Failed to initialize remote inference client: {e}")
+
+        # Conversation history management
+        # Use a list to store several rounds of conversation, each item is a dict containing 'role' ('user' or 'assistant') and 'content'
+        self.conversation_history: List[Dict[str, str]] = []
+        # Keep the most recent number of rounds (user+assistant), truncate when exceeded
+        self.max_history_turns = max_history_turns
+
+    def reset_history(self):
+        """
+        Clear conversation history, call when starting a new multi-turn conversation scenario.
+        """
+        self.conversation_history = []
+
+    def add_user_message(self, message: str):
+        self.conversation_history.append({"role": "user", "content": message})
+        # If exceeding maximum rounds (here a round refers to user+assistant), remove earliest rounds
+        # Calculate current entries, if len > 2 * max_history_turns, truncate the earliest two entries
+        max_items = 2 * self.max_history_turns
+        if len(self.conversation_history) > max_items:
+            # Discard the earliest two entries
+            self.conversation_history = self.conversation_history[-max_items:]
+
+    def add_assistant_message(self, message: str):
+        self.conversation_history.append({"role": "assistant", "content": message})
+        # Similarly truncate history
+        max_items = 2 * self.max_history_turns
+        if len(self.conversation_history) > max_items:
+            self.conversation_history = self.conversation_history[-max_items:]
+
+    def build_prompt_with_history(self, new_user_input: str) -> str:
+        """
+        Concatenate history rounds with current user input into a prompt string.
+        Example:
+        User: ...
+        Assistant: ...
+        User: new_user_input
+        Assistant:
+        """
+        prompt_parts = []
+        for turn in self.conversation_history:
+            if turn["role"] == "user":
+                prompt_parts.append(f"User: {turn['content']}")
+            else:
+                prompt_parts.append(f"Assistant: {turn['content']}")
+        # Add new user input, model reply will be generated at the end
+        prompt_parts.append(f"User: {new_user_input}")
+        prompt_parts.append("Assistant:")  # Guide model generation
+        full_prompt = "\n".join(prompt_parts)
+        return full_prompt
+
+    def image_to_datauri(self, img: Image.Image, max_size=640, jpeg_quality=70):
+        # 先按最长边缩放到 max_size
+        w, h = img.size
+        scale = max_size / max(w, h)
+        if scale < 1.0:
+            new_w, new_h = int(w*scale), int(h*scale)
+            img = img.resize((new_w, new_h), Image.BILINEAR)
+        # 转 JPEG
+        buffered = io.BytesIO()
+        img.save(buffered, format="JPEG", quality=jpeg_quality)
+        img_b64 = base64.b64encode(buffered.getvalue()).decode('utf-8')
+        return f"data:image/jpeg;base64,{img_b64}"
+
+
+
+    def query_llm_multimodal(self, text: str, indexed_images: list, use_history: bool = True, max_tokens: int = 1024):
+        messages = []
+        if use_history:
+            for turn in self.conversation_history:
+                messages.append({"role": turn['role'], "content": turn['content']})
+        # 先处理文字部分（如果有）
+        if text:
+            messages.append({"role": "user", "content": [{"type": "text", "text": text}]})
+            self.add_user_message(text)
+        # 逐帧添加图片输入
+        for clip_idx, (video_idx, img) in enumerate(indexed_images):
+            uri = self.image_to_datauri(img, max_size=512, jpeg_quality=60)
+            # 只对最关键的少量帧调用，或在调用前筛选
+            msg_content = [
+                {"type":"image_url","image_url": {"url": uri}},
+                {"type":"text","text":f"The {clip_idx}th image is the {video_idx+1}th frame in the video, please analyze and summarize the content. Use the original frame index ({video_idx+1}) for reminder."}
+            ]
+            messages.append({"role":"user","content":msg_content})
+            self.add_user_message(f"[IMAGE frame {video_idx}]")
+            # 如帧过多，可在此处 break，或只处理前 K 帧
+        try:
+            response = self.client.chat.completions.create(messages=messages, max_tokens=max_tokens)
+            reply = response.choices[0].message.content
+        except Exception as e:
+            raise RuntimeError(f"Multimodal inference error: {e}")
+        self.add_assistant_message(reply)
+        return reply
+
+    def query_llm(self, prompt: str, max_length: int = 2048, use_history: bool = True) -> str:
+        if use_history:
+            messages = []
+            for turn in self.conversation_history:
+                messages.append({"role": turn['role'], "content": turn['content']})
+            messages.append({"role": "user", "content": prompt})
+            self.add_user_message(prompt)
+        else:
+            messages = [{"role": "user", "content": prompt}]
+        try:
+            response = self.client.chat.completions.create(messages=messages, max_tokens=max_length)
+            reply = response.choices[0].message.content
+        except Exception as e:
+            raise RuntimeError(f"Remote inference error: {e}")
+        self.add_assistant_message(reply)
+        return reply
+
+    # Other methods remain unchanged...
+    def downsample_video(self, input_path: str, output_path: str, downsample_rate: int) -> Tuple[str, int]:
+        cap = cv2.VideoCapture(input_path)
+        if not cap.isOpened():
+            raise RuntimeError(f"Cannot open video file {input_path}")
+        fps = cap.get(cv2.CAP_PROP_FPS)
+        new_fps = max(1, int(fps / downsample_rate))
+        width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
+        height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
+        fourcc = cv2.VideoWriter_fourcc(*"mp4v")
+        out = cv2.VideoWriter(output_path, fourcc, new_fps, (width, height))
         frame_count = 0
-        
+        frames = []
         while True:
             ret, frame = cap.read()
             if not ret:
                 break
-            
-            # 根据下采样率选择帧
             if frame_count % downsample_rate == 0:
-                frames.append(frame)
-            
+                out.write(frame)
+                frames.append((frame_count, Image.fromarray(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB))))
             frame_count += 1
-        
+        cap.release() 
+        out.release()
+        return output_path, new_fps, frames
+
+    def run_pose_estimation(self, tmp_dir, video_path: str) -> str:
+        base_name = os.path.splitext(os.path.basename(video_path))[0]
+        out_dir = os.path.join(tmp_dir, "output")
+        os.makedirs(out_dir, exist_ok=True)
+        cmd = [
+            "python", "WHAM/demo.py",
+            "--video", video_path,
+            "--save_pkl",
+            "--output_pth", out_dir
+        ]
+        out_dir = os.path.join(out_dir, base_name)
+        result = subprocess.run(cmd, capture_output=True, text=True)
+        if result.returncode != 0:
+            raise RuntimeError(f"Pose Estimation failed: {result.stderr}")
+        result_path = os.path.join(out_dir, "wham_output.pkl")
+        if not os.path.exists(result_path):
+            raise FileNotFoundError(f"Result file not found: {result_path}")
+        return result_path
+
+    def load_pose_data(self, pth_path: str):
+        try:
+            data = torch.load(pth_path, map_location="cpu")
+            return data
+        except Exception as e:
+            raise RuntimeError(f"Failed to load Pose data: {e}")
+
+    def extract_frames(self, video_path: str, frame_skip: int = 1) -> List[Tuple[int, Image.Image]]:
+        """
+        Eagerly read and return all frames as a list of (frame_index, PIL.Image).
+        """
+        frames = []
+        cap = cv2.VideoCapture(video_path)
+        if not cap.isOpened():
+            return frames
+        idx = 0
+        while True:
+            ret, frame = cap.read()
+            if not ret:
+                break
+            if idx % frame_skip == 0:
+                img = Image.fromarray(cv2.cvtColor(frame, cv2.COLOR_BGR2RGB))
+                frames.append((idx, img))
+            idx += 1
         cap.release()
-        return frames, frame_count
-    
-    def mock_pose_estimation(self, frames):
-        """模拟pose estimation处理"""
-        # 这里应该是真实的pose estimation代码
-        # 目前返回模拟数据
-        pose_data = []
-        for i, frame in enumerate(frames):
-            # 模拟关键点数据 (17个关键点，每个点有x,y,confidence)
-            keypoints = np.random.rand(17, 3).tolist()
-            pose_data.append({
-                "frame_id": i,
-                "keypoints": keypoints,
-                "timestamp": i * 0.033  # 假设30fps
-            })
-        return pose_data
-    
-    def mock_chatgpt_analysis(self, pose_data, custom_prompt):
-        """模拟ChatGPT分析"""
-        # 这里应该调用实际的ChatGPT API
-        analysis = f"""
-        基于您的提示词: "{custom_prompt}"
-        
-        姿态分析结果：
-        - 检测到 {len(pose_data)} 个有效帧
-        - 主要动作模式：步行/站立
-        - 姿态稳定性：良好
-        - 运动幅度：中等
-        - 异常检测：未发现明显异常
-        
-        建议：保持当前运动模式，注意关节角度的稳定性。
+        return frames
+
+
+    def process_video(self, video_file, downsample_rate, progress=gr.Progress()):
         """
-        return analysis
-    
-    def mock_python_processing(self, chatgpt_result):
-        """模拟Python程序处理"""
-        # 这里应该运行实际的Python分析程序
-        python_output = f"""
-        执行Python分析程序...
-        
-        处理ChatGPT结果: {len(chatgpt_result)} 字符
-        计算统计指标...
-        生成可视化图表...
-        
-        结果摘要:
-        - 处理状态: 成功
-        - 分析维度: 多维度姿态分析
-        - 输出格式: JSON + 可视化
-        - 置信度: 0.87
+        修改：process_video 返回 (result_text, downsampled_video_path, downsample_rate) 三元组
+        以便界面显示视频并存储。
         """
-        return python_output
-    
-    def generate_final_result(self, pose_data, chatgpt_analysis, python_output):
-        """生成最终结果"""
-        final_result = {
-            "summary": {
-                "total_frames": len(pose_data),
-                "processing_time": "模拟处理",
-                "confidence_score": 0.87
-            },
-            "pose_analysis": "姿态数据已提取",
-            "ai_insights": chatgpt_analysis,
-            "technical_analysis": python_output,
-            "recommendations": [
-                "继续保持良好的运动姿态",
-                "注意关节角度的协调性",
-                "建议增加运动的多样性"
-            ]
-        }
-        return json.dumps(final_result, indent=2, ensure_ascii=False)
-    
-    def process_video(self, video_file, downsample_rate, custom_prompt, progress=gr.Progress()):
-        """主要的视频处理函数"""
         if video_file is None:
-            return "请先上传视频文件", "", "", ""
-        
+            return "Please upload a video file first", None, None
         try:
-            # 步骤1: 视频上传完成
-            progress(0.1, desc="视频上传完成")
-            time.sleep(1)
-            
-            # 步骤2: 提取帧和Pose Estimation
-            progress(0.3, desc="执行Pose Estimation...")
-            frames, total_frames = self.extract_video_frames(video_file, downsample_rate)
-            pose_data = self.mock_pose_estimation(frames)
-            time.sleep(2)
-            
-            # 步骤3: 时序下采样
-            progress(0.5, desc="时序下采样完成")
-            time.sleep(1)
-            
-            # 步骤4: ChatGPT分析
-            progress(0.7, desc="ChatGPT分析中...")
-            if not custom_prompt.strip():
-                custom_prompt = "请分析这个视频中的姿态特征"
-            chatgpt_result = self.mock_chatgpt_analysis(pose_data, custom_prompt)
-            time.sleep(2)
-            
-            # 步骤5: Python程序处理
-            progress(0.9, desc="运行Python分析程序...")
-            python_result = self.mock_python_processing(chatgpt_result)
-            time.sleep(1)
-            
-            # 步骤6: 生成最终结果
-            progress(1.0, desc="生成最终结果")
-            final_result = self.generate_final_result(pose_data, chatgpt_result, python_result)
-            
-            # 格式化pose数据用于显示
-            pose_summary = f"提取了 {len(pose_data)} 帧数据\n"
-            pose_summary += f"原始帧数: {total_frames}\n"
-            pose_summary += f"下采样率: {downsample_rate}\n"
-            pose_summary += f"有效关键点数: {len(pose_data[0]['keypoints']) if pose_data else 0}"
-            
-            return pose_summary, chatgpt_result, python_result, final_result
+            self.reset_history()
+
+            progress(0.1, desc=self.processing_steps[0])
+            tmp_dir = tempfile.mkdtemp()
+            if hasattr(video_file, 'name'):
+                orig_ext = os.path.splitext(video_file.name)[1]  # e.g. ".avi"
+            else:
+                orig_ext = os.path.splitext(video_file)[1]
+            input_path = os.path.join(tmp_dir, f"input{orig_ext}")
+            os.replace(video_file, input_path)
+
+
+            progress(0.2, desc=self.processing_steps[1])
+            downsampled_tmp = os.path.join(tmp_dir, "downsample.mp4")
+            downsampled_path, new_fps, frames = self.downsample_video(input_path, downsampled_tmp, downsample_rate)
             
+            print(type(frames), len(frames))
+            frr = self.extract_frames(downsampled_path)
+            print(type(frr), len(frr))
+
+            results_dir = "results"
+            os.makedirs(results_dir, exist_ok=True)
+            unique_name = datetime.datetime.now().strftime("%Y%m%d%H%M%S") + "_" + str(uuid.uuid4())[:8] + ".mp4"
+            persistent_path = os.path.join(results_dir, unique_name)
+            # 复制文件
+            import shutil
+            shutil.copyfile(downsampled_path, persistent_path)
+
+
+            progress(0.3, desc=self.processing_steps[2])
+            #pth_path = self.run_pose_estimation(tmp_dir, input_path)
+            pth_path = "C:\\Users\\fqh13\\AppData\\Local\\Temp\\tmpdy5roav4\\output\\input\\wham_output.pkl"
+            with open(pth_path, "rb") as f:
+                import joblib
+                pkl_file = joblib.load(f)
+                subjs = len(pkl_file.keys())
+                if subjs < 1:
+                    return "Failed to detect characters from the video, please update a new video with higher frame rate and .", None, None
+
+
+            #pth_path = "wham_output.pth"
+
+            # Stage1
+            progress(0.4, desc=self.processing_steps[3])
+            stage1_path = os.path.join("prompts", "stage1.txt")
+            if not os.path.exists(stage1_path):
+                raise RuntimeError("Missing prompts/stage1.txt prompt file")
+            with open(stage1_path, 'r', encoding='utf-8') as f:
+                prompt1 = f.read()
+                prompt1_1 = prompt1.split("[IMAGEFLAG]")[0].strip()
+                prompt1_2 = prompt1.split("[IMAGEFLAG]")[1].strip()
+            out_stage1_1 = self.query_llm(prompt1_1, use_history=True)
+            out_images = self.query_llm_multimodal(text="", indexed_images=frames, use_history=True)
+            out_stage1_part2 = self.query_llm(prompt1_2, use_history=True)
+            print(out_images)
+            print("-------")
+            print(out_stage1_part2)
+            # Stage2
+            progress(0.5, desc=self.processing_steps[4])
+            stage2_path = os.path.join("prompts", "stage2.txt")
+            if not os.path.exists(stage2_path):
+                raise RuntimeError("Missing prompts/stage2.txt prompt file")
+            with open(stage2_path, 'r', encoding='utf-8') as f:
+                prompt2 = f.read()
+            prompt2 = prompt2.replace("[FRAMERATE]", str(new_fps))
+            max_retries = 3
+            out_stage2 = ""
+            temp_json_path = os.path.join(tmp_dir, "temp_json.json")
+            for attempt in range(max_retries):
+                out_stage2 = self.query_llm(prompt2, use_history=True)
+                try:
+                    parsed = json.loads(out_stage2)
+                    with open(temp_json_path, 'w', encoding='utf-8') as f:
+                        json.dump(parsed, f, ensure_ascii=False, indent=2)
+                    break
+                except json.JSONDecodeError:
+                    prompt2 = "The previous output was not valid JSON. Please output only valid JSON without any extra content." + "\n" + out_stage2
+                    if attempt == max_retries - 1:
+                        with open(temp_json_path, 'w', encoding='utf-8') as f:
+                            f.write(out_stage2)
+
+            # Evaluator
+            progress(0.6, desc=self.processing_steps[5])
+            evaluator_cmd = ["python", "estimator.py", pth_path, temp_json_path]
+            result = subprocess.run(evaluator_cmd, capture_output=True, text=True)
+            if result.returncode != 0:
+                raise RuntimeError(f"Evaluator error: {result.stderr}")
+            output_txt_path = os.path.join(tmp_dir, "temp_json_output.txt")
+            with open(output_txt_path, 'r', encoding='utf-8') as f:
+                evaluator_output = f.read()
+
+            # Stage3
+            progress(0.7, desc=self.processing_steps[6])
+            stage3_path = os.path.join("prompts", "stage3.txt")
+            if not os.path.exists(stage3_path):
+                raise RuntimeError("Missing prompts/stage3.txt prompt file")
+            with open(stage3_path, 'r', encoding='utf-8') as f:
+                prompt3 = f.read()
+            prompt3 = prompt3.replace("[RESULTS]", evaluator_output)
+            out_stage3 = self.query_llm(prompt3, use_history=True)
+
+            progress(1.0, desc=self.processing_steps[7])
+            # 返回最终文本、持久化保存的视频路径、下采样率
+            return out_stage3, persistent_path, downsample_rate
         except Exception as e:
-            return f"处理出错: {str(e)}", "", "", ""
+            # 出错返回三个值，其中视频路径和下采样率为 None
+            return "Processing error: " + str(e), None, None
+
 
-# 创建应用实例
 app = PoseEstimationApp()
 
-# 创建Gradio界面
 def create_interface():
+    # 预定义两种语言下的文本
+    texts = {
+        "en": {
+            "title_md": "# 🎬 Video Pose Estimation Processing Platform",
+            "description_md": "Upload a video to downsample and perform pose estimation, combine multimodal LLM analysis to generate intelligent insights",
+            "input_settings": "## 📤 Input Settings",
+            "video_label": "Upload video file",
+            "downsample_label": "Temporal downsampling rate",
+            "downsample_info": "Take 1 frame every N frames and reduce frame rate. Higher rate runs faster, lower rate yields more accurate results.",
+            "process_btn": "🚀 Start Processing",
+            "clear_btn": "🔄 Clear",
+            "results_md": "## 📊 Processing Results",
+            "final_tab": "Final Result",
+            "final_label": "Final Comprehensive Result",
+            "rating_label": "Please rate the result (1–5):",
+            "submit_rating_btn": "Submit Rating",
+            "thankyou_msg": "Thank you for your feedback!",
+            "instructions_md": """
+## 💡 Instructions
+1. After uploading a video, the system will generate downsample.mp4 based on the downsampling rate.
+2. Run WHAM/demo.py for Pose Estimation; results are saved in output/<video_name>/wham_output.pth.
+3. The system will automatically read prompts/stage1.txt, stage2.txt, stage3.txt; user custom prompts are not accepted.
+4. Stage1: prompts/stage1.txt can include [POSE_SUMMARY] placeholder, auto-replaced with pose summary.
+5. Stage2: prompts/stage2.txt can include [FRAMERATE] and [STAGE1_RESULT] placeholders, auto-replaced.
+6. Prompts will be forced to output JSON format for Evaluator use.
+7. After Evaluator runs, it generates output.txt; content is automatically passed to Stage3.
+8. Deployment requires HF_TOKEN environment variable set for HuggingFace access token; code uses it automatically.
+9. Ensure project root contains prompts/stage1.txt, stage2.txt, stage3.txt and WHAM/demo.py, evaluator.py.
+"""
+        },
+        "zh": {
+            "title_md": "# 🎬 视频姿态估计处理平台",
+            "description_md": "上传视频进行降采样和姿态估计，结合多模态 LLM 分析生成智能化见解",
+            "input_settings": "## 📤 输入设置",
+            "video_label": "上传视频文件",
+            "downsample_label": "时间降采样率",
+            "downsample_info": "每隔 N 帧取 1 帧并降低帧率。更高的采样率速度更快，但精度可能下降；更低采样率更准确。",
+            "process_btn": "🚀 开始处理",
+            "clear_btn": "🔄 清除",
+            "results_md": "## 📊 处理结果",
+            "final_tab": "最终结果",
+            "final_label": "最终综合结果",
+            "rating_label": "请对结果进行评分 (1–5)：",
+            "submit_rating_btn": "提交评分",
+            "thankyou_msg": "感谢您的反馈！",
+            "instructions_md": """
+## 💡 使用说明
+1. 上传视频后，系统会根据降采样率生成 downsample.mp4。
+2. 运行 WHAM/demo.py 进行姿态估计；结果保存在 output/<video_name>/wham_output.pth。
+3. 系统会自动读取 prompts/stage1.txt、stage2.txt、stage3.txt；不接受用户自定义提示。
+4. Stage1: prompts/stage1.txt 可包含 [POSE_SUMMARY] 占位符，将被自动替换。
+5. Stage2: prompts/stage2.txt 可包含 [FRAMERATE] 和 [STAGE1_RESULT] 占位符，将被自动替换。
+6. 提示将被强制输出 JSON 格式以供 Evaluator 使用。
+7. Evaluator 运行后会生成 output.txt；内容会自动传递到 Stage3。
+8. 部署需要设置 HF_TOKEN 环境变量以获得 HuggingFace 访问令牌；代码会自动使用。
+9. 确保项目根目录下包含 prompts/stage1.txt、stage2.txt、stage3.txt 以及 WHAM/demo.py、evaluator.py。
+"""
+        }
+    }
+
     with gr.Blocks(
         theme=gr.themes.Soft(),
-        title="视频Pose Estimation处理平台",
+        title="Video Pose Estimation Processing Platform",
         css="""
-        .gradio-container {
-            max-width: 1200px !important;
-        }
-        .tab-nav {
-            background: linear-gradient(90deg, #667eea, #764ba2) !important;
-        }
+        .gradio-container { max-width: 1200px !important; }
+        .tab-nav { background: linear-gradient(90deg, #667eea, #764ba2) !important; }
         """
     ) as demo:
-        
-        gr.Markdown(
-            """
-            # 🎬 视频Pose Estimation处理平台
-            
-            上传视频文件，进行姿态估计分析，获得AI驱动的智能洞察
-            
-            **功能流程**: 视频上传 → Pose Estimation → 时序下采样 → ChatGPT分析 → Python处理 → 结果生成
-            """
-        )
-        
+        # 语言状态
+        lang_state = gr.State("en")
+        # 隐藏状态：存储最近处理的视频路径和下采样率
+        last_video_path = gr.State(None)
+        last_downsample_rate = gr.State(None)
+
+        # 语言切换按钮
+        lang_btn = gr.Button("中文")  # 初始语言 en，所以按钮文字为“中文”
+        # 头部 Markdown
+        header_md = gr.Markdown(texts["en"]["title_md"])
+        desc_md = gr.Markdown(texts["en"]["description_md"])
+
         with gr.Row():
             with gr.Column(scale=1):
-                gr.Markdown("## 📤 输入设置")
-                
-                video_input = gr.Video(
-                    label="上传视频文件",
-                    sources=["upload"],
-                    height=300
-                )
-                
+                input_md = gr.Markdown(texts["en"]["input_settings"])
+                video_input = gr.Video(label=texts["en"]["video_label"], sources=["upload"], height=300)
                 with gr.Row():
-                    downsample_rate = gr.Slider(
-                        minimum=1,
-                        maximum=10,
-                        value=2,
-                        step=1,
-                        label="时序下采样率",
-                        info="每N帧取1帧"
-                    )
-                
-                custom_prompt = gr.Textbox(
-                    label="ChatGPT提示词",
-                    placeholder="请输入分析姿态数据的提示词...",
-                    lines=3,
-                    value="请详细分析这个视频中的人体姿态特征，包括运动模式、稳定性和任何异常情况。"
-                )
-                
+                    downsample_rate = gr.Slider(minimum=1, maximum=30, value=10, step=1,
+                                               label=texts["en"]["downsample_label"],
+                                               info=texts["en"]["downsample_info"])
                 with gr.Row():
-                    process_btn = gr.Button("🚀 开始处理", variant="primary", size="lg")
-                    clear_btn = gr.Button("🔄 清除", variant="secondary")
-        
-        with gr.Column(scale=2):
-            gr.Markdown("## 📊 处理结果")
-            
-            with gr.Tabs() as tabs:
-                with gr.TabItem("姿态数据"):
-                    pose_output = gr.Textbox(
-                        label="Pose Estimation结果",
-                        lines=8,
-                        max_lines=15,
-                        placeholder="姿态数据将在此显示..."
-                    )
-                
-                with gr.TabItem("AI分析"):
-                    ai_output = gr.Textbox(
-                        label="ChatGPT分析结果",
-                        lines=8,
-                        max_lines=15,
-                        placeholder="AI分析结果将在此显示..."
-                    )
-                
-                with gr.TabItem("Python输出"):
-                    python_output = gr.Textbox(
-                        label="Python程序输出",
-                        lines=8,
-                        max_lines=15,
-                        placeholder="Python处理结果将在此显示..."
-                    )
-                
-                with gr.TabItem("最终结果"):
-                    final_output = gr.Textbox(
-                        label="综合分析结果",
-                        lines=8,
-                        max_lines=15,
-                        placeholder="最终结果将在此显示..."
-                    )
-        
-        # 事件绑定
-        process_btn.click(
-            fn=app.process_video,
-            inputs=[video_input, downsample_rate, custom_prompt],
-            outputs=[pose_output, ai_output, python_output, final_output],
-            show_progress=True
-        )
-        
-        clear_btn.click(
-            fn=lambda: (None, 2, "", "", "", "", ""),
-            outputs=[video_input, downsample_rate, custom_prompt, pose_output, ai_output, python_output, final_output]
-        )
-        
-        # 示例
-        gr.Markdown(
-            """
-            ## 💡 使用说明
-            
-            1. **上传视频**: 支持常见视频格式 (MP4, AVI, MOV等)
-            2. **设置参数**: 调整下���样率和自定义提示词
-            3. **开始处理**: 点击处理按钮，等待分析完成
-            4. **查看结果**: 在不同标签页中查看详细结果
-            
-            **注意**: 当前版本使用模拟数据，实际部署时需要集成真实的API和算法。
-            """
-        )
-    
-    return demo
+                    process_btn = gr.Button(texts["en"]["process_btn"], variant="primary", size="lg")
+                    clear_btn = gr.Button(texts["en"]["clear_btn"], variant="secondary")
+
+            with gr.Column(scale=2):
+                results_md = gr.Markdown(texts["en"]["results_md"])
+                with gr.Tabs() as tabs:
+                    with gr.TabItem(texts["en"]["final_tab"]):
+                        final_output = gr.Textbox(label=texts["en"]["final_label"], lines=12, max_lines=20)
+                        # 新增：显示降采样视频预览
+                        downsampled_video_output = gr.Video(label="Downsampled Video", interactive=False)
+                        # 新增：评分滑块和按钮
+                        rating_slider = gr.Slider(minimum=1, maximum=5, step=1,
+                                                  label=texts["en"]["rating_label"])
+                        submit_rating_btn = gr.Button(value=texts["en"]["submit_rating_btn"])
+                        # 用于显示提交后的感谢信息
+                        thankyou_text = gr.Markdown("")  # 初始为空
+
+        # 语言切换回调
+        def toggle_language(current_lang):
+            # current_lang: "en" 或 "zh"，返回新的 current_lang 以及一系列组件更新
+            new_lang = "zh" if current_lang == "en" else "en"
+            t = texts[new_lang]
+            # 更新各个组件文本
+            updates = {
+                lang_state: new_lang,
+                header_md: gr.update(value=t["title_md"]),
+                desc_md: gr.update(value=t["description_md"]),
+                input_md: gr.update(value=t["input_settings"]),
+                video_input: gr.update(label=t["video_label"]),
+                downsample_rate: gr.update(label=t["downsample_label"], info=t["downsample_info"]),
+                process_btn: gr.update(value=t["process_btn"]),
+                clear_btn: gr.update(value=t["clear_btn"]),
+                results_md: gr.update(value=t["results_md"]),
+                final_output: gr.update(label=t["final_label"]),
+                rating_slider: gr.update(label=t["rating_label"]),
+                submit_rating_btn: gr.update(value=t["submit_rating_btn"]),
+                thankyou_text: gr.update(value="")  # 切换语言时清空感谢信息
+            }
+            # 语言切换按钮文字也需更新：若当前是英文，则按钮显示“中文”，反之显示“English”
+            btn_text = "English" if new_lang == "zh" else "中文"
+            updates[lang_btn] = gr.update(value=btn_text)
+            return updates
+
+        lang_btn.click(fn=toggle_language,
+                       inputs=[lang_state],
+                       outputs=[lang_state,
+                                header_md, desc_md,
+                                input_md, video_input, downsample_rate, process_btn, clear_btn,
+                                results_md, final_output, rating_slider, submit_rating_btn, thankyou_text,
+                                lang_btn])
+
+        # 处理视频的回调：process_video 返回 (result_text, video_path, downsample_rate)
+        def on_process(video, rate):
+            result_text, video_path, dr = app.process_video(video, rate)
+            # 更新状态
+            # 如果成功，video_path 不为 None
+            return result_text, video_path, dr, gr.update(value=None), gr.update(value=None), gr.update(value="")
+
+        # 注意：outputs 顺序对应 on_process 返回值
+        # outputs: final_output (文本), downsampled_video_output (视频), last_video_path (state), last_downsample_rate (state), rating_slider (复位), thankyou_text (清空)
+        process_btn.click(fn=on_process,
+                          inputs=[video_input, downsample_rate],
+                          outputs=[final_output, downsampled_video_output,
+                                   last_video_path, last_downsample_rate,
+                                   rating_slider, thankyou_text])
+
+        # 清除按钮：重置所有
+        def on_clear():
+            return None, 10, None, None, gr.update(value=10), gr.update(value=""), "中文" if lang_state.value=="en" else "English"
+            # 返回顺序：video_input, downsample_rate, last_video_path, last_downsample_rate, rating_slider, thankyou_text, lang_btn
+        clear_btn.click(fn=on_clear,
+                        outputs=[video_input, downsample_rate,
+                                 last_video_path, last_downsample_rate,
+                                 rating_slider, thankyou_text, lang_btn])
+
+        # 提交评分回调：读取 last_video_path, last_downsample_rate, rating_slider.value
+        def save_rating(video_path, dr, rating, current_lang):
+            # video_path 可能为 None，需检查
+            if (video_path is None) or (dr is None):
+                # 没有有效的视频，忽略
+                msg = texts[current_lang]["thankyou_msg"]
+                return msg
+            # 确保 results_dir 中存在 ratings.json
+            ratings_file = "results/ratings.json"
+            os.makedirs("results", exist_ok=True)
+            record = {
+                "timestamp": datetime.datetime.now().isoformat(),
+                "video_path": video_path,
+                "downsample_rate": dr,
+                "rating": int(rating)
+            }
+            # 如果文件存在，先读，然后 append；否则新建
+            try:
+                if os.path.exists(ratings_file):
+                    with open(ratings_file, "r", encoding="utf-8") as f:
+                        data = json.load(f)
+                    if not isinstance(data, list):
+                        data = []
+                else:
+                    data = []
+            except Exception:
+                data = []
+            data.append(record)
+            # 写回
+            with open(ratings_file, "w", encoding="utf-8") as f:
+                json.dump(data, f, ensure_ascii=False, indent=2)
+            # 返回感谢信息
+            return texts[current_lang]["thankyou_msg"]
+
+        # 绑定评分提交按钮
+        submit_rating_btn.click(fn=save_rating,
+                                inputs=[last_video_path, last_downsample_rate, rating_slider, lang_state],
+                                outputs=[thankyou_text])
+
+        # 底部说明
+        instructions_md = gr.Markdown(texts["en"]["instructions_md"])
+        # 当切换语言时，上面 toggle_language 已更新 instructions_md
+        return demo
 
-# 创建并启动应用
 if __name__ == "__main__":
     demo = create_interface()
-    demo.launch(
-        server_name="0.0.0.0",
-        server_port=7860,
-        share=True,
-        show_error=True
-    )
+    demo.launch(server_name="0.0.0.0", server_port=7860, share=True, show_error=True)

estimator.py

@@ -0,0 +1,408 @@
+from typing import Tuple, List
+import torch
+import torch.nn.functional as F
+from smplx.lbs import batch_rodrigues
+
+import json
+from typing import Dict
+import numpy as np
+import joblib
+import sys
+import argparse
+
+def get_rotated_axes(global_orient):
+    """
+    输入:
+        global_orient: [T, 3] numpy array (axis-angle)
+    输出:
+        rotated_axes: dict of [T, 3] numpy arrays for X, Y, Z
+    """
+    R = batch_rodrigues(torch.tensor(global_orient).float())  # [T, 3, 3]
+
+    # 局部单位坐标轴
+    x_local = torch.tensor([1.0, 0.0, 0.0])  # X轴：右→左
+    y_local = torch.tensor([0.0, 1.0, 0.0])  # Y轴：下→上
+    z_local = torch.tensor([0.0, 0.0, 1.0])  # Z轴：后→前
+
+    # 应用旋转
+    x_world = torch.matmul(R, x_local)  # [T, 3]
+    y_world = torch.matmul(R, y_local)
+    z_world = torch.matmul(R, z_local)
+
+    return {
+        'x': x_world.numpy(),
+        'y': y_world.numpy(),
+        'z': z_world.numpy()
+    }
+
+
+
+class BaseEvaluator:
+    def __init__(self):
+        pass
+
+    def evaluate(self, joints: torch.Tensor, **kwargs) -> torch.Tensor:
+        raise NotImplementedError
+
+
+class ThreeJointAngleEvaluator(BaseEvaluator):
+    def __init__(self, joint_indices: Tuple[int, int, int], threshold: float, greater_than: bool = True):
+        super().__init__()
+        self.a, self.b, self.c = joint_indices
+        self.threshold = threshold
+        self.greater_than = greater_than
+
+    def evaluate(self, joints: torch.Tensor, **kwargs) -> torch.Tensor:
+        ba = F.normalize(joints[:, self.a] - joints[:, self.b], dim=-1)
+        bc = F.normalize(joints[:, self.c] - joints[:, self.b], dim=-1)
+        cos_angle = (ba * bc).sum(dim=-1).clamp(-1.0, 1.0)
+        angles = torch.acos(cos_angle) * 180.0 / torch.pi
+        return angles > self.threshold if self.greater_than else angles < self.threshold
+
+
+class VectorAngleEvaluator(BaseEvaluator):
+    def __init__(self, pair1: Tuple[int, int], pair2: Tuple[int, int], threshold: float, less_than=True):
+        super().__init__()
+        self.a1, self.a2 = pair1
+        self.b1, self.b2 = pair2
+        self.threshold = threshold
+        self.less_than = less_than
+
+    def evaluate(self, joints: torch.Tensor, **kwargs) -> torch.Tensor:
+        v1 = F.normalize(joints[:, self.a1] - joints[:, self.a2], dim=-1)
+        v2 = F.normalize(joints[:, self.b1] - joints[:, self.b2], dim=-1)
+        angle = torch.acos((v1 * v2).sum(dim=-1).clamp(-1.0, 1.0)) * 180.0 / torch.pi
+        return angle < self.threshold if self.less_than else angle > self.threshold
+
+
+class SingleAxisComparisonEvaluator(BaseEvaluator):
+    def __init__(self, joint_a: int, joint_b: int, axis: str, greater_than=True):
+        super().__init__()
+        self.joint_a = joint_a
+        self.joint_b = joint_b
+        self.axis = axis
+        self.greater_than = greater_than
+
+    def evaluate(self, joints: torch.Tensor, global_orient, **kwargs) -> torch.Tensor:
+        T = joints.shape[0]
+        assert self.axis in ["x", "y", "z"]
+        rotated_axes = get_rotated_axes(global_orient)
+        assert rotated_axes[self.axis].shape == (T, 3)
+
+        axis_tensor = torch.tensor(rotated_axes[self.axis], dtype=joints.dtype, device=joints.device)  # [T, 3]
+
+        vec_a = joints[:, self.joint_a, :]  # [T, 3]
+        vec_b = joints[:, self.joint_b, :]  # [T, 3]
+
+        # 投影到当前帧坐标轴方向
+        a_proj = torch.sum(vec_a * axis_tensor, dim=1)  # [T]
+        b_proj = torch.sum(vec_b * axis_tensor, dim=1)  # [T]
+
+        return a_proj > b_proj if self.greater_than else a_proj < b_proj
+
+
+class JointDistanceEvaluator(BaseEvaluator):
+    def __init__(self, joint_a: int, joint_b: int, threshold: float, greater_than=True):
+        super().__init__()
+        self.joint_a = joint_a
+        self.joint_b = joint_b
+        self.threshold = threshold
+        self.greater_than = greater_than
+
+    def evaluate(self, joints: torch.Tensor, **kwargs) -> torch.Tensor:
+        dist = torch.norm(joints[:, self.joint_a] - joints[:, self.joint_b], dim=-1)
+        return dist > self.threshold if self.greater_than else dist < self.threshold
+
+
+class RelativeOffsetDirectionEvaluator(BaseEvaluator):
+    def __init__(self, joint_a: int, joint_b: int, axis: str, threshold: float, greater_than=True):
+        super().__init__()
+        assert axis in ["x", "y", "z"]
+        self.joint_a = joint_a
+        self.joint_b = joint_b
+        self.axis = axis
+        self.threshold = threshold
+        self.greater_than = greater_than
+
+    def evaluate(self, joints: torch.Tensor, global_orient, **kwargs) -> torch.Tensor:
+        T = joints.shape[0]
+        rotated_axes = get_rotated_axes(global_orient)
+        assert self.axis in rotated_axes
+        assert rotated_axes[self.axis].shape == (T, 3)
+
+        axis_tensor = torch.tensor(rotated_axes[self.axis], dtype=joints.dtype, device=joints.device)  # [T, 3]
+
+        offset_vec = joints[:, self.joint_a, :] - joints[:, self.joint_b, :]  # [T, 3]
+        projection = torch.sum(offset_vec * axis_tensor, dim=1)  # [T]
+
+        return projection > self.threshold if self.greater_than else projection < self.threshold
+
+
+class VelocityThresholdEvaluator(BaseEvaluator):
+    def __init__(self, joint: int, axis: str, threshold: float, greater_than=True):
+        super().__init__()
+        assert axis in ["x", "y", "z"]
+        self.joint = joint
+        self.axis = axis
+        self.threshold = threshold
+        self.greater_than = greater_than
+
+    def evaluate(self, joints: torch.Tensor, global_orient, dt: float = 1.0, **kwargs) -> torch.Tensor:
+        T = joints.shape[0]
+        rotated_axes = get_rotated_axes(global_orient)
+        assert self.axis in rotated_axes
+        assert rotated_axes[self.axis].shape == (T, 3)
+
+        axis_tensor = torch.tensor(rotated_axes[self.axis], dtype=joints.dtype, device=joints.device)  # [T, 3]
+
+        # 对关节位置沿当前坐标轴进行投影
+        joint_pos = joints[:, self.joint, :]  # [T, 3]
+        projection = torch.sum(joint_pos * axis_tensor, dim=1)  # [T]
+
+        # 计算速度（时间差分）
+        velocity = (projection[1:] - projection[:-1]) / dt  # [T-1]
+
+        # 比较阈值
+        result = velocity > self.threshold if self.greater_than else velocity < self.threshold
+
+        # 补齐长度
+        return result
+
+
+class AccelerationThresholdEvaluator(BaseEvaluator):
+    def __init__(self, joint: int, axis: str, threshold: float, greater_than=True):
+        super().__init__()
+        assert axis in ["x", "y", "z"]
+        self.joint = joint
+        self.axis = axis
+        self.threshold = threshold
+        self.greater_than = greater_than
+
+    def evaluate(self, joints: torch.Tensor, global_orient, dt: float = 1.0, **kwargs) -> torch.Tensor:
+        T = joints.shape[0]
+        rotated_axes = get_rotated_axes(global_orient)
+        assert self.axis in rotated_axes
+        assert rotated_axes[self.axis].shape == (T, 3)
+
+        axis_tensor = torch.tensor(rotated_axes[self.axis], dtype=joints.dtype, device=joints.device)  # [T, 3]
+        joint_pos = joints[:, self.joint, :]  # [T, 3]
+        projection = torch.sum(joint_pos * axis_tensor, dim=1)  # [T]
+
+        velocity = (projection[1:] - projection[:-1]) / dt        # [T-1]
+        acceleration = (velocity[1:] - velocity[:-1]) / dt        # [T-2]
+
+        result = acceleration > self.threshold if self.greater_than else acceleration < self.threshold
+        return result  # shape: [T-2]
+
+
+class AngleRangeEvaluator(BaseEvaluator):
+    def __init__(self, joint_indices: Tuple[int, int, int], threshold: float, greater_than=True):
+        super().__init__()
+        self.a, self.b, self.c = joint_indices
+        self.threshold = threshold
+        self.greater_than = greater_than
+
+    def evaluate(self, joints: torch.Tensor, **kwargs) -> torch.Tensor:
+        ba = F.normalize(joints[:, self.a] - joints[:, self.b], dim=-1)
+        bc = F.normalize(joints[:, self.c] - joints[:, self.b], dim=-1)
+        cos_angle = (ba * bc).sum(dim=-1).clamp(-1.0, 1.0)
+        angles = torch.acos(cos_angle) * 180.0 / torch.pi
+        motion_range = angles.max() - angles.min()
+        return torch.tensor([motion_range > self.threshold]) if self.greater_than else torch.tensor([motion_range < self.threshold])
+
+
+class AngleChangeEvaluator(BaseEvaluator):
+    def __init__(self, joint_indices: Tuple[int, int, int], frame1: int, frame2: int, threshold: float, greater_than=True):
+        super().__init__()
+        self.a, self.b, self.c = joint_indices
+        self.frame1 = frame1
+        self.frame2 = frame2
+        self.threshold = threshold
+        self.greater_than = greater_than
+
+    def evaluate(self, joints: torch.Tensor, **kwargs) -> torch.Tensor:
+        def compute_angle(frame_idx):
+            ba = F.normalize(joints[frame_idx, self.a] - joints[frame_idx, self.b], dim=-1)
+            bc = F.normalize(joints[frame_idx, self.c] - joints[frame_idx, self.b], dim=-1)
+            cos_angle = (ba * bc).sum().clamp(-1.0, 1.0)
+            return torch.acos(cos_angle) * 180.0 / torch.pi
+
+        angle_diff = torch.abs(compute_angle(self.frame2) - compute_angle(self.frame1))
+        return torch.tensor([angle_diff > self.threshold]) if self.greater_than else torch.tensor([angle_diff < self.threshold])
+
+
+
+# Joint name to index mapping (for SMPL 24-joint model, common names)
+SMPL_JOINT_NAMES = {
+    "pelvis": 0,
+    "left_hip": 1,
+    "right_hip": 2,
+    "spine1": 3,
+    "left_knee": 4,
+    "right_knee": 5,
+    "spine2": 6,
+    "left_ankle": 7,
+    "right_ankle": 8,
+    "spine3": 9,
+    "left_foot": 10,
+    "right_foot": 11,
+    "neck": 12,
+    "left_collar": 13,
+    "right_collar": 14,
+    "head": 15,
+    "left_shoulder": 16,
+    "right_shoulder": 17,
+    "left_elbow": 18,
+    "right_elbow": 19,
+    "left_wrist": 20,
+    "right_wrist": 21,
+    "left_hand": 22,
+    "right_hand": 23,
+}
+
+# Mapping from JSON "type" to class constructor
+EVALUATOR_CLASSES = {
+    "ThreeJointAngle": ThreeJointAngleEvaluator,
+    "VectorAngle": VectorAngleEvaluator,
+    "SingleAxisComparison": SingleAxisComparisonEvaluator,
+    "JointDistance": JointDistanceEvaluator,
+    "RelativeOffsetDirection": RelativeOffsetDirectionEvaluator,
+    "VelocityThreshold": VelocityThresholdEvaluator,
+    "AccelerationThreshold": AccelerationThresholdEvaluator,
+    "AngleRange": AngleRangeEvaluator,
+    "AngleChange": AngleChangeEvaluator,
+    # PositionRange can reuse RelativeOffset with a max-min wrapper
+}
+
+
+def get_joint_index(name: str) -> int:
+    if name not in SMPL_JOINT_NAMES:
+        raise ValueError(f"Unknown joint name: {name}")
+    return SMPL_JOINT_NAMES[name]
+
+
+def build_evaluator_from_json(json_data: Dict) -> BaseEvaluator:
+    etype = json_data["type"]
+
+
+    if etype == "ThreeJointAngle":
+        a = get_joint_index(json_data["joint_a"])
+        b = get_joint_index(json_data["joint_b"])
+        c = get_joint_index(json_data["joint_c"])
+        return ThreeJointAngleEvaluator((a, b, c), json_data["threshold"], json_data.get("greater_than", True))
+
+    elif etype == "VectorAngle":
+        a1 = get_joint_index(json_data["joint_a1"])
+        a2 = get_joint_index(json_data["joint_a2"])
+        b1 = get_joint_index(json_data["joint_b1"])
+        b2 = get_joint_index(json_data["joint_b2"])
+        return VectorAngleEvaluator((a1, a2), (b1, b2), json_data["threshold"], json_data.get("less_than", True))
+
+    elif etype == "SingleAxisComparison":
+        a = get_joint_index(json_data["joint_a"])
+        b = get_joint_index(json_data["joint_b"])
+        return SingleAxisComparisonEvaluator(a, b, json_data["axis"], json_data.get("greater_than", True))
+
+    elif etype == "JointDistance":
+        a = get_joint_index(json_data["joint_a"])
+        b = get_joint_index(json_data["joint_b"])
+        return JointDistanceEvaluator(a, b, json_data["threshold"], json_data.get("greater_than", True))
+
+    elif etype == "RelativeOffsetDirection":
+        a = get_joint_index(json_data["joint_a"])
+        b = get_joint_index(json_data["joint_b"])
+        return RelativeOffsetDirectionEvaluator(a, b, json_data["axis"], json_data["threshold"], json_data.get("greater_than", True))
+
+    elif etype == "VelocityThreshold":
+        j = get_joint_index(json_data["joint"])
+        return VelocityThresholdEvaluator(j, json_data["axis"], json_data["threshold"], json_data.get("greater_than", True))
+
+    elif etype == "AccelerationThreshold":
+        j = get_joint_index(json_data["joint"])
+        return AccelerationThresholdEvaluator(j, json_data["axis"], json_data["threshold"], json_data.get("greater_than", True))
+
+    elif etype == "AngleRange":
+        a = get_joint_index(json_data["joint_a"])
+        b = get_joint_index(json_data["joint_b"])
+        c = get_joint_index(json_data["joint_c"])
+        return AngleRangeEvaluator((a, b, c), json_data["threshold"], json_data.get("greater_than", True))
+
+    elif etype == "AngleChange":
+        a = get_joint_index(json_data["joint_a"])
+        b = get_joint_index(json_data["joint_b"])
+        c = get_joint_index(json_data["joint_c"])
+        return AngleChangeEvaluator((a, b, c), json_data["frame1"], json_data["frame2"], json_data["threshold"], json_data.get("greater_than", True))
+
+    else:
+        raise ValueError(f"Unknown evaluator type: {etype}")
+
+
+
+# Main function: load motion tensor and json, return evaluation result
+def evaluate_motion_from_json(
+    json_path: str,
+    motion_tensor: torch.Tensor,
+    global_orient_tensor: torch.Tensor = None
+) -> Dict[str, List[bool]]:
+    with open(json_path, "r") as f:
+        configs = json.load(f)
+
+    results = {}
+    for idx, cfg in enumerate(configs):
+        try:
+            evalr = build_evaluator_from_json(cfg)
+        except:
+            continue
+        name = cfg.get("name", f"eval_{idx}")
+
+        # 切片
+        if "start_frame" in cfg and "end_frame" in cfg:
+            s, e = cfg["start_frame"], cfg["end_frame"]
+            seg = motion_tensor[s - 1 : e]           # [T_seg, J, 3]
+            orient_seg = None
+            if global_orient_tensor is not None:
+                orient_seg = global_orient_tensor[s - 1 : e]
+        elif "frame" in cfg:
+            # 兼容旧版
+            f0 = cfg["frame"]
+            seg = motion_tensor[f0 - 1 : f0]
+            orient_seg = None
+            if global_orient_tensor is not None:
+                orient_seg = global_orient_tensor[f0 - 1 : f0]
+        else:
+            seg = motion_tensor
+            orient_seg = None
+            if global_orient_tensor is not None:
+                orient_seg = global_orient_tensor
+
+        # 调用
+        if orient_seg is not None:
+            out = evalr.evaluate(seg, global_orient=orient_seg)
+        else:
+            out = evalr.evaluate(seg)
+        results[name] = out.tolist()
+
+    # 保存
+    out_path = json_path.replace(".json", "_output.txt")
+    with open(out_path, "w") as f:
+        json.dump(results, f)
+
+    return results
+
+if __name__ == "__main__":
+    pkl_file_path = sys.argv[1]
+    json_file_path = sys.argv[2]
+
+
+    with open(pkl_file_path, 'rb') as f:
+        pose = joblib.load(f)
+
+    global_orient = pose[0]['pose_world'][:, :3]
+    global_orient_tensor = torch.from_numpy(global_orient)
+
+    pose = pose[0]['joint'].reshape(-1, 45, 3)[:, :24, :]
+    pose = torch.from_numpy(pose)
+
+    evaluate_motion_from_json(json_file_path, pose, global_orient)
+
+    

prompts/stage1.txt

@@ -0,0 +1,18 @@
+You are a sports motion analyst and coach, helping to evaluate technical correctness of an athletic motion based on 10 sampled keyframes from a video (e.g., of a tennis backhand). I will provide you with several frame images or descriptions. When I upload all the frames, I'll tell you. Before I uploads all the frames, please wait and you can analyze and summarize the content of each picture to facilitate subsequent understanding.
+
+[IMAGEFLAG]
+
+All the frames are uploaded, please analyze the frames based on my further instructions.
+---
+
+### 🥇 Step 1: Phase Assignment
+
+Please assign each of the frames to one of several **action phases** (e.g., "ready position", "backswing", "forward swing", "contact", "follow through"). Label each frame like:
+- Frame 5: [phase]
+- Frame 10: [phase]
+...
+- Frame 45: [phase]
+
+After phase labeling, determine the **segment boundaries**: for every point where the phase changes (e.g., Frame 3→4), treat it as the **midpoint of a transition segment**. Your job is to design evaluation rules for each segment.
+
+

prompts/stage2.txt

@@ -0,0 +1,85 @@
+### 🧠 Step 2: Generate Evaluation Rules
+
+For each phase, please output a list of JSON rules that check whether this phase was performed correctly.
+
+The unit of distance is meter and the frame rate is [FRAMERATE]. Each rule must use one of the 10 allowed evaluator types (defined below), and use **SMPL joint names**. Output the rules as a JSON list, following this format:
+
+---
+
+**Coordinate System (negative → positive):**  
+- **X axis**:   right → left
+- **Y axis**:   down → up
+- **Z axis**:   back → front
+
+### 🎯 Available evaluator types:
+
+1. ThreeJointAngle  
+   → Checks angle(A, B, C)  
+   Fields: phase, start_frame, end_frame, joint_a, joint_b, joint_c, threshold, greater_than
+
+2. VectorAngle  
+   → Angle between two joint vectors  
+   Fields: phase, start_frame, end_frame, joint_a1, joint_a2, joint_b1, joint_b2, threshold, less_than
+
+3. SingleAxisComparison  
+   → Compares one joint's value vs another's along x/y/z  
+   Fields: phase, start_frame, end_frame, joint_a, joint_b, axis, greater_than
+
+4. JointDistance  
+   → Distance between two joints  
+   Fields: phase, start_frame, end_frame, joint_a, joint_b, threshold, greater_than
+
+5. RelativeOffsetDirection  
+   → Whether joint A is in front/above/etc. of joint B  
+   Fields: phase, start_frame, end_frame, joint_a, joint_b, axis, threshold, greater_than
+
+6. VelocityThreshold  
+   → Movement speed along a joint's axis  
+   Fields: phase, start_frame, end_frame, joint, axis, threshold, greater_than
+
+7. AccelerationThreshold  
+   → Change in joint velocity  
+   Fields: phase, start_frame, end_frame, joint, axis, threshold, greater_than
+
+8. AngleRange  
+   → Angle variation over time  
+   Fields: phase, start_frame, end_frame, joint_a, joint_b, joint_c, threshold, greater_than
+
+9. AngleChange  
+   → Angle difference between two specific frames  
+   Fields: phase, start_frame, end_frame, joint_a, joint_b, joint_c, frame1, frame2, threshold, greater_than
+
+10. PositionRange  
+   → Spatial displacement of a joint over time  
+   Fields: phase, start_frame, end_frame, joint, axis, threshold, greater_than
+
+Note that, for the axis, you need to select a axis from [x, y, x]. For all the distance, the unit is meter.
+---
+
+### ✅ SMPL joint names:
+
+pelvis, left_hip, right_hip, spine1, left_knee, right_knee, spine2, left_ankle, right_ankle,  
+spine3, left_foot, right_foot, neck, left_collar, right_collar, head,  
+left_shoulder, right_shoulder, left_elbow, right_elbow, left_wrist, right_wrist, left_hand, right_hand
+
+---
+
+### ✅ Output format:
+
+You MUST output only a JSON list, with no explanatory text, no markdown, no code fences. Output must start with [ and end with ].
+Example:
+[
+  {
+    "type": "ThreeJointAngle",
+    "name": "elbow_straightens",
+    "start_frame": "10",
+    "end_frame": "20",
+    "joint_a": "left_shoulder",
+    "joint_b": "left_elbow",
+    "joint_c": "left_wrist",
+    "threshold": 160,
+    "greater_than": true
+  },
+  ...
+]
+

prompts/stage3.txt

@@ -0,0 +1,12 @@
+### 🧩 Step 3: Interpret binary detection results
+After you've generated evaluation rules, I will run them through a system and return a dict of boolean results, where each key is a rule name and each value is a list of True/False results per frame.
+
+Your job:
+
+Analyze which rules failed (False in any frame).
+
+Provide concise, practical feedback about the motion technique (e.g., "Elbow should be more extended at contact", "Backswing was too short", etc.) Use clear language suitable for an athlete or coach.
+
+
+The boolean results:
+[RESULTS]

requirements.txt

@@ -1,6 +1,35 @@
+# Python 基础库
+numpy==1.24.3
+yacs
+joblib
+scikit-image
+opencv-python
+imageio[ffmpeg]
+matplotlib
+tensorboard
+smplx
+progress
+einops
+munkres
+xtcocotools>=1.8
+loguru
+tqdm
+ultralytics
+gdown==4.6.0
+smplx
 gradio>=4.0.0
-opencv-python-headless>=4.8.0
-numpy>=1.24.0
-Pillow>=9.5.0
+python-multipart>=0.0.6
 requests>=2.31.0
-python-multipart>=0.0.6
+Pillow>=9.5.0
+transformers>=4.28.0
+accelerate
+torch==2.7.0
+torchvision==0.15.2
+torchaudio
+chumpy @ git+https://github.com/mattloper/chumpy
+mmcv==1.3.9
+timm==0.4.9
+setuptools==59.5.0
+
+-e ./third-party/ViTPose
+torch-scatter @ https://data.pyg.org/whl/torch-2.7.0+cpu.html

smplx/smpl/SMPL_NEUTRAL.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4924f235e63f7c5d5b690acedf736419c2edb846a2d69fc0956169615fa75688
+size 247186228

tracking_results.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:aac30cfd9715393f0d600a72c27687ff3866c9a7969cc0d3b8dfc93a84642d82
+size 66