TimeScope: How Long Can Your Video Large Multimodal Model Go?
โข
31
AI & ML interests
The AI community building the future.
Recent Activity
View all activity
Articles
lysandreย
updated
a
dataset
about 3 hours ago
sergiopaniegoย
posted
an
update
about 6 hours ago
Post
60
Want to learn how to align a Vision Language Model (VLM) for reasoning using GRPO and TRL? ๐
๐งโ๐ณ We've got you covered!!
NEW multimodal post training recipe to align a VLM using TRL in @HuggingFace 's Cookbook.
Go to the recipe ๐https://huggingface.co/learn/cookbook/fine_tuning_vlm_grpo_trl
Powered by the latest TRL v0.20 release, this recipe shows how to teach Qwen2.5-VL-3B-Instruct to reason over images ๐
๐งโ๐ณ We've got you covered!!
NEW multimodal post training recipe to align a VLM using TRL in @HuggingFace 's Cookbook.
Go to the recipe ๐https://huggingface.co/learn/cookbook/fine_tuning_vlm_grpo_trl
Powered by the latest TRL v0.20 release, this recipe shows how to teach Qwen2.5-VL-3B-Instruct to reason over images ๐
merveย
updated
a
dataset
about 6 hours ago
Post
117
You would've implemented the 3-loop matrix multiplication many times as a ML practitioner, but the naive implementation is terrible for GPU performance. Modern GPUs achieve peak performance through careful memory access patterns and minimizing scheduling overhead.
In naive matmul (MxK . KxN), the computation happens in tiles - both for the output matrix and for how you read chunks from the input matrices. Each thread-block processes one output tile by loading corresponding tiles from input (for sum-reduction across K dimension), performing the computation, then terminating. The GPU launches many thread-blocks and schedules them across available streaming multiprocessors (SMs). When an SM finishes one tile, it gets assigned a new thread-block for the next uncomputed tile. This way, multiple output tiles are computed in parallel across the SMs, but we pay the cost for launching thread-blocks each time a new tile is computed.
Persistent matmul changes this approach. Instead of launching thread-blocks to compute some output tiles, computing the results on SMs in parallel, and repeating until all output tiles are computed, you launch only as many thread-blocks as you have SMs available (typically 80-132 on modern GPUs). These thread-blocks stay alive until all output tiles are computed, looping through multiple tiles sequentially. Each persistent thread-block may handle multiple output tiles.
The key benefit is the reduced thread-block launch latency. This persistence strategy, combined with other optimizations like coalesced memory loads/stores, block-tiling, warp-tiling, warp-specialization, double-buffering, ping-pong scheduling and other tricks, helps achieve peak performance. More on this in the future!
Code snippet for testing: https://gist.github.com/a-r-r-o-w/28339b442d164084506c0967029968a8
(Bonus: Since I've wanted to learn Manim for a while, this was a great opportunity to make a visualization for Naive VS Persistent matmul. Enjoy โจ)
In naive matmul (MxK . KxN), the computation happens in tiles - both for the output matrix and for how you read chunks from the input matrices. Each thread-block processes one output tile by loading corresponding tiles from input (for sum-reduction across K dimension), performing the computation, then terminating. The GPU launches many thread-blocks and schedules them across available streaming multiprocessors (SMs). When an SM finishes one tile, it gets assigned a new thread-block for the next uncomputed tile. This way, multiple output tiles are computed in parallel across the SMs, but we pay the cost for launching thread-blocks each time a new tile is computed.
Persistent matmul changes this approach. Instead of launching thread-blocks to compute some output tiles, computing the results on SMs in parallel, and repeating until all output tiles are computed, you launch only as many thread-blocks as you have SMs available (typically 80-132 on modern GPUs). These thread-blocks stay alive until all output tiles are computed, looping through multiple tiles sequentially. Each persistent thread-block may handle multiple output tiles.
The key benefit is the reduced thread-block launch latency. This persistence strategy, combined with other optimizations like coalesced memory loads/stores, block-tiling, warp-tiling, warp-specialization, double-buffering, ping-pong scheduling and other tricks, helps achieve peak performance. More on this in the future!
Code snippet for testing: https://gist.github.com/a-r-r-o-w/28339b442d164084506c0967029968a8
(Bonus: Since I've wanted to learn Manim for a while, this was a great opportunity to make a visualization for Naive VS Persistent matmul. Enjoy โจ)
sayakpaulย
updated
a
dataset
about 11 hours ago
sergiopaniegoย
posted
an
update
about 12 hours ago
Post
1063
Just included example scripts for aligning models using GSPO (including VLM example) ๐โโ๏ธ๐โโ๏ธ
GSPO is the latest RL alignment algo by @Alibaba_Qwen and it's already supported in the latest TRL v0.20 release.
Super-easy-to-get-started example scripts below, GO run them!๐ฉโ๐ป๐ฉโ๐ป
๐งโ๐จ Script: https://github.com/huggingface/trl/blob/main/examples/scripts/gspo.py
๐ฆ VLM script: https://github.com/huggingface/trl/blob/main/examples/scripts/gspo_vlm.py
๐งฉ More TRL examples: https://huggingface.co/docs/trl/main/en/example_overview
๐งโโ๏ธ GSPO paper: Group Sequence Policy Optimization (2507.18071)
GSPO is the latest RL alignment algo by @Alibaba_Qwen and it's already supported in the latest TRL v0.20 release.
Super-easy-to-get-started example scripts below, GO run them!๐ฉโ๐ป๐ฉโ๐ป
๐งโ๐จ Script: https://github.com/huggingface/trl/blob/main/examples/scripts/gspo.py
๐ฆ VLM script: https://github.com/huggingface/trl/blob/main/examples/scripts/gspo_vlm.py
๐งฉ More TRL examples: https://huggingface.co/docs/trl/main/en/example_overview
๐งโโ๏ธ GSPO paper: Group Sequence Policy Optimization (2507.18071)
Post
977
๐ฅ July highlights from Chinese AI community
zh-ai-community/july-2025-open-works-from-the-chinese-community-686586f1a8840797e477ae5a
โจ Another "DeepSeek moment" - Kimi K2 ๐
โจ Qwen goes fully matrixed - Instruct / Thinking / Coder models across 30B - 480B ๐คฏ
โจ The multimodal wave๐
- GLM-4.1V-Thinking: Image+Text > Text
- Intern-S1: Image+Text > Text
- Wan 2.2 - Text +Image > video
- Skywork-R1V3: Image+Text > Text
- Skywork-UniPic: Text > Image / Image > Text
- Tar-7B: Any-to-Any
- Ming-Lite-Omni-1.5: Any-to-Any
- Step3: Image+Text > Text
- HunyuanWorld-1: Image > 3D
- ThinkSound: Video > Audio
- Neta-Lumina: Text > Image
โจTiny & deployable models ๐ค
- SmallThinker runs on 1GB RAM
โจAgentic coding goes mainstream ๐ป
- Qwen3-Coder: fully spec'd tool calling
- GLM-4.5: browser agents, IDE assistant
- Qwen3 WebDev demo: text-to-frontend code
โจDomain-Specific & Utility Models/Tools/Dataset
- Science one S1: Scientific model
- Agentar DeepFinance: Finance dataset
- ObjectClear: Interactive Vision Tool
- Qwen3 MT Demo: Machine Translation Tool
โจ Big month not only for models, but for policy too๐๏ธ
- Announced Global Action Plan for AI Governance
- Proposes to set up a World AI Cooperation Organization in Shanghai
- Released International AI Open Source Collaboration Initiative
- Published Risk Assessment Guidelines for Endpoint AI Agents
โจ Big event - WAIC
- 355K offline visitors
- 108 new released in 4 days
- 145 sessions across key domains
Iโve been tracking things closely, but Julyโs open-source wave still blew me away. Canโt wait to see whatโs coming next! ๐
zh-ai-community/july-2025-open-works-from-the-chinese-community-686586f1a8840797e477ae5a
โจ Another "DeepSeek moment" - Kimi K2 ๐
โจ Qwen goes fully matrixed - Instruct / Thinking / Coder models across 30B - 480B ๐คฏ
โจ The multimodal wave๐
- GLM-4.1V-Thinking: Image+Text > Text
- Intern-S1: Image+Text > Text
- Wan 2.2 - Text +Image > video
- Skywork-R1V3: Image+Text > Text
- Skywork-UniPic: Text > Image / Image > Text
- Tar-7B: Any-to-Any
- Ming-Lite-Omni-1.5: Any-to-Any
- Step3: Image+Text > Text
- HunyuanWorld-1: Image > 3D
- ThinkSound: Video > Audio
- Neta-Lumina: Text > Image
โจTiny & deployable models ๐ค
- SmallThinker runs on 1GB RAM
โจAgentic coding goes mainstream ๐ป
- Qwen3-Coder: fully spec'd tool calling
- GLM-4.5: browser agents, IDE assistant
- Qwen3 WebDev demo: text-to-frontend code
โจDomain-Specific & Utility Models/Tools/Dataset
- Science one S1: Scientific model
- Agentar DeepFinance: Finance dataset
- ObjectClear: Interactive Vision Tool
- Qwen3 MT Demo: Machine Translation Tool
โจ Big month not only for models, but for policy too๐๏ธ
- Announced Global Action Plan for AI Governance
- Proposes to set up a World AI Cooperation Organization in Shanghai
- Released International AI Open Source Collaboration Initiative
- Published Risk Assessment Guidelines for Endpoint AI Agents
โจ Big event - WAIC
- 355K offline visitors
- 108 new released in 4 days
- 145 sessions across key domains
Iโve been tracking things closely, but Julyโs open-source wave still blew me away. Canโt wait to see whatโs coming next! ๐
Post
322
๐ค ๐พ Thanks so much to BBC News and the stellar Suranjana Tewari for having me on to talk about US <โ> China relationship in AI, and what it means for AI ethics.
Post
1503
Qwen team did it again!!
They just released Qwen3-Coder-30B-A3B-Instruct on the hub๐ฅ
Qwen/Qwen3-Coder-30B-A3B-Instruct
โจ Apache 2.0
โจ30B total / 3.3B active (128 experts, 8 top-k)
โจ Native 256K context, extendable to 1M via Yarn
โจ Built for Agentic Coding
They just released Qwen3-Coder-30B-A3B-Instruct on the hub๐ฅ
Qwen/Qwen3-Coder-30B-A3B-Instruct
โจ Apache 2.0
โจ30B total / 3.3B active (128 experts, 8 top-k)
โจ Native 256K context, extendable to 1M via Yarn
โจ Built for Agentic Coding
Post
283
Itโs here! After the WAIC announcement, StepFun has just dropped Step 3 ๐ฅ their latest multimodal reasoning model on the hub.
Paper: Step-3 is Large yet Affordable: Model-system Co-design for Cost-effective Decoding (2507.19427)
Model: stepfun-ai/step3
โจ 321B total / 32B active - Apache 2.0
โจ MFA + AFD : cutting decoding cost by up to 70% vs. DeepSeek-V3
โจ 4T image-text pretraining: strong visionโlanguage grounding
โจ Modular, efficient, deployable: runs on just 8ร48GB GPUs
Paper: Step-3 is Large yet Affordable: Model-system Co-design for Cost-effective Decoding (2507.19427)
Model: stepfun-ai/step3
โจ 321B total / 32B active - Apache 2.0
โจ MFA + AFD : cutting decoding cost by up to 70% vs. DeepSeek-V3
โจ 4T image-text pretraining: strong visionโlanguage grounding
โจ Modular, efficient, deployable: runs on just 8ร48GB GPUs
Post
136
The new
I couldn't resist writing a blog post about it:
https://huggingface.co/blog/angt/your-own-gpu-powered-image-generator-with-hf-jobs
hf jobs CLI is absolutely awesome!I couldn't resist writing a blog post about it:
https://huggingface.co/blog/angt/your-own-gpu-powered-image-generator-with-hf-jobs
IlyasMoutawwakilย
posted
an
update
5 days ago
Post
3237
๐ Optimum: The Last v1 Release ๐
Optimum v1.27 marks the final major release in the v1 series. As we close this chapter, we're laying the groundwork for a more modular and community-driven future:
- Optimum v2: A lightweight core package for porting Transformers, Diffusers, or Sentence-Transformers to specialized AI hardware/software/accelerators..
- OptimumโONNX: A dedicated package where the ONNX/ONNX Runtime ecosystem lives and evolves, faster-moving and decoupled from the Optimum core.
๐ฏ Why this matters:
- A clearer governance path for ONNX, fostering stronger community collaboration and improved developer experience..
- Enable innovation at a faster pace in a more modular, open-source environment.
๐ก What this means:
- More transparency, broader participation, and faster development driven by the community and key actors in the ONNX ecosystem (PyTorch, Microsoft, Joshua Lochner ๐, ...)
- A cleaner, more maintainable core Optimum, focused on extending HF libraries to special AI hardware/software/accelerators tooling and used by our partners (Intel Corporation, Amazon Web Services (AWS), AMD, NVIDIA, FuriosaAI, ...)
๐ ๏ธ Major updates I worked on in this release:
โ Added support for Transformers v4.53 and SmolLM3 in ONNX/ONNXRuntime.
โ Solved batched inference/generation for all supported decoder model architectures (LLMs).
โจ Big shoutout to @echarlaix for leading the refactoring work that cleanly separated ONNX exporter logic and enabled the creation of OptimumโONNX.
๐ Release Notes: https://lnkd.in/gXtE_qji
๐ฆ Optimum : https://lnkd.in/ecAezNT6
๐ Optimum-ONNX: https://lnkd.in/gzjyAjSi
#Optimum #ONNX #OpenSource #HuggingFace #Transformers #Diffusers
Optimum v1.27 marks the final major release in the v1 series. As we close this chapter, we're laying the groundwork for a more modular and community-driven future:
- Optimum v2: A lightweight core package for porting Transformers, Diffusers, or Sentence-Transformers to specialized AI hardware/software/accelerators..
- OptimumโONNX: A dedicated package where the ONNX/ONNX Runtime ecosystem lives and evolves, faster-moving and decoupled from the Optimum core.
๐ฏ Why this matters:
- A clearer governance path for ONNX, fostering stronger community collaboration and improved developer experience..
- Enable innovation at a faster pace in a more modular, open-source environment.
๐ก What this means:
- More transparency, broader participation, and faster development driven by the community and key actors in the ONNX ecosystem (PyTorch, Microsoft, Joshua Lochner ๐, ...)
- A cleaner, more maintainable core Optimum, focused on extending HF libraries to special AI hardware/software/accelerators tooling and used by our partners (Intel Corporation, Amazon Web Services (AWS), AMD, NVIDIA, FuriosaAI, ...)
๐ ๏ธ Major updates I worked on in this release:
โ Added support for Transformers v4.53 and SmolLM3 in ONNX/ONNXRuntime.
โ Solved batched inference/generation for all supported decoder model architectures (LLMs).
โจ Big shoutout to @echarlaix for leading the refactoring work that cleanly separated ONNX exporter logic and enabled the creation of OptimumโONNX.
๐ Release Notes: https://lnkd.in/gXtE_qji
๐ฆ Optimum : https://lnkd.in/ecAezNT6
๐ Optimum-ONNX: https://lnkd.in/gzjyAjSi
#Optimum #ONNX #OpenSource #HuggingFace #Transformers #Diffusers
Post
2580
We've crossed 1 million repositories backed by Xet storage on Hugging Face! ๐๐๐
You can follow along our progress converting the Hub from Git LFS to Xet at jsulz/ready-xet-go
We have a lot of repos left to migrate, which means I have plenty of time to add more animations ๐คช
You can follow along our progress converting the Hub from Git LFS to Xet at jsulz/ready-xet-go
We have a lot of repos left to migrate, which means I have plenty of time to add more animations ๐คช
Post
3481
Qwen3-30B-A3B-Thinking-2507 ๐ฅ latest step in scaling thinking capabilities from Alibaba Qwen team.
Qwen/Qwen3-30B-A3B-Thinking-2507-FP8
โจ 30B total / 3B active - Apache 2.0
โจ Native 256K context
โจ SOTA coding, alignment, agentic reasoning
Qwen/Qwen3-30B-A3B-Thinking-2507-FP8
โจ 30B total / 3B active - Apache 2.0
โจ Native 256K context
โจ SOTA coding, alignment, agentic reasoning
sergiopaniegoย
posted
an
update
5 days ago
Post
275
Did you miss this? ๐
๐งโโ๏ธvLLM + transformers integration just got upgraded with direct VLM support.
Select a VLM +
๐งโโ๏ธvLLM + transformers integration just got upgraded with direct VLM support.
Select a VLM +
model_impl=transformers and play via vLLM!
Post
2673
Skywork UniPic ๐ฅa unified autoregressive multimodal model for image understanding, generation, & editing, by Skywork ๅคฉๅทฅ
Skywork/skywork-unipic-6888c0789cdb82457b2acf32
โจ 1.5 B - MIT License
โจ Runs on RTX 4090
โจ Truly unified architecture
Skywork/skywork-unipic-6888c0789cdb82457b2acf32
โจ 1.5 B - MIT License
โจ Runs on RTX 4090
โจ Truly unified architecture
Post
1695
Qwen just released Qwen3-30B-A3B-Instruct-2507 ๐ฅ an upgrade to the non-thinking mode model
Qwen/Qwen3-30B-A3B-Instruct-2507
โจ 30B MoE / 3.3B active - Apache 2.0
โจ Strong gains in reasoning, math, coding, & multilingual tasks
โจ Native support for 256K long-context inputs
Qwen/Qwen3-30B-A3B-Instruct-2507
โจ 30B MoE / 3.3B active - Apache 2.0
โจ Strong gains in reasoning, math, coding, & multilingual tasks
โจ Native support for 256K long-context inputs
Post
2901
๐ฌ From Replika to everyday chatbots, millions of people are forming emotional bonds with AI, sometimes seeking comfort, sometimes seeking intimacy. But what happens when an AI tells you "I understand how you feel" and you actually believe it?
At Hugging Face, together with @frimelle and @yjernite , we dug into something we felt wasn't getting enough attention: the need to evaluate AI companionship behaviors. These are the subtle ways AI systems validate us, engage with us, and sometimes manipulate our emotional lives.
Here's what we found:
๐ Existing benchmarks (accuracy, helpfulness, safety) completely miss this emotional dimension.
๐ We mapped how leading AI systems actually respond to vulnerable prompts. ๐ We built the Interactions and Machine Attachment Benchmark (INTIMA): a first attempt at evaluating how models handle emotional dependency, boundaries, and attachment (with a full paper coming soon).
Check out the blog post: https://huggingface.co/blog/giadap/evaluating-companionship
๐ข We also shipped two visualization tools with Gradio to see how different models behave when things get emotionally intense:
- AI-companionship/intima-responses-2D
- giadap/INTIMA-responses
At Hugging Face, together with @frimelle and @yjernite , we dug into something we felt wasn't getting enough attention: the need to evaluate AI companionship behaviors. These are the subtle ways AI systems validate us, engage with us, and sometimes manipulate our emotional lives.
Here's what we found:
๐ Existing benchmarks (accuracy, helpfulness, safety) completely miss this emotional dimension.
๐ We mapped how leading AI systems actually respond to vulnerable prompts. ๐ We built the Interactions and Machine Attachment Benchmark (INTIMA): a first attempt at evaluating how models handle emotional dependency, boundaries, and attachment (with a full paper coming soon).
Check out the blog post: https://huggingface.co/blog/giadap/evaluating-companionship
๐ข We also shipped two visualization tools with Gradio to see how different models behave when things get emotionally intense:
- AI-companionship/intima-responses-2D
- giadap/INTIMA-responses
sergiopaniegoย
posted
an
update
6 days ago
Post
2531
We just released TRL v0.20 with major multimodal upgrades!
๐๏ธ VLM support for GRPO (highly requested by the community!)
๐๏ธ New GSPO trainer (from @Qwen , released last week, VLM-ready)
๐ New MPO trainer (multimodal by design, as in the paper)
๐ Full release notes here: https://github.com/huggingface/trl/releases/tag/v0.20.0
๐๏ธ VLM support for GRPO (highly requested by the community!)
๐๏ธ New GSPO trainer (from @Qwen , released last week, VLM-ready)
๐ New MPO trainer (multimodal by design, as in the paper)
๐ Full release notes here: https://github.com/huggingface/trl/releases/tag/v0.20.0
Post
3926
๐๐ถ๐ฟ๐๐ ๐๐ฃ๐๐ ๐ ๐ผ๐ฑ๐ฒ๐น ๐๐ถ๐๐ต ๐๐จ ๐๐ฎ๐๐ฎ ๐ง๐ฟ๐ฎ๐ป๐๐ฝ๐ฎ๐ฟ๐ฒ๐ป๐ฐ๐ ๐ง๐ฒ๐บ๐ฝ๐น๐ฎ๐๐ฒ? ๐ช๐บ
With the release of the EU data transparency template this week, we finally got to see one of the most meaningful artifacts to come out of the AI Act implementation so far (haven't you heard? AI's all about the data! ๐๐)
The impact of the template will depend on how effectively it establishes a minimum meaningful transparency standard for companies that don't otherwise offer any transparency into their handling of e.g. personal data or (anti?-)competitive practices in commercial licensing - we'll see how those play out as new models are released after August 2nd ๐
In the meantime, I wanted to see how the template works for a fully open-source + commercially viable model, so I filled it out for the SmolLM3 - which my colleagues at Hugging Face earlier this month ๐ค ICYMI, it's fully open-source with 3B parameters and performance matching the best similar-size models (I've switched all my local apps from Qwen3 to it, you should too ๐ก)
Verdict: congrats to the European Commission AI Office for making it so straightforward! Fully open and transparent models remain a cornerstone of informed regulation and governance, but the different organizational needs of their developers aren't always properly accounted for in new regulation. In this case, it took me all of two hours to fill out and publish the template (including reading the guidelines) - so kudos for making it feasible for smaller and distributed organizations ๐ Definitely a step forward for transparency ๐
To learn more have a look at:
- The SmolLM3 model: HuggingFaceTB/SmolLM3-3B
- Its filled out Public Summary of Training Content: hfmlsoc/smollm3-eu-data-transparency
- And if you're interested, some previous remarks on regulatory minimum meaningful standards for data disclosure: https://huggingface.co/blog/yjernite/naiac-data-transparency
With the release of the EU data transparency template this week, we finally got to see one of the most meaningful artifacts to come out of the AI Act implementation so far (haven't you heard? AI's all about the data! ๐๐)
The impact of the template will depend on how effectively it establishes a minimum meaningful transparency standard for companies that don't otherwise offer any transparency into their handling of e.g. personal data or (anti?-)competitive practices in commercial licensing - we'll see how those play out as new models are released after August 2nd ๐
In the meantime, I wanted to see how the template works for a fully open-source + commercially viable model, so I filled it out for the SmolLM3 - which my colleagues at Hugging Face earlier this month ๐ค ICYMI, it's fully open-source with 3B parameters and performance matching the best similar-size models (I've switched all my local apps from Qwen3 to it, you should too ๐ก)
Verdict: congrats to the European Commission AI Office for making it so straightforward! Fully open and transparent models remain a cornerstone of informed regulation and governance, but the different organizational needs of their developers aren't always properly accounted for in new regulation. In this case, it took me all of two hours to fill out and publish the template (including reading the guidelines) - so kudos for making it feasible for smaller and distributed organizations ๐ Definitely a step forward for transparency ๐
To learn more have a look at:
- The SmolLM3 model: HuggingFaceTB/SmolLM3-3B
- Its filled out Public Summary of Training Content: hfmlsoc/smollm3-eu-data-transparency
- And if you're interested, some previous remarks on regulatory minimum meaningful standards for data disclosure: https://huggingface.co/blog/yjernite/naiac-data-transparency