@HappyyPablo: open sourcing Marlin-2B 🐟 a tiny VLM to extract structured information from videos Marlin is finetuned for two questio…
Shubham Sharma
@HappyyPablo
open sourcing Marlin-2B 🐟
a tiny VLM to extract structured information from videos
Marlin is finetuned for two questions devs want to ask in their videos: what is happening, and when?
Best open model in its weight class, competitive with Gemini-2.5-flash at only 2B params 🧵
💬 25 · 🔁 163 · ❤️ 1.2K · 👁 56.9K
3:49 AM · May 20, 2026 · View on X
Replies (5)
Shubham Sharma @HappyyPablo
Marlin was trained on two modes:
1. marlin.caption() returns a structured Scene + Events JSON with second-precise timestamps.
You can use it to caption ig reels, index a video library or give your agent context of what happened and when in a video feed
Shubham Sharma @HappyyPablo
2. marlin.find() returns (start, end) timestamps for any natural-language query over a video.
fast enough to run inline in an agent loop, you can use it to locate sub-second moments in videos
Shubham Sharma @HappyyPablo
strongest open model in its weight class on dense captioning (DREAM-1K, CaReBench), beating Tarsier-2 7B/34B and competitive with Gemini-2.5-flash.
On temporal grounding (TimeLens-Bench) it sits a few points behind TimeLens-8B and close to Gemini-2.5-pro.
Shubham Sharma @HappyyPablo
Back story: @rethinkNow348 and I were building video editing agents and using Gemini-2.5-Flash to parse ig reels for events. Kept hitting content filters on benign clips so we put a month and two H100s into this side quest and this is the result :)
Shubham Sharma @HappyyPablo
🤗 Model: http://huggingface.co/NemoStation/Marlin-2B…
🎮 Hosted Demo: http://vlm.nemostation.com
Training recipe + a new dense-captioning/grounding benchmark dropping soon!!
Comments (0)
No comments yet