@composio: We ran DeepSeek V4 Flash through 4 more agent harnesses (Hermes Agent, Pi Agent, Prime Agent, Deep Agents) on 30 challen…

Here’s how the 4 new harnesses scored: - Pi Agent passed 20 of 30 tasks. - Deep Agents (by LangChain) passed 16 of 30 tasks. - Hermes Agent passed 15 of 30 tasks. - Prime Agent passed 15 of its 24 valid runs (6 runs were excluded; more on that later). Here are the final scores across all 8 harnesses (we covered Codex, Claude Code, OpenCode, and OMP in a previous thread a few days ago):

Here is the estimated cost per successful task at current API rates: - Pi Agent: $0.028 per successful task - Deep Agents: $0.045 per successful task - Hermes: $0.056 per successful task - Prime Agent: $0.131 per successful task And how all 8 harnesses compare:

..and the median time it took to complete a task: - Pi Agent: 132.2s - Hermes Agent: 175.5s - Deep Agent: 187.1s - Prime Agent: 242.1s Claude Code and OpenCode were faster than Pi here in the overall ranking:

Note: we excluded 6 Prime Agent runs our grader could not score. On 2 of them the verifier timed out against Prime's giant sessions (up to 3.5M tokens and 33 tool calls), and 4 never recorded. Prime ran the heaviest sessions of all 8 harnesses, which is worth noting.

Conclusion: The same model delivered 47–67% task success, cost $0.019–$0.104 per task, and took 122.7–272.4s median time per task depending on the harness. Benchmark the model-harness pair you’ll actually use, not the model in isolation.

@thdxr Fuck you

It's curious that Pi with no sub-agents, no durable state record, no task tracking and no robust compaction is able to outperform Pi with those things [OMP]. I wonder what the eval traces say.

OMP lovers in tears right now @zekramu

so OpenCode's still bad

Interesting numbers. 🤔 I’ve been running Hermes with multiple specialized profiles instead of one general agent. Curious how that kind of setup would score on these benchmarks versus a single harness.

interesting why is there such a gap betwee omp and pi

Grok build?

I am surprised Hermes is that low ? @Teknium o_O

oh-my-pi is the most unserious dev tool I think I've ever seen. At this point it's starting to feel like an art piece. It's honestly incredible.

btw just found this

Oh wow, I got quoted by Theo. Great day! :D

For real though, OMP is a great tool with a lot of options. Snap compaction is just one small thing that I'm not even sure is the default?

@davis7, save me! lol

Ben agrees with me that OMP is slop. Liking pi almost inherently requires that you hate OMP

insufferable take. omp is anything but slop and the team does not deserve this disrespect by a big account. omp is the best harness available today

Oh it did that because after installing it printed the ENTIRE CHANGELOG FROM DAY 1 and hosed my terminal. I'm sorry but this is clearly slop and I will be removing it from my machine immediately.

you only hate it because theres a handful of people that troll you on here that also happen use it

I can only name one OMP user and I actually quite like him. I hate OMP purely for the fact that it is slop.

focus on your work, why shitting on others while you haven't used it? I don't hate you, I find some of your thoughts are very insightful & creative, but not this one you're doing exactly what you told people to avoid

I would never refer to my statements about OMP as "feedback". That would be a pathetic attempt to defend the rudeness of my comments. I just genuinely don't like the thing and I'm tired of having to explain why I don't like the insane things it does

That’s how I feel about mcp

The recent version of MCP is significantly better and does actually help a lot! It was hard to take seriously when it was stateful

the install script alone is a masterpiece of dark patterns

It froze ghostty on my 128 GB MacBook Pro TWICE during setup 🙃

explain

There are 37 em dashes on this page

i think it has interesting idea, and no hate to whoever uses it or its creator, but i think it suffers from stuffing a ton of stuff just because some people want it

Legit. I read the repo, and just thought to myself, why the fuck are you molesting Pi like this

I thought I was insane for not liking it

If by "unserious" you mean "the best", then i fully agree.

It's crazy saying this while quoting an actually very interesting and useful thing in it OwO

It needs work out of the box like any other omp tool, but it’s great to try different things out. I have some great results using it with ds 4.1

The only “oh my” project that’s ever been worth it is @justsisyphus ohmyopencode.

Agreed, it's sooo slow and slows down my profile so bad

Omp may be slop and this idea sounds bat shit insane, but there is actual research behind this. We don't know if that's what openAI does with its server side compaction, but we know there are other ways than asking a model to summarize

I swear I downloaded omp it was so bloated in every single way that I almost killed myself

i an curious about prewalk though that seems like a cool feature but haven’t tried it because pi solves all of my needs

this is art. the snap compaction bit is the most unserious genius thing ever

it just bugs me that the chat bar isn't fixed when i scroll. i've started copy/pasting the output into another editor so i can go line by line and address everything. maybe i'm just stupid.

is incredible for some time now. I'm amazed that it is not more popular than it is as of now.

OMP is garbage software the moment it scores lower than bare metal Pi on ALL benchmarks. It was already proven.

can you provide omp support in t3code

Been shipping with OMP lately. Learned a lot from this. Thank you.

Omp is like the opposite of pi, let’s call it 3.141592653

t3 code is the most unserious dev tool i think ive ever seen. at this point its starting to feel like an art piece. Its honestly incredible

it’s why the bitter lesson is called the bitter lesson people always think their smart tricks can outperform a simpler approach of leveraging compute

why do you hate omp so much lol snap compaction works BRILLIANTLY (at least for non OpenAI models), and it is kind of incredible how much usage you can save on subscriptions using omp. on a pro 5x codex plan, I saw my usage go from 100->80% in 1h45mins on Codex, meanwhile on

You don't get it, is what I'll say, omp has a Very different philosophy from pi, omp2 doesn't even have an inch of pi. their philosophy believes in adaptability (multiple options) and tooling. ofc they benchmark they're product features they don't just release thing willy nilly

I was curious about the t3code compression method, but thinking about it, it seems like t3code is something that doesn't have that kind of technology at all.

?

Jev solved compaction, context management and KV cache constraints.

omp consistently gets shit on by stock pi in literally every eval anyone has ever run. It is a meme harness that started out as some nice QoL features but now is just a bloated mess that makes the models you use slower, worse and more expensive.

Unserious?

people raved about it but sometimes you can just tell from the web page when something is slop. if you have no taste in the way you present your project to the world you obviously had no taste building it

oh my pi is still better then t3 code

it's incredibly bad

Lemmie gues t3code is the gift from god

Omp with Oss models and advisor feature really works well tho and the agent hub looks good . eats a lot of ram tho...

Pi with custom tools is the way to go. Just have everything you want nothing less nothing more

its clearly better than your app lmao

why the hate?

The context window has been reclassified as an eye test.

ngl the error messages have better lore than most protocols 🫧

I keep seeing people recommend this tool but never given my time to try it. codex and t3code is perfect for me and I don't see the need to swap out to something else. I don't even have the tokens to be as productive as I can with t3code. Lets me manage my time much better until I

thoughts on bend2?

Unserious tools that still teach a real idea usually stick longer than serious ones that bury the idea. This one feels like that.

you haven't tried osa

OMP: If you never fail, are you really trying?

what are the biggest problems with it?

haha

Is pi support coming soon to T3code?

How does the cost work out? Vision tokens are way more expensive than text tokens. Does the KV savings make up for it?

oh-my-pi is the most unserious dev tool I think I've ever seen. At this point it's starting to feel like an art piece. It's honestly incredible.

btw just found this

Oh wow, I got quoted by Theo. Great day! :D

For real though, OMP is a great tool with a lot of options. Snap compaction is just one small thing that I'm not even sure is the default?

@davis7, save me! lol

Ben agrees with me that OMP is slop. Liking pi almost inherently requires that you hate OMP

insufferable take. omp is anything but slop and the team does not deserve this disrespect by a big account. omp is the best harness available today