@patrick_oshag: Neil Movva ([@neilmovva](https://x.com/neilmovva)) started his career at Nvidia, working on GPUs and kernels, and has an…
Patrick OShaughnessy
@patrick_oshag
Neil Movva (@neilmovva) started his career at Nvidia, working on GPUs and kernels, and has an unusually deep understanding of inference, from software to chips to power.
We spend a lot of time on each of those layers, how they connect, and where the important tradeoffs are.
What makes this conversation special is how detailed it is (like a 401-level class), yet Neil makes it remarkably clear and easy to follow.
Today he runs Sail Research, a company building infrastructure for agents to make tokens as cheap as possible.
We discuss:
- Latency versus throughput
- Why there are no bad chips, only bad pricing
- The end of kernel engineering
- Buying chips and power no one else wants
- New chip architectures
- Nvidia lore + his contrarian view of the company
- Open source and the frontier labs
I learned a ton. Enjoy!
TIMESTAMPS
0:00 Intro
0:38 Building a “Token Factory”
4:21 The Future of Background Agents
13:09 Nvidia and the GPU Stack
23:27 Chips, Memory, and Transformers
36:14 The Future of AI Training Data
44:32 Chip Scarcity and Compute Arbitrage
52:44 Reinventing the AI Data Center
59:01 Power and the “Scavenger Strategy”
1:10:10 Open vs. Closed AI
💬 129 · 🔁 1.2K · ❤️ 9.7K · 👁 3.9M
7:25 PM · Aug 25, 2026 · View on X
Thread (1 more)
Patrick OShaughnessy @patrick_oshag
Replies (94)
Abdu Sindarov @AbduSindarov
@grok breakdown what concepts r discussed in eli5 manner. Explain basics first
Kumairu Aketo @kumairuAketo
i don't usually watch podcasts at night but this is so addicting I just can't stop. His ability to explain such complex ideas in such a simple way is truly remarkable. As an ugrad EE student, imma be ur full time stalker from now on. Looking forward to meeting you in real life.
Neko Legends @softpoo
@grok can you summarize the important parts for me in 10 bullet points, i fell asleep after the first 15 mins
Matt Logan @mcd_logan
It was so good and information dense I literally listened to it twice. I feel like I went from 201 to 301 if not 401 in the process.
Thank you for helping uncover incredible talents like @neilmovva!
sudarshan @ItzSuds
Insane pull to get Neil on so early. Unbelievable
WaspsWax™ @WaxWasps
@grok so his company basically routes requests to different LLMs, is that correct?
How can they do this cheaper than others?
Michelle Fang @michejafang
very good listen
Kraig Adams @kraigaadams
this was fantastic. feeding the transcript to my agent now
Harjiv Singh @harjivsingh
Brilliant interview enjoyed it deeply. Thanks Patrick & Neil.
Adrian Draghici @adrxdr
first time I hear @neilmovva. very articulate stuff, great listen!
Dr V @buildthefutures
I certainly get the impression Neil is explaining things logically and in an articulate and simple manner. Yet I still understood only about 2% of it. Some dudes just built different I guess.
Tarique Sha @tariquesha1
note to self:
well if @naval recommending it, this it is must watch n worth taking notes
Cryptonian @crypto292929
KINDLY AVOID CUTTING THE PAUSES BETWEEN SENTENCES. ITS SO INFO DENSE, IT WAS VERY DIFFICULT TO FOLLOW.
Patrick Rooney @patrickrooney
Bad pricing can be more easily aligned via futures markets which are a tremendous source for price discovery
Ar₿en @arben777
Harker school alumni strikes again
J.R. Storment @stormental
I started to cringe when he said "Token cost - is token cost the right thing to talk about? Absolutely." I disagree that token cost is the thing to focus on... wrong metric, not consistent between models, not tied to value, etc…. but he does recover it with "after Tokens you
Atherton Hobo @athertonhobo
@neilmovva is the goat
Lemich @Lemichan1
This is not "cheap" 😕
Magicfit @magicfitai
token factory is the right framing
Michael @mws48chch
This was so damn good. Thank you
Jiya @jiayaohe000
Love it, a really great podcast episode
AI & Robotics News @curiousaihere
That was a good conversation! This guy is smart! Great speaker will go far
Autistic Intelligence @Autistigent
what type of retarded format is this splicing together completely disjointed segments
Korra @Korradotio
Super informative.
咖啡与提示词 @JCStart7yr
A person who's gotten hands-on with everything from chips to the inference stack really has the goods when they talk about tradeoffs—way stronger than those who just mess with applications. Saved.
Germán B @gbanchio
"Models have seen the entire internet many times over at this point. And there is not a whole lot more to be done on human data from the internet. The next phase of data in my mind is model self-improvement [...] We don't benefit from getting random user interactions with AI"
Durr @durr0_0
Invest Like the Best is one of my favorite podcasts.
Jaya krishna Jasti @JayakrishnaJast
Awesome
The Idea Guy @gill_moreno
This guy is an AI from the future!
Investradler @BrijTrader
@nikesharora, from 7:10 to 8:40
aedifex @a3dif3x
@grok give me a summary, for all the chapters in this podcast.
Olivier K. ⓥ @remboli
At just 28, he has a rare gift for making the complex sound effortlessly simple
shashi rai @shashi_rai
Great interview ! Just one contrarian view , Jensen will support neoclouds till open source models are mainstream at scale and there are PC hardware to support it. Nvidia will make more if they sell hardware to individuals vs to neoclouds
Aryans @haalkidda
Open source wins mindshare, closed labs win revenue
Murda @murda_trades
What a listen. Kid is sharp as a tack
Nicolas Fonteyne @nakfonteyne
So many interesting takes here. The one on long-running bg agents vs. the ones designed to interface with us humans caught me wondering again: How many AI things do we, as humans, end up designing/optimizing for the way we understand and use AI? But we are the bottleneck! Maybe
Jacob Saxton @Saxtonlife
Amazing!
Anjal @anjalsaneen
neil possesses an extraordinary level of depth
A @buscadordverdad
Amazingly good conversation
distilled @distilledvideo
Sail Research just raised $80M (seed + Series A) at a $450M valuation to build exactly this — the buy-any-chip-anywhere, buy-any-power-anywhere arbitrage. And on the AMD point: MI300X is already renting 10-30% cheaper than H100 on-demand, which is the mispricing he's describing
Abeshek Narayan @abeshek452000
Clarity of thought is clarity of speech, amazing to witness such flow in explaining frontier tech
Annie Hage @annie_hage
44:00 about GB300 rack ->
"if you want to lowest possible latency and highest possible throughput you should probably be using that chip as of right now" ✍️
Jack @scheifly
Harker 🌲
TLDR @theJosh___
💡
Venix @0xVenix
I didn't expect the video to have so much useful info
tyler @imtyl3r_
Excellent talk! Thank you.
FastCortexAI @fastcortex_ai
We saw Neil give a very helpful and detailed presentation at @agihouse_org this past weekend. It's great to have another chance to hear more from him
David @opeoluwadav1
This is educative. He broke this down quite simply and still detailed.
NotaDEV @leononrails
an nvidia guy with a contrarian view on nvidia is the only nvidia take worth 70 minutes
Ajinkya @ajinkya_ml
Fantastic
Crispaul @caobanya
Gonna bookmark this for when I hopefully eventually start going through my bookmarks
sghosh @ghosh121
Great episode. A wild idea I got after listening to it, is sooner or later, there will be exchanges for compute. Like IEX in India. With DC offering rates buyers giving bids. And the exchange matching it in real time, so that the need for compute & its availability is matched.
AI Mastery Guide @aiseomastery
Software to power, full stack view
Kavin Anand @kavin_anand
love to see it. lots of gems here.
“it’s an open race to figure out who can program a whole rack scale computer as efficiently as possible”. the sail ethos on winning in throughput and speed :)
santi @santiayz
Somehow the guests just get more and more impressive, bravo Patrick
Aquib @BaigAquib
PNDRN.ARND @PndrnArnd
Awesome thanks
no mans land @nomanslandtsts
Cheap tokens are a systems outcome: kernel efficiency, memory movement, power pricing, and workload scheduling all shape the final cost.
Varsityblues3540 @Varsityblu88422
Deport.
CryptoPatrick @cryptopatrick
Excellent interview!
I do feel that Neil kind of glossed over the complexity of things at the 09:55 minute mark:
"The long lens view to take on this is that we have a form of intelligence that can tackle any verifiable problem."
First: Verifiable...but when?
For many verifiable
▶
mrshootingstar @_sour__cherry_
@grok what are 5 important lessons from this podcast?
Jarret Blaze @jarretblaze
Someone make an app where you ask a question and it sounds like Neil explaining it to you
Adel Bucetta @adelbucetta
that's what makes neil such a valuable voice in our field he knows the hardware and software sides intimately.
Wesley Mouch @WesleyMouch3
Deport, deport, deport!
Billy Bob from Bunfuck, Alabama can do his job easily. He once helped his uncle change a tire on a tractor. The front, smaller one. This isn’t much different.
Harsh @HarshHolkar
Wow ...
braai engineer @BraaiEngineer
nased on this video, I tried to create an alt called autism capital, but it was alert taken by @AutismCapital
Peter James @heypeterjames
great talk, need to migrate to sail. long horizon overnight agents are the future.
Theodor Marcu @theodormarcu
Great interview!
Elite Eyeroller @clowntown75
@grok are there any Caucasian guys within the industry that have contributed similar insights?
Andrei Kozyrev @andreikzrv
Fantastic, detailed interview
Applied History_65 @history_applied
Eye opening. Profound implications.
Malika Aubakirova @MaikaThoughts
very cool @neilmovva
Fratrilogos @Tele_us_mock_us
Model determined token budget is just another opaque pricing model for industry. A token budget set by finance serve the employim, a model set budget serves the model company.
Why would a large company use your service if you're optimizing for perceived latency? Why not just OS?
Andrew @Wesley_possum
This dudes so white
Tahi @TahiGichigi
"The best latency is no latency at all."
Vinay Rao @wizardofid
Wonder what is take is on Ox Alpha/ GLM 5.3 and the 1T tokens on Huawei infra.
Dean Meyer @DeanMeyerrr
Great pod
Ombudsman4Truth @ombudsman4truth
Training cost is important. Inferece cost is important. But really AI will ultimately be measured by customer outcomes. What do customers need done, what will they pay, and can you provide that service and make a profit.. That simple
Joe Xu @JoeGXu
cheap compute? sign me up
Mustufa Khan @mustufa4socials
one of the good reasons to follow @naval is that he always posts & reposts great things on his profile. you guys did a great job with this podcast, learned lots. @patrick_oshag & @neilmovva
TimBrrrr @iamTWB_
Does Neil have a reading list?
abhi @_abhixyz
Incredibly articulate guy
Yang Fan Yun @YangFanYun
LFG @neilmovva the GOAT 🐐
Nids Pacho @PachoNids
This is a serious deep dive.
Sanchit monga @sanchitmonga22
Great pod!
Freeman Jiang @freemanjiangg
rare spiritually chinese company
Healthcare AI Guy @HealthcareAIGuy
Would love to hear Neil’s top ideas on what long horizon agents unlock in healthcare
TTD @imanpyudha
Agreed
Julian C @JulianUnlocked
The fresh perspective from newcomers beats all the hype for sure.
228 @iamLanez
Brilliant guy.
Leni @Leni3500
Thank you
Akecheta @aki_cheta
Neil Movva doesn't know shit about the current risk of the AI Funding/Debt spiral. Nice Net Income bro.
Md Abdul Based @AbdulBased5498
done
Comments (0)
No comments yet