Hello,
A data broker was recently trying to sell a dataset to an AI lab. He priced it by the terabyte, the current standard because nobody in the industry has figured out how to calculate the data’s actual worth yet, and he shipped it without ever opening the files himself. When the lab opened it, they found about 4,000 copies of Frozen 2 buried at the bottom of the folder structure because someone padded the whole thing with Disney movies to make it look bigger.
I don’t think the broker will ever hear from the lab again, but the wild part is that this isn’t some shady back-alley operation. AI data brokering has become a proper commodity market assembled in the past two years, purely around selling all kinds of proprietary data to frontier labs. One brokerage alone is pulling in $144 million a year in revenue.
I spent the last few weeks pulling apart how this market actually works and how companies see it; let’s dig in!
StablesReward: Your AI Agent Spends, You Earn
AI agents can surf the web, read reviews, and book hotels for you. All these tasks cost money. But what if these tasks could earn you back something?
StablesReward does just that. Once you connect your agent to an MCP endpoint, it gives your agent a menu of paid tools. Every call your agent pays for earns you miles or stocks. And this is in addition to the rewards you earn anyway on your own spending across 800+ merchants.
Every purchase pays you back. The best part is that you can choose how you’d like to be rewarded. Transfer miles to airlines and hotels, or let your stocks build a portfolio from the spend you were making anyway.
Launching soon. Free to join.
How a Market Forms From Nothing
The first thing you need to understand is why this market exists at all, because just a few months ago nobody could have imagined it as a market.
For the last decade, every major AI lab trained its models on the open web. They used Wikipedia, Reddit, digitised books, old forum posts, literally anything they could scrape and feed into the machine. When GPT-3 was launched, it consumed roughly 500 billion tokens of this material, and GPT-5, three years later, consumed several trillion.
Each generation was hungrier than the last, and around 2025, consumption outpaced the available new knowledge base. Epoch AI estimated that high-quality public text will be functionally exhausted anytime between 2026 and 2032. Reddit, which every major lab has used to scrape for free, now charges roughly $60 million a year for access to its archives.
Millions of websites today have blocked crawlers entirely. And a 2024 paper showed that training models on AI-generated text, also called synthetic data, the obvious workaround when human writing runs short, causes outputs to degrade across successive generations. The researchers called it “model collapse,” which is a polite way of saying that if you feed a machine its own words often enough, it forgets how to think.
So the open web kind of completely closed as a variable training source and the labs then went looking somewhere else.
And what they found was that the richest remaining source of the data they needed had been accumulating for years inside ordinary companies’ private systems. That meant Slack archives, email threads, CRM databases, internal wikis, the operational exhaust of daily work generated as a byproduct of people doing their jobs- had been piling up in cloud storage for a decade, and nobody had ever thought it could be sold too.
This data is often called “data exhaust” because people produce it so unconsciously. Nobody sits down at their desk in the morning thinking about the training value of the Slack message they are about to send. The data is a byproduct of doing the job, the way exhaust is a byproduct of driving. You leave it behind without realising it, and it accumulates in volumes nobody inside the organisation has ever thought to measure, let alone monetise.
Hernando de Soto once captured this kind of moment in his book The Mystery of Capital, even though he was writing about something entirely different. He argued that in the developing world, families owned homes, ran businesses, and farmed plots of land but could never turn any of it into capital. The assets were real, and people lived on them and fed their families from them. But from the financial system’s perspective, they didn’t exist.
In these dead assets, wealth couldn’t participate in the broader economy because the institutional machinery to convert it into tradable instruments had never been built. Every company with five years of Slack history was pretty much in the same position until about 2024. Their employees had been generating valuable operational data as a byproduct of daily work, and it had been accumulating in cloud storage with no mechanism to price it because the market for trading it simply had not been built yet.
What is also interesting is how quickly that infrastructure appeared once demand materialised. Oil had been actively traded for more than a century before the NYMEX introduced standardised crude futures in 1983. But for AI training data, the market compressed that entire trajectory into less than 24 months.
There are now standardised deal terms with two-year exclusivity windows, thirty-day inspection periods for buyers, sixty-day payout schedules, and referral networks that pay brokers up to 50% of the transaction price.
The average deal falls somewhere between $200,000 and $300,000. If you described the mechanics of a typical transaction to someone, they would mostly guess commercial real estate or a speciality financial instrument. But they would never be able to guess five years of a roofing contractor’s Slack messages.
What the Labs Are Actually Buying
The AI labs are no less insane in their data demands, with very high specificity about what they’re actually requesting.
Brokers receive detailed intake lists from the labs specifying exactly which industries they want data from at any given moment. When I started pulling these apart, the granularity was very interesting data in itself! They were looking for Chip design verification workflows, wet-lab experiment logs, clinical trial records, and factory-floor head-cam footage.
One broker described getting requests for bizarre internal workflows of a roofing contractor or a tax firm or a freight company. And the labs want these records across every conceivable industry vertical, from aerospace to insurance to veterinary medicine. Think about what this tells you. When a company spending billions of dollars a year on training sends a broker a shopping list that specifically asks for freight companies’ and roofing contractors’ internal workflow documentation, it reveals its product roadmap.
That level of granularity isn’t just for vague ambitions about making models smarter. It is for building AI systems that can perform the actual day-to-day work of specific industries. The list of industries they target is broad enough that the implications extend to almost every white-collar profession. But the supply side of this market barely understands what any of this means.
The Quality Problem
About half of everything submitted to the major brokers gets thrown out immediately. The rejection stories are as important because they show how far sellers are from understanding what buyers actually want.
Brokers have reported receiving forty gigabytes of WhatsApp messages labelled “HIGH QUALITY CONVERSATIONS” that turned out to be one person’s entire chat history with their friends and family. They have received thirteen hours of doorbell camera footage marketed as a “real human emotion” dataset.
The submissions that look the best usually the most absurd and funny. Because the market still hasn’t figured out how to measure what makes data valuable. So sellers mostly price by the terabyte because nobody has proposed anything smarter, creating a direct incentive to make files as large as possible regardless of what’s inside. Buyers use employee headcount and years of operation as rough proxies because those are the only metrics both sides can agree on. But nobody actually captures much about the quality of the underlying material, which means the entire market is flying on vague instruments and measurements.
This is very similar to the cotton trade in the 19th century, where the merchants who worked the Liverpool Cotton Exchange had access to telegraph reports from ports across the American South, to real-time intelligence on crop yields and shipping schedules, and to demand figures from textile mills across Lancashire.
But the plantation owners who actually grew the cotton knew almost none of this. They sold at whatever price the merchant offered and had no independent way to verify whether that price was fair, and the asymmetry persisted for decades before futures contracts and public pricing mechanisms gave the growers enough visibility to negotiate on equal footing.
The data market is at the very beginning of that same arc. And the sellers here are even more disadvantaged than the cotton growers were, because at least the growers understood what cotton was used for.
How AI Labs Evaluate Data
Let me explain with a real-life example I heard from an AI broker I chatted with.
A SaaS business owner submitted ten years of internal communications spanning Slack, WhatsApp, and Discord. It was a decade’s worth of operational data from a company that was still running. On paper, that is exactly the kind of archive the frontier labs are paying millions of dollars for.
But when the broker assessed the dataset, they valued only the most recent two years. Why? Because the company had switched messaging platforms every few years, and each switch shattered the continuity that makes operational data useful for training.
What the labs need to see is how a team escalates a problem, the full arc of how a group of people actually worked through something together. When the conversation jumps from Slack to WhatsApp halfway through a project, that arc disappears. The record becomes fragmented, and fragmented records have almost no training value.
The reason they have no training value is that the labs are not paying for the messages themselves. They are paying for the behavioral patterns embedded in those messages, the accumulated record of how experienced people actually exercise judgment when the situation is ambiguous and the manual does not help.
Michael Polanyi, the Hungarian-British polymath, coined the term “tacit knowledge” in the 1950s to describe this. His classic example was riding a bicycle: you know how to do it, but you cannot write down instructions that would teach someone else, because the knowledge lives in your reflexes and your muscle memory and the only way to transfer it is to let another person spend time on the bicycle until their own body figures it out.
Organizations are full of this kind of tacit knowledge. A senior engineer at a SaaS company knows which production alert at 3am requires waking up the team and which ones can wait until morning, and that judgment has nothing to do with the severity labels in the monitoring dashboard. It comes from years of accumulated pattern recognition, from having been wrong enough times to develop an instinct that no runbook captures.
Management consulting firms have spent half a century trying to extract and codify this kind of operational judgment. McKinsey built entire practices around the effort. And the fundamental challenge was always the same: you could sit inside an organization for months, interview dozens of people, observe how they worked, and still walk away with only a fraction of what the organization actually knew how to do, because the knowledge was too deeply embedded in daily habits for anyone to articulate it clearly enough to hand over.
AI training changes this completely. A model can ingests five years of continuous internal communications from a company extract it, compress into model weights, and reproduce in a completely different context.
That is what created this market. The labs figured out that the richest source of tacit knowledge on earth is the operational data that companies generate as a byproduct of doing business, and nobody had ever thought of it as something worth buying because nobody had ever had the technology to process it at that scale.
That is also why dead startups were the natural starting point. The moral friction was zero. But the market did not stay there, since the dead company supply is limited by definition, and the labs are building toward something that requires a continuous, renewable feed of behavioral data from people who are still actively working.
So the gradient kept moving. From the archives of companies that no longer exist, to active companies selling their employees’ ongoing communications while those employees are still generating them, to a tech giant installing monitoring software on its own workforce’s computers to record the granular process of how they do their jobs.
If the next generation of AI depends on proprietary behavioral data rather than public internet text, then the competitive advantage in AI shifts in ways that might not even be visible. Right now the conversation is dominated mostly by the labs, the GPU manufacturers, the cloud providers, and the companies that won the compute race.
But compute is a scaling problem, and scaling problems can be solved with money. Behavioral data is a trust problem. You cannot buy your way into a hospital’s patient interaction history or a law firm’s thirty years of case files with a bigger check if the people who generated those records do not trust you with them. The organisations sitting on the deepest archives of tacit knowledge, the manufacturers, the financial institutions, the professional services firms with decades of accumulated operational judgment, hold something the labs cannot replicate and cannot route around. And most of them have no idea about this!
That’s all for today!
Vaidik
Token Dispatch is a daily crypto newsletter handpicked and crafted with love by human bots. If you want to reach out to 165,000 subscriber community of the Token Dispatch, you can explore the partnership opportunities with us 🙌
📩 Fill out this form to submit your details and book a meeting with us directly.
Disclaimer: This newsletter contains analysis and opinions of the author. Content is for informational purposes only, not financial advice. Trading crypto involves substantial risk - your capital is at risk. Do your own research.