It’s fascinating to read Marc Andreessen’s Software is Eating the World again in 2023. Aside from its prescience, it highlights how the frontiers between incumbent businesses and software companies have effectively been eliminated in the decade or so since it was written.
Today, we take much of what Andreessen wrote in 2011 as a given. Software has already eaten the world, and some of the then fledgling software companies that he presents in the article have since emerged as behemoth hyperscalers.
These businesses are no longer concerned with what from the traditional world gets etherealized as software on the cloud, but rather where that data resides and who is in control of it. This is becoming increasingly important in a world of big data and AI, which were all-but absent from Andreessen’s article.
Hyperscale Cloud DisruptionÂ
Global cloud infrastructure spending surged to $61.5 billion in 2022, up more than 20% from 2021. This figure is expected to increase to $93.7 billion in 2023. To put this growth in context, as the cloud continues to become more of a global utility than a luxury, it will account for an increasingly larger percentage of overall global energy demand. This figure currently stands at around 1.5% and is estimated by the International Energy Administration (IEA) to reach 10% by 2023.
In a world where everything is migrating to the cloud, one of the new frontiers where big tech companies are currently engaged is in the data center vertical. Existing providers are now in danger of being unseated by hyperscalers like Amazon, Microsoft, and Google, each with its own extensive cloud infrastructure in on-going development and expansion.
Hyperscale Market Data Disruption
Were this trend to play out in our own vertical, you can expect hyperscalers to disrupt market data providers by making their own connections to global exchanges and attempting to compete directly with incumbent players.
There are, of course, technical obstacles to this becoming a reality. Traditionally, the exclusivity of quality market data came as a consequence of these non-trivial technical overheads.
Market data consumers are incredibly demanding where latency and packet loss are concerned. In a world where data confers advantages to one participant over another, where tolerances are so fine that cable lengths have to be factored in, you can understand why the delivery of low-latency multicast market data isn’t exactly low-hanging fruit from the perspective of these companies.
While these challenges are at the bleeding-edge of financial software development, high quality, low latency market data over the cloud is becoming increasingly feasible. Ironically, the solutions to the above problems are most likely to come from the same players now at risk of being disrupted by the hyperscalers.
If hyperscale providers were to make significant inroads into this space, you can expect many existing providers to struggle. As has been the playbook with disruptions that have occurred in other verticals, incumbents are unable to compete with the sheer pricing power and vertical integration of these new hyperscale entrants.
The result is that existing business models will need to be re-thought or risk becoming completely obsolete in the worst cases. As mentioned above, while existing players possess the domain-specific expertise and experience that the hyperscalers lack, the resources that the big tech companies have at their disposal allow them to acquire whatever technology, talent, and even companies are required for this not to present a meaningful obstacle to their ambitions.
If past is prologue and the above playbook is anything to go by, then waves of initial M&A activity among the most powerful incumbents may stave off the initial threat, but many existing players will undoubtedly fall throughout these turbulent shifts.
Hyperscale Promises
As far as dxFeed is concerned, we’ve been providing our data services over the cloud since even before Andreessen’s article was published and have grown extremely adept at optimizing what can be done to wring every possible microsecond of latency out of cloud-based deployments. We’re also completely cloud agnostic, meaning that we can offer our services on any cloud environment or platform.
We view the situation as more nuanced than simply a battle of David versus Goliath. One group may currently possess the know-how, while the other has the resources and reach, but on the other side of this disruption one can expect a revolution in the access to high-quality market data, with massive opportunities for data vendors, exchanges, as well as for a host of new consumers of that data, both human and non-human.
A consequence of hyperscale disruption in other industries has been cheaper, more scalable, and more widely available services. As mentioned above, the current assortment of walled gardens in the market data industry is as much a result of the steep technical requirements as it is by design to keep outsiders out. The coming revolution in cloud-based market data is sure to alter this situation, making high-quality market data available to a much larger percentage of the global population.
The Elephant in the Room
AI looms large over all of the above considerations. It’s why we can be reasonably certain that at least one of the big cloud four will at some point make an entry into the market data space. Data will be the differentiating factor in the coming AI wars and what could be more important or valuable than market data? Especially considering that these same hyperscalers are also investing heavily in the development of AI.
With this in mind, at dxFeed we’re already working on combining developments in AI with our vast databases of proprietary market data in the search of new insights, new ways of doing things, and new products.
If giving ChatGPT access to a static snapshot of the Internet can be so revolutionary for the fields of research and content creation, what could setting the next generation of AI loose on the world’s entire extant history of market data lead to? The potential outcomes are almost too far-fetched to imagine, both in terms of the new services and modes of analysis, but also the ingenuity of individual users who will inevitably push these technologies to their very limits and get them to do things that they weren’t originally supposed to.
In Conclusion
The market data industry stands on the verge of a sea change in how market data is provided and by whom. While we’re still in the earliest innings of this coming disruption, it’s only a matter of time before hyperscale businesses make their own entries into the world of market data provision.
This will certainly lead to disruption of the existing status quo, in which many of the incumbent providers will either be acquired outright, or be forced to merge with each other in order to remain competitive.
While there are undoubtedly negative outcomes for many of the current industry players on the horizon, this disruption will lead to a great deal of innovation, and to many new business models and opportunities in the space.
AI will be a game-changer in our industry, and it will be fascinating to see how the future of this space unfolds, what the landscape will look like in ten years, and who will be the industry’s main players. As far as dxFeed is concerned, we’re not afraid of disruption, and we’re incredibly excited to be a key contributor to this new market data landscape as it continues to evolve.Â
