A Compute Upheaval Triggered by “Running Out of Quota”: Inference Demand Is Tearing a Crack in the Old Order

Over the past week, the most striking developments in the compute field were not a refresh of some chip’s specifications, nor the powering-up and commissioning of a data center, but a series of signals that appear isolated yet actually point to the same hidden thread: users began complaining that their “quotas were burning too fast,” major companies began frantically snapping up a graphics card that was not designed for data centers, and an AI company not yet public laid out its compute bills for the next decade or more before Wall Street in a prospectus. Placed on the same timeline, these signals are not random market noise but projections of the same structural change onto different links in the chain: the center of gravity of compute demand is shifting from training to inference, while the old chip supply order, cloud service pricing logic, and the cost structure of AI companies are not yet fully prepared for this shift.

Quota Anxiety: The First Alarm as Inference Demand Surfaces

On October 24, Moonshot AI released Kimi K3.1 Pro, with a parameter scale reaching the trillions and support for a 1-million-token context. The company highlighted the model’s capabilities in programming, 3D creation, and long-horizon Agent tasks. From the perspective of model capability, this was a typical “capability upgrade”: a longer context means the model can process more complex materials in one go, and stronger Agent capabilities mean the model no longer merely answers questions passively but can proactively break down tasks, call tools, and execute step by step. Yet soon after launch, some paying users found that before their tasks were finished, their quotas were almost exhausted. On October 26, Moonshot AI publicly apologized, acknowledging rate limiting and “unnecessary quota consumption,” and compensated subscribers with quota equivalent to 25% of their monthly total.

On the surface, this was a mistake at the product operations level; at a deeper level, it was a public preview of how the logic of compute consumption changes once inference demand explodes. In the traditional chatbot model, a user asks a question and the model answers, so compute consumption is relatively predictable. In long-context and Agent modes, however, what the user sees is one task, while what runs in the background may be a chain of subtasks: decomposition, retrieval, analysis, verification, summarization, and further verification. Every tool call, every round of context backfilling, and every parallel run of a sub-Agent translates into additional token consumption and compute occupancy. What users perceive is the “quota”; what the platform bears is the concurrency pressure, memory bandwidth pressure, and scheduling pressure of an inference cluster.

The problem is not unique to Kimi. Similar complaints have appeared in the Claude user community: some users reported that when Claude Opus 5.5 organized 87 text materials, it automatically launched 12 peer sub-Agents at the same level, exhausting the entire five-hour usage quota in five minutes. What the user saw was one task; what actually ran in the background was a series of parallel sub-Agent calls—decomposition, retrieval, analysis, verification, and summarization. When AI moves from “answering questions” to “executing tasks,” the logic of computing resource consumption undergoes a qualitative change. In the past, evaluating a model focused more on parameter scale, benchmark scores, and training cost; now, evaluating an AI product increasingly requires looking at token consumption per task, concurrency scheduling efficiency, inference throughput, and cost per unit task. The essence of quota anxiety is not that platforms are stingy, but that inference economics has not yet fully matured: model capabilities have run ahead of cost control, and Agent architectures have run ahead of billing systems.

This is precisely the micro-level foundation for the explosion of inference compute demand. Li Bin, senior vice president of Sugon, made an incisive judgment at CNCC 2026: “As model parameters grow and intelligent agents accelerate into deployment, compute systems are moving toward intensive and large-scale development.” Sun Jianpeng, an expert at Hygon Information, provided another piece of evidence from the perspective of model scale: “Large model parameter counts are leaping from the hundreds of billions to the trillions, even tens of trillions; single cards and single machines can no longer support large-scale training and inference.” Training demand is still growing, but the demand curve on the inference side is rising at a steeper slope, and this change has not yet been fully absorbed by the existing chip supply structure.

More specifically, inference demand has three characteristics that differ from training demand. First, inference is an ongoing expense: as long as the service is online, compute consumption does not stop; training is more like phased capital expenditure, with concentrated input and concentrated consumption. Second, inference workloads are highly fragmented. Different tasks vary greatly in context length, concurrency, response latency, and tool-calling requirements, and a single chip specification can hardly cover all cases. Third, inference is extremely cost-sensitive, because every call must be factored into the unit economics model, and platforms must repeatedly weigh model capability, response speed, and token cost. Taken together, these three points mean that inference compute is no longer merely an “appendage of training compute” but is becoming the core variable determining whether AI products can achieve commercial scale.

$35 Billion for a Card That Is “Not Top-Tier”: Rational Madness Under Supply Constraints

The pressure of inference demand travels up the industrial chain and first hits the chip supply side.

A potential order disclosed by Reuters pushed this pressure to an industrial scale: the combined order demand from ByteDance, Alibaba, and Tencent for NVIDIA’s RTX PRO 5500 exceeds 3 million units. At roughly $11,400 per unit, the potential order size exceeds $35 billion. But NVIDIA cannot guarantee meeting all demand, and actual deliveries may fall far below order intentions. Some analyses suggest actual supply may cover only 5% to 10% of orders.

What is puzzling is that the RTX PRO 5500 is not NVIDIA’s top-tier training chip for data centers but a workstation graphics card, with “performance inferior to top-tier data center chips.” Why would Chinese major companies concentrate on snapping up such a product? If one looks only at peak compute, this order seems insufficiently “rational”; but if supply constraints, compliance conditions, and inference economics are considered together, it is precisely a rational choice under constraints.

The answer lies in the gap between supply constraints and inference economics. Against the backdrop of continued restrictions on exports of high-end data center chips to China, “compute that can be bought and deployed is more important than chip specifications that are ideal on paper.” The RTX PRO 5500 has significantly improved low-precision compute such as FP4 and FP8 compared with previous restricted products, exactly matching inference scenarios’ demand for low-precision, high-throughput performance, rather than training scenarios’ demand for extreme interconnect bandwidth and memory capacity. In other words, what the major companies are buying is not “the best chip” but the most accessible one among available chips in terms of inference cost-effectiveness.

This choice itself declares a fact: the supply-demand gap in inference compute has grown large enough that companies are willing to abandon the game of “waiting for the ideal chip” and accept the reality of a second-best solution. More importantly, this rush to buy will further change market expectations. When leading cloud vendors and internet platforms simultaneously place concentrated orders for one compliant product, the scarcity of the supply chain will be rapidly amplified, and rental prices, delivery cycles, and supporting server costs will fluctuate accordingly. For small and medium-sized AI companies, this means the threshold for obtaining inference compute may rise further; for domestic chip manufacturers, this means the window period contains both substitution opportunities and the risk of short-term squeeze by compliant NVIDIA products.

From a longer-cycle perspective, the RTX PRO 5500 buying spree also exposes a deeper problem: the “barrel” of compute supply is not determined by the chip plank alone. Even after buying chips, one still needs servers, high-speed interconnect, power and cooling, cluster scheduling, and supporting optimization of model inference frameworks. The bottleneck of an inference cluster often lies not in single-card peak compute but in memory capacity, memory bandwidth, inter-GPU communication, batching efficiency, and fault recovery capability. The major companies’ rush to buy cards is only the first step; the real competition is who can convert the cards they buy into stable, low-cost, schedulable inference throughput faster. This is also why the same chips can produce vastly different token costs per unit in the hands of different cloud vendors.

The Other Side of $657 Billion: Compute Is Becoming a “Fixed Asset” on the Balance Sheet

If the buying behavior of Chinese major companies reflects distortions on the supply side, then Anthropic’s IPO prospectus reveals from the demand side the magnitude of compute commitments in the inference era.

From October 26 to 27, Anthropic formally submitted an IPO application to the U.S. Securities and Exchange Commission. The prospectus shows that it has signed compute contracts worth at least $657 billion, with terms extending as long as 2045; including incremental amounts under negotiation, the potential total could rise to about $750 billion. Of this, at least $518 billion in committed spending on cloud services, computing power, and infrastructure over the next decade, with about 80% non-cancelable or payable regardless of actual usage.

What is shocking about these numbers is not their “size” but the structural business paradigm shift they reveal. The core cost of traditional software companies is labor; the core cost of cloud computing companies is depreciation; and in the inference era, the core cost of AI model companies is becoming a long-term, rigid, non-cancelable compute commitment. In the past, investors evaluating an AI company were accustomed to looking at model capabilities, user growth, and revenue growth; in the future, they may also need to look at how much compute it has locked in, whether the locked-in price is reasonable, how high the cancelable proportion of compute contracts is, and whether cost per unit token can decline with scale. Compute is no longer merely a backend cost item but a core asset affecting valuation, cash flow, and competitive moats—and at the same time a heavy rigid liability.

Anthropic directly lists “obtaining sufficient computing resources” in its operating and risk framework in the prospectus, while warning that compute expansion will continue to consume large amounts of cash. This sends a clear signal: in the inference era, the ability to obtain compute is itself part of competitiveness. Without sufficient compute, models cannot be trained, services cannot be scaled, and Agents cannot run stably; but locking in too much compute may create idle capacity and financial pressure when the economic cycle fluctuates or model efficiency improves. AI companies are thus caught in a dilemma: without locking in compute, they may miss the growth window; with excessive compute lock-in, they may mortgage future cash flow for years in advance.

NVIDIA and Oracle announced on October 27–28 that they would expand deployment of the Vera Rubin architecture to the multi-gigawatt level and, for the first time, decouple CPU and GPU resources so customers can scale independently to match different workloads— “GPU-intensive training and inference” and “CPU-intensive Agent tasks.” This is precisely an infrastructure response to the aforementioned structural change. Agent tasks consume not only GPU inference compute but also require substantial CPU-side tool calls, data retrieval, state management, and orchestration scheduling. Decoupling CPU and GPU is essentially an acknowledgment of a fact: the compute system in the inference era is no longer a single stack of GPUs but a system engineering task in which multiple resources are dynamically allocated according to workload proportions. Whoever can more finely match GPU-intensive inference with CPU-intensive orchestration tasks may gain an advantage in cost per unit task.

The “Asymmetric Window” for Domestic Chips: The Other Side of the Buying Spree

Returning to the Chinese market context, does the major companies’ rush to buy RTX PRO 5500 mean domestic chips are being “squeezed out”? The answer is not simple.

In the short term, the supply release of NVIDIA compliant products will indeed squeeze the penetration pace of domestic chips in the inference market. Faced with migration costs, ecosystem maturity, and delivery certainty, leading customers often prefer products that have already been validated; even if domestic chips have cost-effectiveness in specific scenarios, they need time to complete framework adaptation, operator optimization, and cluster tuning. This short-term squeeze is real and cannot be concealed simply with the slogan of “domestic substitution.”

But the deeper change is that the explosion of inference demand is changing the criteria for evaluating compute competition. A Huaxi Securities research report points out that the driving force for domestic compute has shifted from “single-card performance catch-up” to “system-level engineering capability + supply assurance capability.” CSC Financial also lists “domestic chips and supernodes and compute services” as directions to watch, noting that rents for some domestic configurations and overseas high-end GPUs are rising, and momentum in compute services continues. This means that the competition logic for domestic chips is shifting from “benchmarking the peak parameters of a certain NVIDIA chip” to “whether they can provide stable, affordable, and deliverable system-level solutions in specific inference scenarios.”

A noteworthy parallel development is that on October 28, Moore Threads open-sourced the large-model training infrastructure project Megatron-MUSA-Titan, focusing on trillion-parameter MoE model training on ten-thousand-GPU clusters and covering pre-training and reinforcement learning scenarios. DeepSeek open-sourced the DeepInfer inference framework, supporting domestic platforms such as Huawei Ascend, aiming to lower the cost and threshold for enterprises to build inference clusters based on domestic chips. The significance of these actions is not to replace NVIDIA in the short term but that, amid the structural growth of inference demand, domestic chips are shifting from “performance benchmarking” to “scenario adaptation”—when the diversity of inference workloads far exceeds the uniformity of training workloads, system-level optimization in specific scenarios may have more commercial value than peak compute numbers.

The third-quarter reports of domestic compute chip companies also confirm the transmission of prosperity: Hygon Information’s third-quarter revenue reached 5.856 billion yuan, a single-quarter high, with revenue in the first three quarters up 57.58% year on year; Sugon’s net profit attributable to shareholders in the first three quarters rose 78.48% year on year; Enflame Technology expects revenue in the first three quarters to grow 325.78% to 455.36% year on year. These figures show that the expansion of inference demand is not a zero-sum game; beyond squeeze and substitution, it is also expanding the bottom of the entire compute market. The real opportunity for domestic chips may not necessarily lie in head-on competition for the high ground of training clusters but in undertaking inference scenarios that are cost-sensitive, require high customized adaptation, and demand supply certainty. Whoever can solidly build software stacks, framework adaptation, and cluster operations in these scenarios can gain an asymmetric entry window in the inference era.

Conclusion: The “Barrel” of Compute Competition Is Being Reassembled

The noise in the compute field over the past week is essentially a forced correction by the demand structure against the supply structure. When intelligent agents break every user instruction into a string of backend calls, when inference token consumption expands at a rate far exceeding training workloads, and when AI companies have to write their compute bills for the next two decades into prospectuses, the competition logic of the compute industry can no longer remain on the single dimension of “whose chip has higher peak compute.”

What Sugon’s Li Bin calls “integrated, tightly coupled system innovation around computing, storage, interconnect, power supply, and cooling,” and NVIDIA and Oracle’s attempt to decouple CPU and GPU to match differentiated workloads, point in the same direction: compute competitiveness in the inference era is a comprehensive question of system engineering, not a multiple-choice question about chip specifications. The “irrational” behavior of Chinese major companies snapping up workstation graphics cards, Anthropic’s “high-stakes bet” of locking in compute with sky-high contracts, and domestic chips’ intensive actions in system adaptation are all different solutions to this new question.

What is more worth vigilance is that the explosion of inference demand will not automatically bring stability to the industrial order. On the contrary, in the short term it may intensify fluctuations in compute prices, supply tension, and the tearing of business models. Platforms need to find a new balance between quota-based billing and user experience; cloud vendors need to rearrange their procurement mix between compliant chips and domestic chips; AI companies need to redesign their capital structure between compute lock-in and financial flexibility; and chip vendors need to shift from selling hardware to selling system capabilities. Whoever can first complete the turn from “selling compute” to “using compute well” under supply constraints may find the ticket to the next cycle in the crack torn open by inference demand.