Demand for Inference: Designing Healthy, Scalable Networks in Verifiable AI
In today’s AI-powered world, inference is no longer just a backend process — it’s a foundational commodity. With generative models becoming foundational infrastructure, the question isn’t just what AI can do — it’s whether we can trust how it does it, prove it, and scale those guarantees across real-world demand.
At Inference Labs, we’re designing a new kind of network — one where Proof of Inference (a system requiring cryptographic evidence that an AI computation was performed correctly), network hygiene, and participation incentives aren’t afterthoughts, but first principles. Proof of Inference means cryptographically proving that an AI model performed exactly as claimed — no black boxes, no hidden processes. This article explores our approach to developing scalable, verifiable AI systems and includes insights directly from the team behind these innovations.
Generative AI is changing the game — but also rewriting the rules of trust. As @cdixon notes, AI’s power to generate at scale has outpaced our ability to prove what’s real, who owns what, and how decisions were made.
Meanwhile, researchers like @SashaMTL and @sarahookr are sounding the alarm: inference isn’t free. It’s expensive, resource-hungry, and often misunderstood — especially when you start layering verification on top.
At Inference Labs, we believe verifiable inference is the backbone of a trustworthy AI ecosystem — not just for proving correctness, but for scaling trust itself. We don’t merely publish benchmarks; we share the rationale behind our systems transparently, making our reasoning as open as our circuits.
For this article, we conducted a rapid Q&A with Spencer (Senior Software Developer), Colin (Co-Founder), and Eric (MLOps Engineer). Consider this a glimpse into the whiteboard discussions shaping our roadmap.
Network Hygiene & Ecosystem Design
What is network hygiene, particularly in the context of verifiable inference? How do you prevent spammy inference requests or resource abuse?
SPENCER: Network hygiene is essentially about maintaining a secure, efficient, and stable system. In a verifiable inference marketplace, like Subnet 2 or Sertn, this becomes especially critical. Verifiable inference involves significant computational resources, both for generating proofs and verifying them, making it expensive. We take several key measures to maintain strong network hygiene:
- First, we rigorously analyze our smart contracts and dependencies, running state-of-the-art bug bounty programs to quickly detect and address vulnerabilities. Additionally, we invest in thorough audits of critical components to guarantee network reliability.
- Our team constantly seeks to simplify network operations by reducing complexity, streamlining communications, and automating deployments wherever possible.
- Clean Codebases, we adhere strictly to best practices, such as checks-effects-interactions and single-responsibility principles, ensuring code clarity and network stability.
In practice, core network operations involve transactions that prevent abuse by compensating users who perform intensive computations. This structure safeguards against resource flooding and ensures a fair, secure environment.
How do you balance scalability with decentralization? Can a healthy network handle thousands of inference tasks without becoming centralized?
SPENCER: Balancing scalability and decentralization is fundamental to maintaining a healthy and resilient network. Our strategy includes:
- By supporting multiple proof systems and a diverse range of AI models, we create an environment where numerous operators can specialize in various areas. This approach prevents market dominance by a small group of centralized entities.
- As demonstrated in Subnet 2, the network thrives because operators can optimize for different tasks, from FPGA and GPU-based systems to specialized software adjustments. This diversity fosters healthy competition and prevents centralization.
- Implementation of strategic incentives to maintain network scale in relation to demand for inference tasks. Through systems such as dutch auctions and reward curves, network participants are incentivized to maximize performance and availability. Should one node or a group of nodes go down, another party will assume the responsibilities of the downed node.
What are early indicators of whether a verifiable inference network is healthy or becoming overloaded? Is latency or volume more critical?
SPENCER: Proof latency — how quickly the network verifies inference requests — is the primary health indicator, ideally faster than users performing local inference. Latency reflects technical health, while volume represents overall engagement and market demand. Both metrics are crucial but offer different insights.
Proof Systems & Infrastructure
One of the cornerstones of our approach is JSTProve, a specialized zero-knowledge machine learning (zkML) library. When used alongside the Expander framework, JSTProve dramatically reduces the computing power needed for verification. It does this through advanced math tools (like polynomial commitments) and cryptography designed to work efficiently with modern hardware.
Put simply, JSTProve and Expander make it possible to generate secure proofs at scale, handling thousands of inferences with ease.
How does our zkML (Zero-Knowledge Machine Learning) infrastructure (like JSTProve + Expander) change the cost equation of inference?
COLIN: The biggest challenge with inference verification right now is the high computational cost. Our advanced proof systems, combined with network-scale crypto-economic strategies, introduce clever methods to cut down this overhead without compromising security. Examples of this approach include optimistic inference, liveness guarantees, and proof deferral — each significantly reducing verification costs.
What’s the most misunderstood piece about inference verification
COLIN: One common misunderstanding is:
“Open-source or publicly available models don’t require verification since anyone can re-run the inference themselves.”
While this sounds correct, and often can be, there are important cases where the input data must remain private (such as KYC or age verification). As privacy regulations tighten, we anticipate a growth in applications that perform inference on users’ edge devices, limiting the company’s direct access to private data.
In these scenarios, users will input private data into an open-source model and then produce a cryptographic proof of execution, which can be verified by a third party without revealing the sensitive data itself.
Why did you push for full transparency and open source from day one?
COLIN: The rapid rise of companies like OpenAI and Meta’s LLAMA shows how much investor interest there is in frontier technologies. These technologies have dramatically changed the way we work and access information. With the inherent dynamic between these two facts, the companies responsible for their creation will inevitably be at odds on balancing them. Will they remove accurate but inconvenient content to maintain favor with certain governments? How will they balance maximizing profits with the broader public good?
Providing open-source solutions and exploring alternative business models may help address these challenges. Many companies, including OpenAI, initially promised transparency and openness, acknowledging the potential danger and dragons in the path forward. They have yet to live up to these promises.
Incentive Design & Participation
From a system design perspective, how do you encourage honesty in a proof-optional network?
ERIC: To enable proof-optional networks, we must incentivize honesty through random proof requirements. When proof requirements are lower (meaning proofs are requested less frequently), the temptation to cheat naturally increases, as cheating offers higher rewards for less work. To counteract this, we need stronger penalties or disincentives for cheating. This ensures the expected return from cheating remains lower than the return from being honest.
For example, let’s say a truthful inference requires 1 unit of work for 1 unit of reward, but a dishonest participant could submit a fake inference requiring 0 work and still get 1 reward. If proofs are randomly required roughly once every 100 inferences, penalties for getting caught must be high enough that the dishonest participant’s overall expected return stays below the honest participant’s expected return.
What’s something you think people underestimate about network incentive design?
ERIC: How creative bad actors can be. There might be hidden, external incentives influencing behavior — factors we don’t usually consider when calculating expected value. For example, the immediate reward and the work avoided might not be the only motivations. A malicious actor might specifically target a certain individual or inference type due to external motivations.
In critical scenarios, like an inference influencing a high-value smart contract outcome, a malicious actor might build up trust over a long period, behaving perfectly honestly, only to exploit that trust at a critical moment. I’m thinking specifically about the recent XZ Utils attack and similar extreme scenarios — like repeatedly cheating whenever possible, or never cheating until the exact moment when the payoff is highest. These kinds of possibilities need serious consideration in incentive design.
Final Thoughts: Building Trustworthy AI Networks
The future of AI depends not only on innovative technology but also on the trust we build around it. At Inference Labs, we’re committed to creating transparent, scalable, and secure AI networks through verifiable inference. By emphasizing network hygiene, sophisticated proof systems, and thoughtful incentive design, we aim to empower users and organizations alike with the confidence to rely on AI solutions.
In essence, treating AI inference as a foundational commodity demands that we prioritize transparency and performance in tandem. By doing so, we ensure generative models can scale responsibly, fostering an AI landscape that is both powerful and accountable to its users.
