Scoring proven and verifiable goals
Inference Labs is a leader in verifiable oracles, running one of the largest decentralized zkML proving clusters in the world called DSperse (Bittensor Subnet 2).
Subnet 44 offers state of the art detection and tracking for a variety of vision applications. One of the largest use cases comes in the form of real time soccer (football) game predictions, where score acts as a high quality and near-instant source for the highest quality predictive data regarding player movements, team strategies and ball positioning.
The cost of failure
Generally speaking, computer vision based models are used to inform actions, taken either by humans or agents. Those actions are often high-stakes, such as adjusting open positions on prediction markets / sports betting in football applications, changing input parameters in the case of self-driving vehicles and even directing use of defense for airspace security. In virtually all use cases of vision models, failure has a high cost.
Inference Labs brings verification to AI models and other forms of computations through the use of zero knowledge cryptography. By converting AI models into verifiable “circuits”, cryptographic proofs can be generated for the model when it performs inference, these proofs prove mathematically that the inputs, model inference, and outputs completed in an untampered and fully secure manner. This means that actions performed based on vision outputs can be confident that they’re receiving correct and honest output, regardless of who conducted the inference.
The challenge
The largest constraint of zkML remains in the computational (and subsequent temporal) overhead incurred from proving large models. The relationship between the size of a model, the number of inputs and outputs the model has and the resulting circuit is not necessarily linear in nature as larger models balloon memory usage and overwhelm consumer hardware. To provide verifiable inference on a model as large as YoloV8X while retaining accuracy in it’s outputs was previously deemed too difficult for even cutting edge zkML systems.
Where we started
Inference Labs creates novel and innovative pioneering approaches to verifiable inference. Recently, we released two state of the art frameworks designed to accelerate our mission towards verification for all inferences.
DSperse — A framework for distributed, verifiable inference. DSperse (proprietary patent pending) analyzes AI models to determine optimal locations to create smaller, sub-models (slices) suitable for compilation into zero knowledge circuits. It then selects the best prover backend given the context around which operations each slice performs and creates an optimized chain of zero knowledge circuits to prove the model.
JSTprove — An in-house prover framework which translates AI operations into low-level zero knowledge circuit using sumcheck and GKR. This framework introduces novel and significant performance boosts both in terms of accuracy and resource usage, and expands our ability to support advanced AI models.
This combination of technologies places Inference Labs and Subnet 2 in the best place possible to tackle the problem of large vision model verification. Initial tests were promising. We were able to completely break apart the model into smaller slices and reconstruct identical outputs by running inference on each individual aspect. However a significant challenge was discovered: Despite slicing, each layer was too large to circuitize. Even with enterprise grade hardware, the memory constraints were far too high for feasible usage.
In order to further reduce the size of each individual slice, our first objective was to analyze the structure of the models themselves.
┌───────────┬───────┐
│ Operator │ Count │
├───────────┼───────┤
│ Conv │ 104 │
├───────────┼───────┤
│ Mul │ 100 │
├───────────┼───────┤
│ Sigmoid │ 98 │
├───────────┼───────┤
│ Constant │ 21 │
├───────────┼───────┤
│ Add │ 21 │
├───────────┼───────┤
│ Concat │ 19 │
├───────────┼───────┤
│ Split │ 9 │
├───────────┼───────┤
│ Reshape │ 5 │
├───────────┼───────┤
│ MaxPool │ 3 │
├───────────┼───────┤
│ Resize │ 2 │
├───────────┼───────┤
│ Div │ 2 │
├───────────┼───────┤
│ Slice │ 2 │
├───────────┼───────┤
│ Sub │ 2 │
├───────────┼───────┤
│ Transpose │ 1 │
├───────────┼───────┤
│ Softmax │ 1 │
├───────────┼───────┤
│ Shape │ 1 │
├───────────┼───────┤
│ Gather │ 1 │
└───────────┴───────┘The most common operation in the model are convolutions. Based on this, our first task was to determine optimizations for conv operations.
Convolutions
To optimize convolutions we must first understand the fundamentals of what they are and what they do. Perhaps the easiest way to explain this is visually.
A convolution simply applies a weighted transformation to matrices. In the above illustration, we extract two dimensions from the image on the left hand side (height and width) to generate a matrix of pixels within the image. From there, we define a “kernel” which is displayed in the centre of the view. The kernel is usually a small grid, commonly 3x3 as shown above. This grid has numbers assigned to each of the positions. These numbers are called “weights”.
Convolutions work by passing the weighted kernel over top of the provided inputs, multiplying the values beneath (in the original image) with the values defined in the grid (the weights). This process leads to the creation of another image, shown on the right hand side. This convolved output image is referred to as a “feature map”. You may have seen something similar when applying photo filters in photoshop. Succinctly, a convolution applies a learnable weighted filter (kernel) at every spatial position of the input (usually with appropriate padding), producing a feature map through local weighted summation.
Now the question is: How can we optimize this operation such that it fits within a memory-constrained environment such as a zero knowledge circuit? Moreover, how can we better leverage distributed systems to conduct proving work on these operations?
The answer to that question lies in a feature used commonly by GPUs to accelerate graphics rendering: Tiling.
This method essentially entails breaking images or large pieces of data into smaller chunks called tiles. The key benefit of this approach in conventional graphics acceleration comes from improved cache locality and drastically reduced external memory bandwidth usage. In tile-based rendering (common in mobile and embedded GPUs like PowerVR or many ARM Mali designs), the framebuffer and associated data (depth, color, textures) are processed one small tile at a time — often 16×16 or 32×32 pixels — entirely within fast on-chip memory. This avoids repeated round-trips to slower off-chip DRAM for every pixel operation, as intermediate reads/writes (e.g., depth testing, blending, texture fetches) stay local. Spatial locality is exploited because neighboring pixels tend to access similar data, keeping more relevant information in cache and minimizing expensive memory transfers.
Analogously, in zero-knowledge circuits — where prover memory is often severely constrained (sometimes to just a few GB even for massive computations in zk-SNARKs or zk-STARKs), tiling offers a path to handle operations that would otherwise exceed available RAM during proving or polynomial commitment steps. This mirrors how tiled GPUs bin geometry per tile and defer full shading until the tile fits on-chip.
Why tiling works
Since convolutions require context about surrounding data to calculate individual points of data, a valid and reasonable question to ask would be “Can convolutions be divided in a lossless manner?”.
The answer to this question is yes with important constraints. Each tile must be slightly overlapped with other, nearby tiles in order to avoid scenarios where the kernel incorrectly factors the edges of the tiles into it’s calculation. In practise, this works by adding a number of pixels around each tile, performing the convolution, then discarding the outputs from the overlapped pixels. This process then allows the non-discarded portion to maintain 100% parity with a convolution performed over that region of pixels and thus can be stitched together with the results of the other tiled convolutions to form the final feature map without losing data. Technically, due to overlaps, the total size of the data convolved is larger than the original size of the image. However, due to the performance gains from parallel processing and reduction in individual input size per-tile versus the full image, the tradeoff is worth it.
Parallel Tiling
Now that we’ve successfully extracted tiles into their own respective zk circuits, near-real time proving can be conducted on each of the tiles in parallel across cores or nodes in distributed networks. This leads to a massive increase in the performance of the operation. Previously it was not possible to prove large vision models due to extreme circuit sizes, even with a vast expanse of compute. Through a thoughtful redesign of the process for generating proofs from a first-principles perspective, it’s now feasible and fast.
Next Steps
Though the above optimizations were a substantial unlock for proving efficiency, we’re not stopping there. Similar approaches are being explored and implemented with a variety of other operations within the model to provide the fastest and highest quality result for vision’s real time needs.
Inference Labs has deployed all three of score’s base models to our subnet, leveraging real, production grade proving of large vision models at scale with proof coverage in excess of 30% per model.
These advancements position Subnet 2 at the forefront of verifiable AI and computation more broadly. Our in-house DSperse and JSTprove frameworks, combined with novel methods for breaking up large computations in zero knowledge proving, mark the beginning of a new era for verifiable oracles.
