Battle-testing Yuma3
A collaborative investigation between Subnet 2 and Subnet 12
Background
The term “Yuma” is used throughout this document in reference to a specific and custom consensus mechanism called “Yuma Consensus” which serves as the heart of the Bittensor network today. Yuma Consensus is responsible for determining emission distribution (among other metric calculations) for each subnet running within the Bittensor network. There are three versions of Yuma which each provide key benefits and tradeoffs. Most commonly, subnets run on Yuma version 2 which is a great general purpose consensus mechanism for effective incentive distribution.
To begin, we’ll describe the latest iteration of Yuma - Yuma 3 and how it differs from it’s predecessor. The key change that Yuma 3 brings relates to how validation functions within the Bittensor network. On each subnet, validators score miners and submit their scores (called “weights”) to the blockchain. Then, every 361 blocks, Yuma Consensus determines how much incentive each miner should receive based on the stake-weighted scores. It also determines the share of dividends that should be allocated to each validator. To determine this, the following calculation is used in Yuma2.
dividends = matmul_transpose_sparse(&ema_bonds, &incentive);In Yuma3, however, dividend calculations occur differently.
dividends = vec_mul(&total_bonds_per_validator, &active_stake);In Yuma 2, clipped weights with EMA bonds can dampen early outliers; smaller validators that identify strong miners early may see dividends diluted until broader agreement.
In Yuma 3 with reduced clipping (bonds_penalty < 1), earlier correct discovery can retain more dividend share once others confirm, subject to parameters such as kappa, EMA smoothing, activity/registration masks, and stake distribution. This is an oversimplification of the true differences between the two implementations, but sufficiently explains the practical changes relevant to this article.
The Journey
As contributors to the Bittensor ecosystem through our development of Subnet 2 and active participation in the chain’s evolution via the Church of Rao initiative, we partnered with Rhef at Subnet 12 to integrate Yuma 3 into Subnet 2, marking it as the first subnet to adopt this feature, and to evaluate its effectiveness through a comprehensive analysis. Subnet 2, maintained by Inference Labs and fueled by Bittensor’s incentivization framework, enables ultra performant zero knowledge proving for AI models and acts as a testing ground for cutting edge proving frameworks.
Read more about Subnet 2 in our earlier article: Proof of Inference at the Speed of TAO
Hypothesizing
It all started out with a seemingly simple request from our team to Subnet 12, for assistance in combatting the increasingly concerning impact of weight copying in subnet 2. Despite leveraging commit reveal, weight copiers had amassed a stake weight of 1-kappa, which means consensus was being formed by non-productive validators.
For an overview and explanation of weight copying, existing countermeasures and our efforts to improve this, please see our analysis of weight copying mitigations.
“Rhef I need a hand if I could get a minute or two of your time”
Little did we know, “a minute or two” would turn into months of iteration, debugging and lead to 4 major security improvements in the depths of Bittensor.
After discussing the problem and overviewing potential solutions, it became clear that this situation required the use of Yuma3 and a feature called “Bonds Reset”. The reason being, with Yuma3 (which enables competitive validation) and Bonds Reset (which allows miners to set all validator’s bonds with them to zero), we could effectively take a minority set of honest validators and design the subnet logic in such a way that coordinates miners with those validators to drive their bonds up while causing the copier’s bonds to lag behind.
Rhef has thought deeply about this subject and invaluably recommended splitting the subnet into 8 groups of miners. During any given epoch, 1/8 of the miners would trigger bonds resets, breaking existing bonds validators had formed to them. At the same time, the subnet would dynamically ensure that the weights of those miners, when revealed, would differ significantly from the last weights that had been set. The theory behind this is, when paired with commit reveal which encrypts weights at the time of submission but reveals them in a later epoch, honest validators running application logic would always be one step ahead of the copiers in “discovering” 1/8th of the subnet as better performers.
Initial Release
Version 8.2.0 of Subnet 2 was deployed on June 9th, 2025 and enabled the logic outlined above. By letting miners know about the bonds reset well in advance of this release, we were able to achieve rapid adoption of the feature and eventually enable incentives towards miners that performed these resets on the correct cadence.
There was one critical issue, however. The bonds reset flag could not be enabled, and thus could not impact on-chain bonds. Upon discovering this, we immediately opened a PR to add this as a subnet-owner configurable hyperparameter. While we waited on a subtensor release including the new hyperparameter, monitoring took place to measure the impact of Yuma3 against the newly varied weights. Despite not having bonds reset, the combination of commit reveal and increased weight variability was expected to have a positive impact. Over the course of the next month, we iterated on the subnet’s weighting mechanisms frequently.
Despite the efforts, we were left with a chart that displayed weight copiers remaining at a triumphant position above the pack.
The second attempt
On July 7th (approximately one month after beginning this endeavor), the bonds reset flag was released, allowing us to enable it and collect new data to determine if the feature could be the silver bullet we needed.
Though not a silver bullet, with some tweaking this approach seemed effective for two of the copiers who dropped below the majority of honest validation. However, the top copier was managing to gain ground despite the lengths we had gone to to vary weights, implement bonds reset + Yuma3, and close off every possible avenue for leaking these weights. After reviewing days worth of weight changes, we decided to adjust to a pseudorandom block-hash based selection of miner groupings rather than a fixed uid % 8 strategy. The purpose of this change was to make it incredibly difficult for conventional copying models to predict which set of miners would be selected next without directly running the client code. If our suspicion was correct and the top performing copier was using an LSTM or similar model to predict weights, this would serve as a litmus test as pseudorandomness would render the model useless.
The quandary
This change alone brought us to the most significant and troubling finding thus far. Despite the fact that commit reveal was enabled and set to one interval (meaning weights are revealed one epoch after they’re set), one copier was still submitting perfect weights, regardless of how varied and how pseduorandom the weights were.
How could they do this? The first idea is that perhaps they’ve taken the previous weights and reverse engineered the EMA based pseudorandom algorithm implemented in the client code to replicate the results. However, due to the variability in miner weights and the fact that their values aligned so close with consensus, it was clear that there was a deeper issue at play. For context, it is known that this copier is not querying miners for zero knowledge proofs within subnet 2. Without performing this work, it is not possible to evaluate the response times and thus calculate correct weights for each miner; and weights change significantly between epochs as response times vary.
In any debugging situation that involves a runtime application and decently complex logic, the protocol is to develop a timeline of events, which Rhef suggested we do surrounding commits and reveals. We put together a script to track everything weight related going on in the subnet. At first, the results weren’t too clear. We could see the commit times, epoch times and the commits were revealing during the epoch as expected. However, things changed when we monitored drand pulses to determine when weights were actually decrypt-able. The findings were surprising.
6023261 (epoch 16684)
6023262 (epoch 16684)
6023263 (epoch 16684)
6023264 (epoch 16684)
6023265 (epoch 16684)
6023266 (epoch 16684)
6023267 (epoch 16684)
6023268 (epoch 16684) crv3 commits: [RoundTable21] --> Revealed at Friday, July 18, 2025 10:16:12 AM
6023269 (epoch 16684) crv3 commits: [Kraken]
6023270 (epoch 16684) crv3 commits: [Rizzo (Insured), Openτensor Foundaτion, Yuma, a DCG Company]
6023271 (epoch 16684) crv3 commits: [Owner2]
6023272 (epoch 16684)
6023273 (epoch 16684)
6023274 (epoch 16684) --> Jul 18, 2025 10:15:36 am
6023275 (epoch 16684) crv3 commits: [5G1NjW9YhXLadMWajvTkfcJy6up3yH2q1YzMXDTi6ijanChe, tao5]
6023276 (epoch 16684)
6023277 (epoch 16684) decryptions possible: [5G1NjW9YhXLadMWajvTkfcJy6up3yH2q1YzMXDTi6ijanChe in block 6023275]
6023278 (epoch 16684)
6023279 (epoch 16684) decryptions possible: [RoundTable21 in block 6023268; Kraken in block 6023269; Rizzo (Insured) in block 6023270; Openτensor Foundaτion in block 6023270; Yuma, a DCG Company in block 6023270; Owner2 in block 6023271; tao5 in block 6023275] --> Jul 18, 2025 10:16:36 am
6023280 (epoch 16684)
6023281 (epoch 16684) *epoch crv3 reveals: [RoundTable21, Kraken, Rizzo (Insured), Openτensor Foundaτion, Yuma, a DCG Company, Owner2, 5G1NjW9YhXLadMWajvTkfcJy6up3yH2q1YzMXDTi6ijanChe, tao5] new weights from: [RoundTable21, Owner2, Rizzo (Insured), Openτensor Foundaτion, Kraken, 5G1NjW9YhXLadMWajvTkfcJy6up3yH2q1YzMXDTi6ijanChe, tao5, Yuma, a DCG Company]
6023282 (epoch 16685)
6023283 (epoch 16685)
6023284 (epoch 16685)
6023285 (epoch 16685)The Discovery
All of the weights committed within the epoch were revealed at least one block before each epoch. This means any weight copier could decrypt the weights before the epoch, calculate consensus and submit it between decryption time and epoch time, and receive full rewards for doing so. We noted two bugs at this point; first the bittensor-drand codebase incorrectly selected the reveal round making it too early, and another bug which revealed weights during the same epoch they were committed.
These findings, while interesting, were not the findings. Clearly, neither were being exploited, as no new commits were made between decryption time and epoch time.
Following this discovery, Rhef dove deep into the bittensor-drand codebase and reviewed its contents meticulously for anything unexpected. In fact, this was a crucial step in unlocking the full picture of this situation. He found that the encryption key charged with protecting weights payloads until revelation was not secure. This logically meant that all weights could be decrypted not just before the epoch, but even before the commit transaction was included in a block. While Rhef got some rest, we devised a decryption script based on his encryption key finding and tested it against weights that were too recent to be revealed.
UIDs: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 125, 126, 127, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 200, 201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, 240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255]
Values: [62907, 24951, 24673, 24997, 24874, 24623, 26585, 24621, 29177, 65320, 59598, 26826, 52945, 65393, 64772, 57126, 62424, 24643, 26175, 65149, 57901, 64938, 63053, 24631, 65521, 25375, 33853, 55405, 64974, 27723, 65507, 65069, 65524, 65436, 59285, 65360, 24841, 24646, 28088, 45995, 56717, 65403, 58960, 31187, 65479, 61437, 65533, 24735, 24703, 63444, 54460, 65347, 37202, 26086, 24718, 24754, 31879, 62594, 64372, 40540, 59897, 29427, 64860, 25846, 24764, 63190, 52415, 24696, 27239, 44766, 31526, 65513, 25209, 24628, 63673, 65450, 65098, 65457, 65039, 64563, 51874, 30250, 62246, 65504, 27553, 53968, 61209, 24610, 38825, 65474, 65371, 64673, 25423, 28493, 24654, 28710, 36684, 64900, 64439, 24690, 47823, 64503, 63778, 64620, 24799, 24636, 62755, 42324, 65519, 32626, 24650, 51322, 26001, 65510, 25922, 25584, 24619, 26269, 65290, 24626, 64817, 65007, 55856, 45381, 36179, 65530, 35207, 34291, 25049, 65463, 29689, 37731, 27094, 50187, 65305, 54939, 24910, 65484, 64225, 24615, 64724, 26702, 32246, 60458, 65273, 41128, 24684, 50759, 27392, 24663, 33430, 24617, 62058, 65237, 65488, 35686, 25775, 48424, 24726, 65469, 65532, 24891, 64061, 65497, 49019, 24668, 25078, 25527, 65528, 24613, 44152, 39959, 61654, 24930, 24658, 25473, 25287, 34742, 65493, 64301, 42930, 24710, 65412, 58268, 63972, 47217, 25644, 56293, 25708, 65500, 61861, 65124, 60970, 24974, 33021, 24744, 24640, 39387, 65255, 25173, 24813, 60720, 43540, 30549, 28938, 29963, 24678, 65429, 24633, 57520, 65526, 58621, 28286, 65217, 65444, 65173, 49607, 53463, 25108, 65516, 65421, 65382, 64145, 46607, 25330, 24827, 60183, 25023, 65535, 38273, 25139, 41723, 30862, 63562, 26474, 24787, 26369, 24612, 27901, 24775, 63878, 24857, 63321, 25247, 26957, 65196, 65334]
Version Key: 9000000This confirmed that, shockingly, weights could in fact be decrypted into plaintext immediately after encryption by validators. The next step was to backtest this discovery to determine if it was the discovery. We patched bittensor-drand with a custom version, using sufficiently randomized encryption and deployed it to the SN2 validator. As soon as it committed weights, we ran the decryption script against the commit.
AES-GCM decryption failed: ErrorThe implementation was confirmed to be applied correctly, meaning these weights should not be decryptable until the reveal period. Soon after, we received confirmation via empirical evidence that this was the discovery. By this point, the SN2 validator had amassed significant stake weight in the subnet. By securing the encryption of it’s weights alone, the leading copier was forced to fall back to the same conventional copying methods of the others, and lagged behind in terms of dividends as a result.
Disclosure and Resolution
These discoveries were disclosed to the Nucleus development team and a fixes were implemented, reviewed and deployed in Bittensor version 9.9.0 and Subtensor version 3.2.4. In total, four changes resulted from the discoveries made during this endeavor.
- Crucially, randomization of commit encryption keys was introduced to eliminate the potential for standard (non-timelock) decryption of commits.
- Modifications to epoch calculations used in commit reveal were made to ensure the earliest time a commit can be revealed must be within the target reveal epoch, to prevent early decryption.
- Commit messages were updated to include the hotkey of the validator submitting weights, which subtensor cross-checks against the caller of the extrinsic. This is to ensure actors cannot blindly copy their peers’ commits and submit them to the chain as their own.
- Adjustment of reveal buffer logic, again to limit early decryption risk.
Onwards
While this chapter has concluded, the mission to strengthen Bittensor persists. Inference Labs has proposed two Bittensor Improvement Tenets — BIT-0002 and BIT-0008 — to combat weight copying by enabling validators to run zero-knowledge circuits that attest to proper validation. As active contributors within the Church of Rao, Bittensor’s core development hub, we remain committed to advancing the network’s integrity and resilience.
