Analysis of Weight Copying Mitigations in Bittensor

15 min readJul 24, 2024

Introduction

As blockchain technology evolves, it consistently presents new opportunities and challenges. Bittensor, a blockchain explicitly designed to serve as an incubator for scalable, decentralized AI infrastructure, confronts one such challenge: weight copying by validators. This practice, involving the replication of weight values by non-participating validators, is one that the blockchain aims to reduce through the introduction of the countermeasures outlaid below.

Summary

This document examines recent strategies implemented by Bittensor, a blockchain designed for scaling AI infrastructure, specifically targeting the practice of weight copying among validators. This analysis looks at two main countermeasures — Commit Reveal and Consensus-Based Weights — evaluating their effectiveness and pinpointing potential tradeoffs. By thoroughly assessing these strategies, the document highlights possible circumvention in which weight copying could continue.

Background

Bittensor is a blockchain ecosystem designed to scale AI infrastructure through a network of specialized subnetworks (“subnets”). Each subnet implements a unique incentive mechanism for its participants: validators and miners. Validators query miners with specific workloads and evaluate their responses based on the subnet’s predefined criteria, assigning “weights” to miners accordingly. Crucially, validators are incentivized to submit accurate assessments, as they are rewarded for weight submissions that closely align with those of their peers, promoting consensus and reliability. Miners, in turn, are motivated to provide high-value outputs that align with the subnet owner’s objectives. This competitive structure encourages continuous improvement among both validators and miners, as they strive for economic advantages by enhancing their performance and accuracy. The result is a dynamic, self-optimizing system that efficiently allocates resources, rewards innovation, and maintains high standards in AI development.

There is a key issue in the above described process, however. Since communication between validators and miners, and associated weight calculation processes occur purely off chain, it is possible for validators to choose to avoid the process of communicating with miners and instead copy weights from other validators to remain in consensus despite their choice not to interact with miners in the subnet. This process is called “weight copying”.

Weight copiers can also earn higher dividends than any participating validator by copying consensus weights; the weights that the chain ultimately uses based on the stake-weighted average. By submitting consensus weights on chain, the copier validator is maximally rewarded because it is in full agreement with all other validators in a stake-weighted fashion.

As well, weight copying validators can entirely avoid costs associated with maintaining subnet specific infrastructure, which adds to the incentive for weight copying. For example, around 10 subnets require GPUs for validation. Using RTX4090 as an approximate average, this adds an extra cost of over $300 monthly [5] which can be completely avoided by weight copiers.

Lack of verification behind weight calculations presents several challenges such as centralization risk, and even cases where subnet validators could operate entirely independently from the subnet owner’s specifications, copying off of one another in a perpetual loop and leading to an unproductive subnet.

Weight Copying Strategies

There are two strategies of weight copying that we’ll focus on for the purposes of this document.

Consensus copying. This strategy involves copying weights that will result in maximal rewards, by setting weights that are the stake-weighted average of all validator’s weights. Since the weights set are exactly the same as what the blockchain calculates, the validator is provided the most dividends for being in full agreement with all other validators.
Single validator copying. This is a simpler form of weight copying, which allows the copier to selectively copy the weights being set by another specific validator. This strategy is less profitable than consensus copying because it relies on the performance of the validator being copied, rather than using fully accurate consensus values.

Current Countermeasures

Two key strategies have been implemented by OTF to combat weight copying.

Commit Reveal
Consensus-Based Weights (also known as “Liquid Alpha”)

Opentensor provides in depth documentation of both of these implementations, which can be found in the references section. For a brief introduction to both of these changes, please refer to the relevant documentation excerpts and commentary directly below.

Commit Reveal

The commit reveal feature changes the way the subnet validator weights are recorded to the chain. Rather than submitting weights openly to the chain that can be seen by anyone on the next block, subnet validators will upload an encrypted hash of their weights. This encrypted hash will be automatically decrypted after a set number of blocks. [1]

This feature operates essentially by encrypting validator submitted weights for the subnet owner’s defined interval in order to obfuscate weights being submitted from potential weight copiers. The desired outcome of this algorithm is to remove the economic incentives of weight copying by forcing weight copiers to use outdated data when copying, leading to lower consensus values and thus lower dividends for the weight copier. Commit reveal has yet to be implemented successfully on any subnet. Subnet 1 attempted to enable the feature, but failed to perform the necessary software changes beforehand and quickly disabled commit reveal. Subnet 23 attempted to enable commit reveal twice, but each time reverted the change due to errors.

Consensus Based Weights

With this feature, the dividends to a subnet validator are better correlated to the performance of the subnet miner on which the subnet validator is setting the weights. [2]

This is a complementary feature to commit reveal, which further increases the severity of validator penalties for assigning weights that diverge from the consensus. Consensus based weights allows subnet owners to specify a range that essentially defines how consensus is interpreted into alpha. Subnet owners will adjust alpha high and low based on the level of consensus in their subnet. If consensus is relatively high across miners, the range can be very small. Likewise, if there is significant divergence between validators in terms of agreement on weights, the range of alpha low and alpha high must be higher to avoid penalizing good validators. Consensus values are mapped onto the range of alpha low to alpha high to produce alpha. As consensus increases, alpha approaches zero, and inversely as consensus decreases, alpha approaches 1.

The primary goals of this change are twofold;

To encourage rapid scoring and agreement on that scoring between validators. Since the value of alpha is at its highest when consensus is low, validators that are quick to submit weights which prove to match with other validator’s weights will be rewarded with a rapidly accumulating bond, leading to economic incentive for the validator
To discourage weight copying in commit reveal enabled subnets. Due to the latency imposed via commit-reveal, weight copying validators will always be slower to respond to changes in weights than validators that participate directly in the network. This will result in a slower bonding accumulation since the weight copier will lag consensus values for the number of epochs that weights are encrypted for, and thus will see reduced bond accumulations due to a failure to quickly adopt weights consistent with others.

Commit Reveal introduces a time delay that prevents real-time weight copying, forcing potential copiers to lag behind the true consensus. Consensus-Based Weights then amplifies this disadvantage by dynamically adjusting bond growth rates based on consensus alignment. Together, they create a compounding penalty for copiers: not only are copied weights outdated due to the reveal delay, but they also accrue bonds more slowly due to their misalignment with the current consensus. This combination significantly enhances the first-mover advantage for validators performing genuine, timely evaluations, while simultaneously improving the overall quality and reliability of the consensus mechanism. This process is illustrated in the charts provided in the consensus based weights document, referenced below.

Charts depicting the effects of commit reveal and liquid alpha, from the OTF provided demo notebook

Limitations of Countermeasures

Despite these efforts to impede the validator’s ability to copy weights, several methods remain viable to do so. In addition, the implementation of commit reveal has the potential to hide time-critical data from miners regarding their performance in the subnet. These issues are expanded upon and demonstrated below.

Lack of subnet adoption

The commit reveal feature was released on June 11, 2024. Since then, 2 out of 41 subnets have attempted to enable the feature but reverted the changes soon afterwards due to errors. Subnet 1 faced a problem where validators did not receive updated code compatible with the commit reveal strategy and as a result could not use the feature. Subnet 23 implemented the necessary code, but validators were unable to reveal their weights and as a result the change was reverted. This shows a general lack of enthusiasm towards the commit reveal feature, as less than 5% of subnets have attempted to implement the change, and of those 5% none have been successful, leading to no further attempts since.

Lack of miner visibility

Due to the changes made in commit-reveal, miners are unable to receive immediate feedback regarding their performance due to the time based encryption of weights on-chain. This information is critical to the miner because it provides all of the data relevant to their performance within a subnet. This can cause situations where miners are unknowingly put into the deregistration zone and cannot respond in time to revive their scores due to subsequent weight changes also being delayed. Likewise, miners who make changes will not be able to see an immediate increase in incentives and may walk back subnet-improving changes due to lack of confidence. Based on the provided guidance on commit reveal intervals, the average recommendation is an interval in excess of 14 hours. This is far too long for the vast majority of subnets where significant metagraph swings occur in shorter intervals. For example, the recommended interval for subnet 4 is 8.4 hours. The below heat map depicts changes in the metagraph over time. It is evident based on the chart provided that the metagraph fluctuates dramatically in a period of 8 hours. Missing this window of visibility is potentially make-or-break for miners participating within the subnet.

Screenshot from Bactensor displaying changes in UID ordering over time

Off-Chain weight copying

The changes both in commit reveal and in consensus based weights do not prevent weight copying in all forms — only weight copying using data provided by the blockchain. There are two situations that should be considered where weight copying would still be viable and circumvent the measures described above entirely.

Collusion with an operating validator. Since validators operating within the network are still generating raw weights (weights are encrypted when they are posted on-chain but not locally), it is possible for these validators to sell their weights to weight copiers in exchange for a cut of the weight copier’s emissions. As long as the source validator is in good consensus with other validators and helps to establish a consensus for low consensus nodes, the weight copier will also receive the same benefits but without any of the overhead costs associated with operating a validator that interacts legitimately with the subnet. This could especially occur in situations where the source validator is offered this type of transaction from another validator with significant delegation. This can also occur directly to miners in subnets that opt to share validator weights back to miners directly via synapses. A copying validator could theoretically compensate miners a small amount for what is otherwise non-monetizable data in exchange for the weight values that are directly sent back from validators.
Weight exposure via public dashboards. Currently, about 70% of subnets offer some form of public dashboard which exposes validator weights either directly or indirectly. Subnets are likely to continue leveraging these public dashboards especially after an integration with commit reveal to provide miners with an idea of their performance on the network in near real time. This is a great user experience feature for participating miners and validators, but it comes at the expense of providing a trivially easy path for copying validators to perform off-chain weight copying and completely bypass the mechanisms in place to prevent on-chain weight copying. For example, a weight copier could gather weight or scoring data from a public WandB dashboard and determine average weights from any of the validators actively logging. By submitting these weights to the chain, the effects of commit reveal and consensus based weights are mitigated since the weight copier is submitting up-to-date data. Pseudocode illustrating the ease of copying through this method is provided below.

api = WandB.api()
runs = api.runs()
weights = []
for run in runs:
  data = run.data()
  for index, row in data.iterrows():
    for i in range(256):
      weights[i] = row[f"weights.{i}"]
_do_set_weights(weights)

Incentive to weight copy remains

Both commit reveal and consensus based weights rely on variability within subnets in order to operate effectively. If scoring is very stable for a given subnet, it may be impossible to choose a reveal interval that effectively decreases weight copier dividends without causing major adverse effects due to extreme update delays. An excerpt from the OTF provided commit-reveal diagnostic notebook is provided below, outlining this issue:

If given a conceal period long enough (>15 hours) and the SN still [fails] to produce enough [loss] in dividend[s to weight copying validators], it means that there is not enough churn and weight movement in the SN, so [commit reveal] may not work for your SN. Depending on the situation, you may choose to increase competitiveness/ churn in your SN or just leave the weight copier as is. [When] there is no churn in the SN, there would be no movement in consensus as well, so the weight copier would not be as beneficial.
…
Note that when the conceal period was set too long, it would slow down the discovery of new miners, putting them at risk for deregistration. Further more, it would means that any change in the network would only be observable after 360 * conceal_period blocks.

A chart is attached below depicting the effect of commit reveal intervals on the relative dividend rate of a consensus weight copying validator as per the commit reveal developer notebook. In this chart, G=1 is the rate of dividends towards the median validator within the subnet. Values below 1 indicate that the copier is earning less than the median. Values less than 0.82 indicate lack of incentive to validate at all; the validator would earn more by nominating their stake. We can observe that for most subnets, commit reveal becomes most effective as the interval increases. On average, it takes an interval in excess of 6 hours for a commit reveal to remove incentive for nominators to delegate, and no subnets are able to reduce copier dividends below the point where the copier loses incentive to validate. This indicates that though the commit reveal strategy does reduce the total amount of dividends to copying validators, it fails to remove incentive to copy entirely.

Dividend rate chart generated using OTF provided demo notebook

An alternative — Proof of Weights

Based on the limitations outlined above, it is clear that the implementation of commit reveal and consensus based weights reduce dividends for weight copiers to some degree, but issues in miner visibility and removing incentive for validators to copy entirely remain unsolved.

Another issue exists as well, which was alluded to in the last paragraph of the background section; a situation where validators could copy off of each other perpetually, with no validator actually participating in a subnetwork. The root of this issue is actually in verification that validators are performing work that is relevant to the subnet. Since the majority of subnets implement communications purely via off-chain requests, there is no verification that validators are running the code necessary to score miners at all.

Inference Labs has proposed a solution that covers this problem while limiting any negative side-effects through the use of zero knowledge verification, called Proof of Weights. While the full implementation is outlined in the document included in references at the end of this document and linked here: https://docs.omron.ai/proof-of-weights; a brief overview is provided below.

A subnet owner defines their reward function, which accepts scoring-relevant inputs from queries that are made to miners within the network and outputs weight changes for each miner UID. This function, normally defined in code and run by validators without any form of verification on-chain, is then converted into a provable zero knowledge circuit. This circuit is unique to the reward function. Once this circuit is created, the subnet owner possesses a verification key, which can be used to verify that validators are operating correctly.

Validators then use this zero knowledge circuit provided by the subnet owner to calculate changes in weights for each miner it interacts with. Since the circuit is tamper-proof, it can be run on any machine as long as it produces a valid proof, which verifies that the weight change occurred in accordance with the subnet owner’s defined reward mechanism. Due to this ability to run in any environment without sacrificing security, it is possible to offload this work to miners in the network and completely avoid calculation of weight changes within the validator’s machine, while still ensuring full security.

In contrast to the above described mitigations implemented by opentensor, proof of weights still allows for immediate on-chain publishing of weights, removing any issues related to miner visibility. Additionally, off-chain copying risks are mitigated. Validators have no incentive to copy since they must provide a valid proof along with their submitted weights, which depending on subnet-specific implementation can include validator-specific metadata that forces the submitting validator to generate their own proofs. In this case, the validator is required to do the work of calculating weight changes themselves anyways and derives no benefit from seeing other weights submitted. Since proof of weights provided a very objective mechanism for determining if weights were calculated correctly, the effects of an invalid proof can optionally be severe, allowing the chain to directly reduce the incentive to weight copy by slashing dividends for invalid weights.

Proof of weights also overcomes liveness issues through two main pillars. For one, the Omron subnet is fully open source, meaning that though it is used today as the proving cluster, it can be forked and reimplemented within subnets or as a new subnet if deregistered. As well, proofs of weights are published on-chain. This ensures that the data is preserved on the public ledger, ensuring the proofs are accessible to all in perpetuity. For more information relating to this feature, see the linked documentation post in the below references section.

Conclusion

It is challenging to address weight copying effectively in the context of the Bittensor ecosystem. While the OpenTensor Foundation has implemented innovative countermeasures such as Commit Reveal and Consensus-Based Weights, our analysis reveals that these strategies, though well-intentioned, fall short of comprehensively addressing the problem. The primary limitations of the current approaches include:

Low adoption rates of new features across subnets
Reduced visibility for miners, potentially hampering their ability to optimize performance
Persistent vulnerabilities to off-chain weight copying methods
Ineffectiveness to fully remove incentive to weight copy

These shortcomings highlight the need for a more robust and holistic approach to reducing weight copying. The Bittensor ecosystem must evolve to address not only on-chain vulnerabilities but also off-chain collusion and data exposure risks. Moving forward, it is crucial for the Bittensor community to:

Encourage wider adoption of security features across subnets
Develop solutions that balance miner visibility with weight copying solutions
Explore innovative approaches to detect and disincentivize off-chain weight copying
Design adaptive mechanisms that remain effective across various subnet dynamics

As proposed, the alternative solution in proof of weights removes the associated limitations of currently implemented mitigations while providing a full security assurance that weights are being calculated correctly by validators participating.

Ultimately, the challenge of weight copying underscores the complexity of creating truly decentralized AI infrastructure. As Bittensor continues to grow and evolve, ongoing research, development, and community engagement will be essential in refining existing strategies and developing new solutions to ensure the network’s long-term viability and integrity. Solutions such as proof of weights offer an alternative to the implemented weight copying mitigations and integration at a protocol level should be explored to provide subnet owner’s the ability to enforce validator participation through the use of zero knowledge proofs.

Definitions

“Hyperparameters” — Various chain level adjustment variables, configurable by a subnet owner. Key hyper parameters relevant to this discussion are:

commit_reveal_weights_enabled— Determines whether the commit reveal feature is enabled on a specific subnet
commit_reveal_weights_interval— Defines the interval of weight encryption
weights_rate_limit — Defines the minimum duration of blocks between weight setting
liquid_alpha_enabled — Determines whether the consensus based weights (“liquid alpha”) feature is enabled on a specific subnet
alpha_values — Specifies an upper and lower alpha bound for consensus based weights

“OTF” — The OpenTensor Foundation

“Weights” — A normalized array of floating point values which determines the economic reward distribution between miners in a subnetwork

“Subnet” / “Subnetwork” — A network within the bittensor blockchain which contains validators, miners and an owner who provides an incentivization mechanism for the subnet to follow.

“Owner” — The wallet address who was responsible for registering the subnetwork, and controls hyperparameters within that subnet.

“Commit Reveal” — A blockchain mechanism where validators submit encrypted data that is only revealed and decrypted after a set number of blocks, to prevent realtime on-chain weight copying.

“Consensus Based Weights” or “Liquid Alpha” — adjusts validators’ rewards based on how closely their submissions align with the average consensus of all validators, promoting accuracy and collaboration.