Incentives

The Swarm network comprises many independent nodes, running software which implements the Swarm protocol. It is important to realise that even though nodes run the same protocol, the emergent behaviour of the network is not guaranteed by the protocol alone; as nodes are autonomous, they are essentially "free" to react in any way they desire to incoming messages of peers. It is, however possible to make it profitable for a node to react in a way that is beneficial for the desired emergent behaviour of the network, while making it costly to act in a way that is detrimental. Broadly speaking, this is achieved in Swarm by enabling a transfer of value from those nodes who are using the resources of the network (net users) to those who are providing it (net providers).

3.1 Sharing bandwidth

3.1.1 Incentives for serving and relaying

Forwarding Kademlia and repeated dealings

The retrieval of a chunk is ultimately initiated by someone accessing content, and therefore all costs related to this retrieval should be borne by them. While paid retrievals may not sound like a popular idea when today’s web is "free", many of the problems with the current web stems from consumers’ inability to share the costs of hosting and distribution with content publishers directly. In principle, the retrieval of a chunk can be perceived as a functional unit where the storer acts as a service provider and the requestor as consumer. The provider renders a service to the consumer, and in return, the consumer should provide compensation. Such a direct transaction would normally require that transactors are known to each other, so if we are to maintain the anonymity requirement on downloads, we must conceptualise compensation in a novel way.

As we use forwarding Kademlia, chunk retrieval subsumes a series of relaying actions performed by forwarding nodes. Since these are independent actors, it is already necessary to incentivise each act of relaying independently. Importantly, if only instances of relaying matter, then transactors are restricted to connected peers, regardless of the specifics of accounting and compensation (see 3.2.1). Given that the set of ever connected peers forms a quasi-permanent set across sessions, we are able to frame the interaction within the context of repeated dealings. Such a setting always creates extra incentive for the parties involved to play nice. It is reasonable to exercise preference for peers showing an untainted historical record. Moreover, since the set of connected peers is logarithmic in the network size, any book-keeping or blockchain contract that the repeated interaction with a peer might necessitate is kept manageable, offering a scalable solution. Turning the argument around, we could say that keeping balances with a manageable number of peers, as well as the ambiguity of request origination are the very reasons for nodes to have limited connectivity, i.e., that they choose leaner Kademlia bins.

Charging for backwarded response

If accepting a retrieve request already constitutes revenue for forwarding nodes, i.e. an accounting event crediting the downstream peer is triggered before the response is delivered, then it creates a perverse incentive not to forward the requests. Conditioning the request revenue fulfilment on successful retrieval is the natural solution: The accounting event is triggered only when a requested chunk is delivered back to its requestor, see Figure 3.2.

If, however, there is no cost to a request, then sending many illegitimate requests for non-existing chunks (random addresses) becomes possible. This is easily mitigated by imposing sanctions on peers that send too many requests for chunks that do not exist (see 3.2.5).

Once a node initiates (starts or forwards) a request, it commits to pay for that chunk if it is delivered within the defined time to live (TTL), therefore there is never an incentive to block timely deliveries when the chunk is passed back. This commitment also dissuades nodes from frivolously asking too many peers for a chunk, since, if multiple peers respond with delivery, each must be paid.

3.1.2 Pricing protocol for chunk retrieval

Next, we describe the protocol which nodes use to communicate their price for delivering chunks in the Swarm network. Building on top of this protocol, strategies can then be implemented by nodes who wish to compete in the market with other nodes in terms of quality of service and price.

Price discovery

The main merit of the protocol is that it allows for the mechanisms of price discovery to be based only on local decisions, which is essential for the following reasons: (1) Bandwidth costs are not homogeneous around the world: Allowing nodes to express their cost structure via their price will enable competition on price and quality, ultimately benefiting the end-user. (2) The demand for bandwidth resource is constantly changing due to fluctuations in usage or connectivity. (3) Being able to react directly to changes creates a self-regulating system.

Practically, without this possibility, a node operator might decide to shut down their node when costs go up or, conversely, end-users might overpay for an extended period of time when costs or demand decrease and there is no competitive pressure for nodes to reduce their price accordingly.

Bandwidth is a service that comes with "instant gratification" and therefore immediate acknowledgement and accounting of its cost are justified. Since it is hard to conceive of any externalities or non-linearities in the overall demand and supply of bandwidth, a pricing mechanism which provides for both (1) efficient and immediate signalling and (2) competitive choice with minimal switching and discovery costs is most likely to accommodate strategies that result in a globally optimal resource allocation.

To facilitate this, we introduce a protocol message that can communicate these prices to upstream peers. We can conceptualise this message as an alternative response to a request. Nodes maintain the prices associated with each peer for each proximity order. Therefore, when they issue a retrieve request, they already know the price they commit to pay as long as the downstream peer successfully delivers the valid chunk within the time-to-live period. However, there is no point in restricting the price signal just to responses: If, for whatever reason, a peer decides to change the prices, it is in the interest of both parties to exchange this information even if there is a request to respond to. In order to prevent DoS attacks by flooding upstream peers with price change messages, the rate of price messages is limited. Well-behaved and competitively priced nodes are favoured by their peers; if a node’s prices are set too high or their prices exhibit a much higher volatility than others in the network, then peers will be less willing to request chunks from them.¹

For simplicity of reasoning, we posit that the default price is zero, corresponding to a free service (altruistic strategy).

Differential pricing of proximities

If the price of a chunk is the same at all proximities, then there is no real incentive for nodes to forward requests other than the potential to cache the chunk and earn revenue by reselling it. This option is hard to justify for new chunks, especially if they are in the shallow proximity orders of a node where they are unlikely to be requested. More importantly, if the pricing of chunks is uniform across proximity orders, colluding nodes can generate chunk traffic and pocket exactly as much as they send, virtually a free DoS attack (see Figure 3.3).

To mitigate this attack, the price a requestor pays for a chunk needs to be strictly greater than what the storer node would receive as compensation when a request is routed from requestor to storer. We need to establish a pricing scheme that rewards forwarding nodes, hence, this necessitates the implementation of differential pricing based on node proximity. If the price of delivery is lower as a node gets further from the chunk, then the request can always be sent that way because the forwarder will pocket the difference and therefore make a profit. This means that an effective differential scheme will converge to a pricing model where delivery costs more if the peer is further from the chunk address, i.e. rewards for chunk deliveries are a decreasing function of proximity.

Due to competitive pressure along the delivery path and in the neighborhood, we expect that the differential a node is applying to the downstream price to converge towards the marginal cost of an instance of forwarding. The downstream price is determined by the bin density of the node. Assuming balanced bins with cardinality $2^n$, a node can guarantee to increase the proximity order by $n$ in one hop. At the same time, it also means that they can spread the cost over $n$ proximity bins pushing the overall price down.

Uniformity of price across peers

Take a node $A$ that needs to forward a request for a chunk which falls into $A$’s PO bin $n$. Notice that all other peers of $A$ in bins $n+1,n+2,...$, just like $A$ also have the chunk in their PO $n$. If any of these peers, say $B$, has a price for proximity order $n$ cheaper than $A$, $A$ can lower its price for PO bin $n$, forward all increased traffic to $B$ and still pocket the difference, see Figure 3.4. Note that this is not ideal for the network as it introduces a spurious hop in routing, i.e., in relaying without increasing the proximity.

Similarly, peers of $A$ in shallower bins that have lower price than $A$ for their respective bins, e.g., $B$ in bin $n-1$ being cheaper than $A$ in bin $n$, then $A$ can always forward any request to $B$ and pocket the difference.

Now let’s assume that all peers have price tables which are monotonically decreasing as PO decreases. Also assume that shallower bins have higher prices for bins less than $n$, and all deeper peers in bins higher than $n$ have the same prices for $n$. Let $B$, $C$, $D$ and $E$ be the peers in bin $n$ densely balanced. $A$ wants to forward a chunk to a peer so that the PO with its target address increases by 3. If peers $B$ and $C$ attempt to collude against $A$ and raise the price of forwarding chunks to bin $n+3$, they are still bound by $D$ and $E$’s price on PO bin $n+2$. In particular, if they are lower than $B$ and $C$ for $n+3$.

Such price discrepancies offer nodes an arbitrage opportunity; the strategy to forward to the cheapest peer will direct traffic away from expensive peers and increase traffic for cheaper ones. As a consequence, prices will adjust.

All else being equal, this price arbitrage strategy will achieve (1) uniform prices for the same proximity order across the network, (2) prices that linearly decrease as a function of proximity (3) nodes can increase connectivity and keep prices lower. In this way, incentivisation is designed so that strategies that are beneficial to individual nodes are also neatly aligned in order to benefit the health of the system as a whole.

Bin density

Charging based on the downstream peer’s proximity to the chunk has the important consequence that the net revenue earned from a single act of non-local delivery to a single requestor is a monotonically increasing function of the difference between the chunk’s proximity to the node itself and to the peer the request was forwarded to. In other words, the more distance we can cover in one forward request, the more we earn.

This incentive aligns with downloaders’ interest to save hops in serving their requests, leading to lower-latency delivery and reduced bandwidth overhead. This scheme incentivises nodes to keep a gap-free balanced set of addresses in their Kademlia bins as deep as possible (see Figure 2.4), i.e, it is better for a node to keep dense Kademlia bins than thin ones.

Nodes that are able to maintain denser bins actually have the same cost as thinner ones, but saving hops will improve latency and make the peer more efficient. This will lead to the peer being preferred over other peers that have the same prices. Increased traffic essentially can also lead to bandwidth contention, which eventually allows the raising of prices.

Note that such arbitrage is more efficient in shallow bins where the number of peers to choose from is higher. This is in major opposition to deep bins in the area of responsibility. If a node does not replicate its neighbourhood’s chunks, some of these chunks will need to be requested by the node closer to the address, but further from the node. This will only be possible at a loss. An added incentive for neighbours to replicate their area of responsibility is discussed in 3.4. With the area of responsibility stored however, a node can choose to set their price arbitrarily.

Caching and auto-scaling

Nodes receive a reward every time they serve a chunk, therefore the profitability of a chunk is proportional to its popularity: the more often a chunk is requested, the higher the reward relative to the fixed cost of storage per time unit. When nodes reach storage capacity limits and need to decide which chunks to delete, a rational agent seeking to maximise profit would opt to remove chunks with the lowest profitability. A reasonably² good predictor for this is the age of last request. In order to maximise the set of chunks to select from, nodes engage in opportunistic caching of the deliveries they relay as well as the chunks they sync. This results in popular chunks being more widely spread and faster served, transforming the whole of Swarm into an auto-scaled and auto-balanced content distribution network.

Non-caching nodes

Any scheme that ensures relaying nodes make a profit creates a positive incentive for forwarding-only non-caching nodes to enter the network. Such nodes are not inherently beneficial to the network as they are creating unnecessary bandwidth overhead. On the one hand, their presence could, in principle, unburden storer nodes from relaying traffic, so using them in shallow bins may not be detrimental. On the other hand, closer to neighbourhood depth, their peers will favour a caching/storing node to them because of their disadvantage at least for chunks in their hypothetical area of responsibility. Non-caching nodes can also contribute to increase anonymity (see 2.3.1).

3.1.3 Incentivising push-syncing

The push-sync (see 2.3.2) protocol ensures that chunks uploaded into the network arrive at their designated address. In what follows, we will explain how forwarding is incentivised. ³

Push-syncing is analogous to the retrieval protocol in the sense that their respective message exchange sequences travel the same route. The delivery of the chunk in the push-sync protocol is analogous to a retrieval request and, conversely, the statement of custody receipt in push-sync is analogous to the chunk delivery response in retrieval.

In principle, push-syncing could be left without explicit forwarding incentives. Due to the retrieval protocol, as nodes expect chunks to be found in the neighbourhood of their address, participants in Swarm are at least weakly incentivised to help deliver uploaded chunks to their destination. However, we need to provide the possibility that chunks are uploaded via nodes further from it than the requestor (light nodes or retries). Thus, if push-syncing was free, nodes could generate wasteful amounts of bandwidth.

Requiring payment only for push-sync delivery by downstream peers would put the forwarder in a position to bargain with a storer node regarding the delivery of the chunk. The possession of a chunk is valuable for the prospective storer node because there is also a system of rewards for storage (see 3.4). Based on this, the forwarder node could, in theory, hold onto the chunk unless the storer node pays marginally more than the value of possessing that chunk, factoring in the profit potential due to storage incentives. In particular, since forwarders on the route from the uploader are not numerous, any profits generated from a storage reward mechanism might be captured by these forwarding nodes.

Instead, in push-sync, by making the statement of custody receipt a paid message, the roles switch. The forwarder node is no longer in the position to bargain. To understand why, let’s consider a scenario where a forwarding node tries to hold on to a chunk to negotiate a price for pushing it to a storer node. In this case, the uploader will not get a statement of custody receipt within the expected time frame. As a result, the uploader will assume that the attempt has failed and re-upload the chunk via a different route. Now, suddenly the original forwarding node is forced to compete with another forwarding node in getting compensation for their bandwidth costs. Since all forwarding nodes are aware of this dynamic, emergent behaviour will produce a series of peers that are willing to forward the chunk to the storer node for a relatively small compensation and the bandwidth costs incurred. This eliminates the need for the original forwarding node to try and bargain with the storer node in the first place: Instead, they can generate a small profit immediately by simply returning the statement of custody receipt.

This scheme highlights why the incentivisation of the two protocols relies on the same premises: there are many sellers (forwarders) and only one buyer (uploader) for a homogeneous good (the statement of custody receipt). As a result, the price of the service (delivering the chunk to the storer) is determined by the sum of the marginal costs of forwarding for each node along the route. At the same time, the storer node can capture all the profits from the storage compensation scheme.

In this way, we can make sure that (1) storers actually respond with receipts, and (2) have a way to detect timed out or unsolicited receipt responses to protect against DoS, see Figure 3.5.

Similar to the retrieval protocol, the pricing in this scheme is expected to vary based on different proximities (see 3.1.2.0). Additionally, as the costs of the nodes in the network fluctuate (depending on capacity utilization and node efficiency), the pricing will also be subject to change over time. Given that the compensation is calculated for one chunk and one shorter message (retrieve request and custody receipt) during the accounting process, we can safely conclude that the price structure for forwarding for both protocols is identical. Consequently, a unified pricing scheme for forwarding can be applied to both protocols, as discussed in 3.1.2.0. What distinguishes push-sync from the retrieval protocol is that, unlike in retrieval where the chunk is delivered back and its integrity can be validated, the accounting event in push-sync is a statement of custody which can be spoofed. Due to the forwarding incentive, nodes may be motivated to withhold forwarding and impersonate a storer node by issuing the statement of custody. This makes it advisable to query (retrieve) a chunk via alternative routes. If such retrieval attempts fail, it may be necessary to try push-syncing chunks through alternative routes.

3.2 Swap: accounting and settlement

This section covers aspects of incentivisation relating to bandwidth sharing. In 3.2.1, we introduce a mechanism to keep track of the data traffic between peers and offer peer-to peer-accounting for message relaying. Subsequently, in 3.2.2, we describe the conditions of compensating for unbalanced services and show how settlement can be achieved. In particular we introduce the concept of a. cheques and the chequebook contract. In 3.2.3, we discuss waivers as an optimisation that allows for additional savings on transaction costs. In 3.2.4 we discuss how an incentivised service of sending in cashing transactions enables zero-cash entry to Swarm and, finally, in 3.2.5 we delve into the fundamental set of sanctions that serve as incentives for nodes to play nice and adhere to the protocols.

3.2.1 Peer to peer accounting

Trón et al. (2016) introduce a protocol for peer-to-peer accounting, called swap. Swap is a tit-for-tat accounting scheme that scales microtransactions. The scheme allows directly connected peers to swap payments or payment commitments. The system’s key characteristics are captured playfully with different mnemonic resolutions of the acronym SWAP:

Swarm accounting protocol

A scheme used by Swarm for keeping a record of reciprocal sharing of bandwidth.

service wanted and provided

Allows service exchange.

settle with automated payments

Send a cheque when payment threshold is exceeded.

send waiver as payment

Debt can be waived in the value of un-cashed cheques.

start without a penny

Zero-cash entry is supported by unidirectional swap.

Service for service

swap allows service for service exchange between connected peers. When there is equal consumption with low variance over time, bidirectional services can be accounted for without the need for any payments. Data relaying is an example of such a service, making Swap well-suited for implementing bandwidth incentives in content delivery or mesh networks.

Settling with payments

In situations where there is high variance or unequal consumption of services, the balance will eventually tilt significantly toward one peer. In such cases, the indebted party issues a payment to the creditor to restore the nominal balance to zero. This process is automatic and justifies the concept of swap as settle (the balance) with automated payments (see Figure 3.6). These payments can be in the form of commitments rather than immediate transactions.

Payment thresholds

To quantify what counts as "significant tilt", the swap protocol requires peers to advertise a payment threshold as part of the handshake: When their relative debt to their peer goes above this threshold, they send a message containing a payment to their peer. It is reasonable for any node to send a message when the debt reaches this level, as there is also a disconnect threshold in place. The disconnect threshold can be set freely by any peer, but it is recommended to choose a value that takes into account the usual variance in accounting balances between the two peers. This can be done by considering the difference between the payment threshold and the disconnect threshold.

Atomicity

Sending the cheque and updating the balance on the receiving side cannot be made an atomic operation without substantial added complexity. For instance, a client could crash between receiving and processing the message, so even if the sending returns with no error, the sending peer can not be sure the payment was received, this can result in discrepancies in accounting on both sides. The tolerance expressed by the difference between the two thresholds ($\mathit{DisconnectThreshold}-\mathit{PaymentThreshold}$) guards against this, i.e. if the occurrence of such crashes is infrequent and happens with roughly equal probability for both peers, the resulting minor discrepancies are filtered out. This mechanism protects nodes from facing sanctions.

3.2.2 Cheques as off-chain commitments to pay

One of the major challenges with conducting direct on-chain payments in a blockchain network is the high transaction costs associated with processing each transaction by every participating node. It is, however, possible to create a payment without presenting this payment on-chain. Such payments are called second-layer payment strategies. One such strategy is deferring payments and processing them in bulk. In exchange for reduced cost, the beneficiary must be willing to incur a higher risk of settlement failure. We argue that this is perfectly acceptable in the case of bandwidth incentivisation in Swarm, where peers will engage in repeated dealings.

The chequebook contract

A simple smart contract called the chequebook contract, introduced in Trón et al. (2016), allows the beneficiary to determine the timing of payments. This contract acts as a wallet that can process cheques issued by its owner. Similar to traditional financial transactions, the issuer signs a cheque specifying the beneficiary, date, and amount, providing it to the recipient as a token of promise to pay at a later date. The smart contract plays the role of the bank. When the recipient wishes to get paid, they "cash the cheque" by submitting it to the smart contract. The contract, after validating the signature, date and the amount specified on the cheque, transfers the amount to the beneficiary’s account (see Figure 3.8). Analogous to the person taking the cheque to the bank to cash it, anyone can send a digital cheque in a transaction to the owner’s chequebook account, initiating the transfer.

The swap protocol specifies that when the payment threshold is exceeded, a cheque is sent over by the creditor peer. Such cheques can be immediately cashed by sending them to the issuer’s chequebook contract. Alternatively, cheques can also be held, which effectively serves as a form of lending on credit, enabling parties to save on transaction costs.

The amount deposited in the chequebook (global balance) serves as collateral for the cheques and is pooled over the beneficiaries of all outstanding cheques. In this simplest form, the chequebook provides the same guarantees as real-world cheques: None. Since funds can be freely moved out of the chequebook wallet at any time, solvency at the time of cashing can never be guaranteed: If the chequebook’s balance is less than the amount specified in a submitted cheque, the cheque will bounce. This is the trade-off between transaction costs and risk of settlement failure.

While, strictly speaking, there are no guarantees for solvency, nor is there an explicit punitive measure in the case of insolvency, a bounced cheque can negatively impact the issuer’s reputation as the chequebook contract records such incidents. On the premise that cheques are swapped in the context of repeating dealings, peers will refrain from issuing cheques beyond their balance. In other words, a node’s interest in keeping a good reputation with their peers serves as a sufficient incentive to maintain its solvency.

Double cashing

Since these digital cheques are files and can therefore be copied, it is crucial to implement measures to prevent the cashing of the same cheque multiple times. Such "double cashing" can be prevented by assigning each cheque given to a particular beneficiary a serial number which the contract will store when the cheque is cashed. The chequebook contract can then rely on the serial number to make sure cheques are cashed in sequential order, thus needing to store only a single serial number per beneficiary.

Alternatively, to address repeated payments to the same beneficiary, the cheques can contain the cumulative total amount ever credited to that beneficiary. The contract maintains a record of the total amount that has been cashed out for each beneficiary, and when a new cheque is submitted, the contract compares the amount on the cheque to the stored total. Cheques with an amount equal to or less than the stored total are ignored, while cheques with a higher total will result in the transfer of the difference to the beneficiary.

This simple trick also makes it possible to cash cheques in bulk because only the most recent "last cheque" needs to be processed, leading to a significant reduction of transaction costs.

Cashing without Ether

Not all peers in Swarm are expected to have the Ether needed to pay for the transaction costs to cash out a cheque. The chequebook allows third parties to cash cheques. The sender of the transaction is incentivised with a reward for the service performed.

3.2.3 Waivers

If the imbalance in the swap channel is due to high variance rather than unequal consumption, after a period of accumulating cheques, the channel balance starts tilting in the opposite direction. Normally, it is now up to the other party to issue cheques to its peer, resulting in uncashed cheques accumulating on both sides. To allow for further savings in transaction costs, it might be desirable to be able to offset these cheques against each other.

Such a process is possible, but it requires certain important changes within the chequebook contract. In particular, cashing cheques can no longer be immediate and must incur a security delay, a concept familiar from other payment channel implementations (Poon and Dryja 2015, Ferrante 2017, McDonald 2017, Tremback and Hess 2015).

Let us imagine a system analogous to cheques being returned to the issuer. Assume peer $A$ issued cheques to $B$ and the balance was brought back to zero. Later, the balance tilts in $A$’s favour, but the cheques from $A$ to $B$ have not been cashed. In the traditional financial world, $B$ could either simply return the last cheque to $A$ or provably destroy it. In our case, it is not so simple; we need some other mechanism by which $B$ commits not to cash that particular cheque. Such a commitment could take several forms; it could be implemented by $B$ signing a message allowing $A$ to issue a new ‘last cheque‘ with a lower cumulative total amount than before, or perhaps $B$ could issue a form of ‘negative‘ cheque for A’s chequebook, effectively offsetting the amount as if a cheque with the same amount had been paid.

These implementations share the characteristics of not allowing the instantaneous cashing of cheques in the chequebook. Upon receiving a cheque-cashing request, the contract must wait to allow the other party to submit potentially missing information about cancelled cheques or reduced totals. To accommodate (semi-)bidirectional payments using a single chequebook, we introduce the following modifications:

• All cheques from user A to user B must contain a serial number.
• Each new cheque issued by A to B must have a serial number higher than the previous one.
• A’s chequebook contract records the serial number of the last cheque that B cashed.
• During the cashing delay, any valid cheque with a higher serial number supersedes any previously submitted cheques, regardless of their face value.
• Any submitted cheque which decreases the payout of the previously submitted cheque is only valid if it is signed by the beneficiary.

With these rules in place, it is easy to see how cheque cancellation would work. Let’s consider the scenario where user $A$ has issued cheques $c_0\dots c_n$ with cumulative totals $t_0\dots t_n$ to user $B$. Suppose that the last cheque $B$ cashed was $c_i$. The chequebook contract has recorded that $B$ has received a payout of $t_i$ and that the last cheque cashed had serial number $i$.

Let us further suppose that the balance starts tilting in $A$’s favour by some amount $x$. If $B$ had already cashed cheque $c_n$, then $B$ would be required to issue a cheque of her own using $B$’s chequebook as the source and naming $A$ as the beneficiary. However, since cheques $c_{i+1}\dots c_n$ are still uncashed, $B$ can instead send to $A$ a cheque with $A$’s chequebook as the source, $B$ as the beneficiary, with serial number $n+1$ and cumulative total $t_{n+1}=t_n-x$. Due to the rules enumerated above, $A$ will accept this as an equivalent payment of amount $x$ from $B$. In this scenario, instead of sending a cheque to $A$, $B$ waives part of their earlier entitlement. This justifies the concept of SWAP as send waiver as payment.

This process can be repeated multiple times until the cumulative total is brought back to $t_i$. At this point, all outstanding debt has effectively been cancelled, and any further payments must be made in the form of a proper cheque from $B$’s chequebook to $A$ (see Figure 3.9).

3.2.4 Zero cash entry

Swap accounting can also work in a one-directional manner. When a party enters the system with zero liquid capital (a newcomer) but connects to a peer with funds (an insider), the newcomer can begin to provide a service (and not use any) in order to earn a positive swap balance.

If the insider has a chequebook, they are able to simply pay the newcomer with a cheque. However, this has a caveat: The newcomer will be able to earn cheques for the services provided but will not have the means to cash them. Cashing cheques requires sending a transaction to the blockchain, and therefore requires gas, unless the node can convince one of its peers to execute the transaction on its behalf. To facilitate this, nodes are able to sign off on a structure that they want to be sent, and then extend the Swap contract with a preprocessing step that triggers payment to the newcomer, covering the transactions gas cost plus a service fee for the sender of the transaction. The newcomer’s cheque may be cashed by any insider (see Figure 3.10). This feature justifies the concept of SWAP as start without a penny, send with a peer.

The possibility to earn small amounts of money without starting capital is crucial, as it provides a way for new users to get access to Swarm without the need to purchase the token. This benefit extends to the Ethereum ecosystem in general: using Swarm, anybody can earn small amounts of money to start paying the gas to fuel their dapps, without the need to go through a painful process of acquiring tokens prior to onboarding.

3.2.5 Sanctions and blacklisting

This section complements the SWAP scheme with additional incentives and protection against foul play.

Protocol breach

In a peer-to-peer trustless setting, implementing nuanced sanctions against undesired peer behaviour can be challenging. However, when the basic rules of interaction are violated, the node that detects it can simply disconnect from that peer. In order to avoid deadlocks due to attempted reconnection, the sanctions imposed on transgressive nodes also include recording the peer’s address into a blacklist. This simple measure is enough to provide a clear disincentive to nodes seeking to exploit the protocol.

Excessive frivolity

Both retrieval and push-sync protocols have an incentive structure where only the response to a request generates income. Although this creates a strong incentive to play ball, it may also be necessary to take measures to ensure that nodes are not able to spam the network with frivolous requests that have no associated cost. In the case of push-syncing, it is especially important not to allow chunks to expunge others at no cost. This will form the topic of a later section where we introduce postage stamps (see 3.3).

In the case of pull-sync retrieval, the potential attack consists of requesting non-existing chunks and causing downstream peers to initiate a lot of network traffic, as well as some memory consumption, due to requests being persisted during the time-to-live period. Surely, there is a possibility that a requestor may unknowingly request non-existing chunks, and what is more, the requested chunk could have been be garbage-collected in the network, in which case, the requestor may have acted in good faith.

To mitigate this, each node maintains a record of the number of retrieve requests from each of its peers and then updates the relative frequency of failed requests, i.e. requests that have timed out even though the node in question had forwarded it. If the proportion of failed requests to successful requests exceeds a certain threshold, sanctions are imposed on the peer: it is disconnected and blacklisted.

By remembering the requests they have forwarded, nodes can distinguish legitimate responses from a potential DoS attack: for retrieval, if the chunk delivered does not fulfil an open request, it is considered unsolicited; for push-sync, if a statement of custody response does not match an existing entry for forwarded chunk, it is considered unsolicited.

Timeouts are crucial here. After the time-to-live period for a request has passed, the record of the open request can be removed. Any response received after this point is considered unsolicited, as it is indistinguishable from messages that were never requested.

To account for slight discrepancies in time measurement, once again a small percentage of illegitimate messages are tolerated from a peer before they are disconnected and blacklisted.

Quality of service

Beyond the rate of unsolicited messages, nodes can cause grievances on other ways, such as by setting high prices, having low network throughput, or long response latencies. Similarly to excessively frivolous requests, there is no need for a distinction between malicious attacks or sub-optimal (poor quality, overpriced) service provided in good faith. As a result, mitigating quality of service issues is discussed in the context of peer selection strategies in forwarding and connectivity.

Blacklisting

Blacklisting is a strategy that complements disconnection as a measure against peers. It is supposed to extend our judgment expressed in the act of disconnection that the peer is unfit for business. Blacklists serve as a reference when accepting incoming connections as well as in the peer suggestion strategy of the connectivity driver. On the one hand, blacklisting can save the node from being deadlocked in a cycle of malicious peers trying to reconnect. On the other hand, care must be taken not to blacklist peers acting in good faith, as this could negatively impact network connectivity.

3.3 Postage stamps

A postage stamp is a verifiable proof of payment associated with a chunk witnessed by the signature of its owner. It serves two purposes: preventing frivolous uploads by imposing an advance cost, and indicating the relative importance of a chunk by ascribing a specific amount of BZZ to it. Storer nodes can then use this information to prioritise which chunks to retain and serve, as well as which ones to garbage-collect in case of limited capacity.

In this section, we first introduce the concept of a postage batch, which allows for the bulk purchase of stamps (3.3.1). In 3.3.2, we explain how limited issuance is represented and enforced. In 3.3.3, we introduce the notion of a reserve and outline the rules governing how storer nodes can maximise its utilisation. We conclude in 3.3.4 with exploring the relationship between reserved capacity, effective demand, and the number of nodes and their impact on the data availability.

3.3.1 Purchasing upload capacity

Uploaders can acquire postage stamps in bulk by purchasing a postage batch from the postage smart contract on the Ethereum blockchain. The creation of a postage batch involves initiating a transaction to the batch creation endpoint of a contract, along with an amount of BZZ tokens and transaction data that specifies some parameters. As the transaction is processed, the postage contract registers a new batch entry with the following pieces of information:

batch identifier

A randomly generated ID that serves as a reference for this batch.

batch depth

Base 2 logarithm of the issuance volume, i.e., the number of chunks that can be stamped using this batch.

owner address

The Ethereum address of the owner entitled to issue stamps, as per the transaction data sent along with the creation or the transaction sender if not specified.

per-chunk balance

The total amount sent along with the transaction divided by the issuance volume.

mutability

A boolean flag indicating whether the storage slots of the batch can be reassigned to another chunk with a stamp if its timestamp is older.

uniformity depth

the base 2 logarithm of the number of equal-sized buckets the storage slots are arranged in.

The postage contract provides endpoints to users to modify the per-chunk balance of batches. This allows users to add funds to extend the validity period of the stamps issued by the batch (top-up) or add volume to decrease it (dilute). While anyone can choose to top up the balance of a batch at a later date, only the owner has the authority to dilute it.⁴

Owners issue postage stamps in order to attach them to chunks. Each batch has a number of storage slots effectively arranged into a number of equal-sized buckets. Issuing a stamp means to assign a chunk to a storage slot. A stamp is a data structure comprising the following fields (see Figure 3.11):

chunk address

The address of the chunk the stamp is attached to.

batch identifier

The ID referencing the issuing batch (generated at its creation).

storage slot

A bucket index referencing one of the equal-sized buckets of the batch, and a within-bucket index referencing the storage slot the chunk is assigned to.

timestamp

The time the chunk is stamped.

witness

The batch owner’s signature attesting to link between the storage slot and the chunk.

A postage stamp’s validity can be checked by verifying that it scores all true on the following five attributes:

authentic

The batch identifier is registered in the postage contract’s storage.

alive

The referenced batch has not yet exhausted its balance.

authorised

The postage stamp is signed by the address specified as the owner of the batch.

available

The referenced storage slot is within valid range based on the batch depth, and, in the case of an immutable batch, has no duplicates.

aligned

The referenced storage slot has the bucket specified and it aligns with the chunk address stamped.

All this can be easily checked by nodes in Swarm only using information available on the public blockchain (read-only endpoints of the postage contract). When a chunk is uploaded, the validity of the attached postage stamp is verified by forwarders along the push-syncing route (see Figure 3.12).

The normalised per-chunk balance of a batch is calculated as the batch inpayment divided by the batch size in chunk storage slots. The chunk balance is interpreted as an amount pre-committed to be spent on storage. The balance decreases with time as if storage rent was paid for each block at the price dictated by the price oracle contract.

This system of prepayment for storage eliminates the need to speculate on the future price of storage or fluctuations in currency exchange rates. At the cost of decreased certainty about the expiration date, one gains resilience against price volatility. On top of this, uploaders can enjoy the luxury of non-engagement by tying up more of the batch balance; while it serves as collateral against price increase, if that does not happen the funds can still be used up (for storing).

3.3.2 Limited issuance

Purchasing a postage batch effectively entitles the owner to issue a fixed amount of postage stamps against the batch ID called the issuance volume or batch size. It is restricted to the powers of 2 and is specified using the base 2 logarithm of the amount which is called batch depth. The storage slots within a batch are arranged into buckets, and each slot is assigned an index within its respective bucket. The number of buckets is restricted to the powers of 2 and is specified using its base 2 logarithm called uniformity depth. To ensure the size limitation of a batch with batch depth $d$ and uniformity depth $u$, the following conditions must be met:

• the bucket index ranges from $0$ to $2^u-1$.
• the within-bucket index ranges from $0$ to $2^{d-u}-1$.
• there are no duplicate indexes.

While the first two criteria are easily verifiable by any third party, the last one is not. In order for index collisions to be detectable by individual storer nodes, the uniformity depth must be large enough to fall within the nodes’ area of responsibility. As long as this condition is maintained, all chunks in the same bucket are guaranteed to land in the same neighbourhood, and, as a result, duplicate assignments can be locally detected by nodes (see Figure 3.13).

In order to keep their stamps collision-free, uploaders need to maintain counters for how many stamps they have issued for each bucket within a batch and ensure that the number does not exceed the maximum bucket size.

In general, the most efficient utilisation of a batch is by filling each bucket to its capacity. Continued non-uniformity (i.e., targeted issuance) leads to underutilised batches, and therefore a higher unit price for uploading and storing each chunk. This feature has the desired side effect that it imposes an upfront cost to non-uniform uploads: the more concentrated the distribution of chunks in an upload, the more unused storage slots in the postage batch. In this way, we ensure that targeted denial-of-service attacks against a neighbourhood (i.e., uploading a disproportionate number of chunks in a particular address range) is costly since the inert cost (due to the degree of under-utilisation of the batch) is exponential in the depth of the skew.

Beyond DoS protection, postage stamps can serve as a fiduciary signal indicating how much it is worth for a user to persist a chunk in Swarm. In particular, the per-chunk balance of batches can provide the differential a priori bias, determining which chunks should be protected from garbage collection when there is no evidence available to predict their profitability from swap.

3.3.3 Rules of the reserve

The reserve refers to a fixed size of storage space dedicated to storing the chunks within a node’s area of responsibility. Chunks within the reserve are protected against garbage collection as long as they have valid postage stamps. When batches expire, i.e., their balance is fully depleted, the chunks they stamped are no longer protected from eviction. Their eviction from the reserve frees up some space that can accommodate new or more distant chunks belonging to valid batches.

From the point of view of incentives, chunks within the same proximity order and the same batch are considered equivalent. When it comes to eviction due to batch expiry, these equivalence classes, called batch bins, are handled as a single unit: the chunks in a batch bin are evicted from the reserve and inserted into the cache in one atomic operation.

Assuming a global oracle for the unit price of rent and a fixed reserve capacity prescribed for nodes, the content of the reserve is coordinated according to a set of constraints on batch bins called the rules of the reserve:

1.

if a batch bin of a certain PO is reserved, then all batch bins at closer proximity orders (higher PO) are also reserved.

2.

if a batch bin is reserved at a certain proximity order (PO), then all batch bins at the same PO belonging to batches with a higher per-chunk balance are also reserved.

3.

the reserve should not exceed capacity.

4.

the reserve should be maximally utilised, i.e, it cannot be extended while still adhering to rules 1-3.

The first rule means the reserve is closed upwards for PO, which encodes a global preference for storing chunks that are closer to the node’s address. This is incentivised by routing: keeping the closest chunks, a node will maximise the number of receipts it can issue and the number of retrieve requests it can respond to. Additionally, storing closer chunks ensures wider coverage within the neighbourhood, even when the neighbourhood no longer provides the desired redundancy.

The second rule expresses the constraint that the reserve for a PO is upward closed for per-chunk balance. This constraint reflects a secondary preference among chunks of the same proximity, favouring those that are stamped using a batch with higher per-chunk balance. This is incentivised by the differential absolute profit that chunks promise: due to the constraint that balances are not revocable, chunks with higher balances expire later and therefore provide greater absolute profit to storers compared to chunks with earlier expiration dates, despite both paying the same rent during their period of validity.⁵

When a new chunk arrives in Swarm through pull-sync, push-sync, or upload, the validity of the attached postage stamp is verified. If the PO of the chunk is lower than the batch depth, the node inserts the chunk into the garbage collection index, indicating that it can be subject to garbage collection. On the other hand, if the PO is equal to or greater than the batch depth, the chunk is considered to be in the reserve. If the reserve size exceeds the designated capacity, a number of batch bins are identified whose combined size is sufficient to cover the excess. These batch bins will be evicted from the reserve, reducing its size to meet the capacity requirement.

3.3.4 Reserve depth, storage depth, neighbourhood depth

Reserve depth

The potential demand for chunks to be stored in the DISC is quantified by the total number of storage slots in valid batches. This is calculated as the sum of the sizes of all non-expired batches. Since the batches and their balances are recorded in the postage contract, the reserved size of the DISC is agreed upon through consensus.⁶

The reserve depth is determined by taking the base 2 logarithm of the DISC reserve size and rounding it up to the nearest integer. It represents the shallowest PO at which disjoint neighbourhoods are collectively able to accommodate the volume of data corresponding to the total number of paid chunks. This assumes that nodes within the neighbourhood have a fixed prescribed storage capacity to store their share of the reserve.

The reserve depth also serves as the safe lower bound for pull-syncing, i.e, the farthest bin a neighbourhood needs to synchronise to guarantee storing the reserve. Conversely, if any neighbourhood marked by reserve depth has no nodes in it, the swarm is not working correctly, i.e., chunks with valid stamps are not being protected from getting lost. See Figure 3.15.

Storage depth

The effective demand for chunks to be stored in the DISC corresponds to the total number of chunks actually uploaded. While each chunk in the reserve is associated with a valid postage batch and assigned to a storage slot, it is possible for a postage batch to have unassigned storage slots. As a result, the number of chunks actually stored in the DISC may be less than the DISC reserve size.

The effective area of responsibility is marked by the proximity order of the farthest batch bin in the reserve, assuming the node complies with the rules of the reserve.

A node’s storage depth is defined as the shallowest complete bin, i.e., the lowest PO at which a compliant reserve stores all batch bins. Unless the farthest bin in the node’s reserve is complete, the storage depth equals the reserve’s edge PO plus one.

The storage depth is the optimal lower bound for pull-syncing, i.e, it indicates the farthest bin the node needs to synchronise with its neighbours to achieve maximum reserve utilisation.⁷ Maximum reserve utilisation should be incentivised as part of the storage incentives.

The gap between actual storage depth and the reserve depth exists because of the bulk purchase of stamps. Since entire batches of stamps reserve storage slots that are assigned to chunks only at later times when they are actually uploaded, the batch utilisation rate can be substantially less than 1. Storage depth and reserve depth will be the same only when all batches are fully utilised.

Neighbourhood depth

Swarm has a requirement on local replication, which states that each neighbourhood designated by the storage depth should contain a minimum of four nodes. If neighbourhoods were made of one node, then the outage of that one node would make the chunks in the node’s area of responsibility not retrievable. With two nodes in a neighbourhood, we significantly improve resilience against ad-hoc outages, but because of connectivity latencies a two-peer neighbourhood may still result in an unstable user experience. The ideal scenario is to have four nodes per full connectivity neighbourhood, which prompts the following definition: neighbourhood depth for a particular node is the highest PO $d$ such that the address range designated by the $d$-bit-long prefix of the node’s overlay contains at least 3 other peers.

Figure 3.15 details the potential relative orders of the three depths and their consequences on the health, efficiency, and redundancy of the swarm.

3.4 Fair redistribution

The system of positive⁸ storage incentives in Swarm aims to redistribute to storage providers the BZZ tokens that uploaders deposited within the postage contract.⁹ The overall balance on the contract covers the reward pot which represents the cumulative storage rent across all postage batches for a particular period. The storage rent must be redistributed to storage providers in a way that guarantees that their earnings are proportional to their contribution, weighing in storage space, quality of service, and length of commitment.

The procedure for redistribution is best conceived of as a game orchestrated by a suite of smart contracts on the blockchain. Nodes earn the right to play through participation in storing and syncing chunks, and the winners can claim their rewards by initiating a transaction with the game contract.

In section 3.4.1, we formulate the idea of redistribution in terms of probabilistic outpayments to allow an easy proof of fairness. We then proceed to outline the mechanics of the redistribution game in 3.4.2. Sections 3.4.3 and 3.4.4 explain how we enforce maximum utilisation of dedicated storage for persisting relevant content redundantly. We conclude in 3.4.5 by discussing how to interpret certain aspects of the game as price signals that render the network self-regulating through automatic price discovery.

3.4.1 Neighbourhoods, uniformity and probabilistic outpayments

In this section, we argue that the efficient use of postage batches incentivises a balanced chunk distribution which, in turn, gives rise to uniform storage depth across neighbourhoods. We then explain how this enables a fair system of redistribution using probabilistic outpayments.

Assuming an oracle that sets the unit price of storage, it becomes possible to calculate the storage rent due for a specific period of time for a given batch. The number of rent units for a batch is the result of multiplying the size of the batch by the number of blocks in the specified period. The price of rent is calculated from the number of rent units multiplied by the unit price.¹⁰ The total storage rent cumulated over all batches for the period between two outpayments constitutes the reward pot for the round.

Instead of dividing the reward pot among neighbourhoods regularly, the entire reward pot can be transferred to (representative nodes in) one target neighbourhood in each round. This probabilistic outpayment scheme ensures fairness among neighbourhoods, as long as the probability of selecting a neighbourhood as the target corresponds to its relative contribution to the overall network storage. Given a constant prescribed reserve capacity and replication of the reserve content by nodes within a neighbourhood, each neighbourhood, defined by storage depth, contributes equally to the network.

In Section 3.3, we mentioned that uploaders have a strong incentive to use their postage batch in a way that the chunks they stamp with it are uniformly distributed across the address space. This being true of all batches creates a situation that chunks are uniformly distributed across the DISC. In particular, the sets of chunks sharing a common prefix are expected to be roughly equal in size. Therefore we expect nodes to fill their prescribed reserve capacity with chunks at the same proximity order, irrespective of their location in the address space, i.e., the storage depth is uniform across nodes and therefore across neighbourhoods.¹¹ With neighbourhoods at equal depth, uniform sampling of neighbourhoods can be modelled by choosing the neighbourhood which contains an anchor (called the the neighbourhood selection anchor) randomly dropped in the address space (see Figure 3.16).

3.4.2 The mechanics of the redistribution game

The collaborative effort among peers to store data redundantly for the network’s benefit is underpinned by a Schelling game aimed at proving that the peers agree on the chunks they need to store and they do, in fact, store them. The redistribution game is orchestrated by the game contract, one of the building blocks of the system of 4 smart contracts which collectively drive the swarm storage incentive system (see Figure 3.17):

Postage contract

serving as the batch store to sell postage batches to uploaders, keeping track of batch balances, batch expiry, storage rent, and the reward pot itself.

Game contract

orchestrates the redistribution rounds interacting with potential winners accepting commit, reveal, and claim transactions from storage providers in selected neighbourhoods.

Staking contract

manages a stake registry, maintaining committed stake and stake balance for nodes based on their overlay; enables freezing and slashing of stake, as well as withdrawal of surplus balance for stakers.

Price oracle

maintains the unit price of storage rent, accepts signals from the game contract to dynamically adjust according to supply and demand, and provides current price oracle service for the other three contracts.

The game is structured as a sequence of rounds. Each round lasts for a fixed number of blocks and recurs periodically. A round consists of 3 phases: commit, reveal, and claim.¹² The phases are named after the type of transaction the smart contract expects during that particular phase and that nodes from the selected neighbourhood need to submit.¹³ See Figure 3.18

Once the reveal phase is over, the neighbourhood selection anchor becomes known. Nodes that have the anchor within their respective neighbourhoods¹⁴ are eligible to participate in the following round (see Figure 3.16).

The storer nodes within a neighbourhood are assumed to have consensus over the chunks that make up their reserve and provide evidence, known as proof of entitlement, to the blockchain (discussed below in detail in 3.4.4). In such a game, the Nash-optimal strategy for each node is to follow the protocols and coordinate with others to ensure that all neighbouring peers arrive at the same proof of entitlement based on the shared information. Since the proof of entitlement needs to be consensual but unstealable,¹⁵ a commit/reveal scheme must be used.

During the commit phase, nodes within a neighbourhood will apply by submitting the reserve commitment obfuscated with an arbitrary key (that they later reveal). The smart contract receiving the commit transaction verifies that the node is staked, i.e., the registry of the staking contract contains an entry for the node’s overlay with a stake value that is higher than the minimum required stake.

During the reveal phase, nodes that had previously committed to a reserve now reveal their commitment by submitting a transaction containing their reserve commitments, storage depth, overlay address, and the key they used to obfuscate the commit. Upon receiving the reveal transaction, the contract verifies that the revealed data, when serialised, does indeed hash to their commitment. It is also checked if the node belongs to the neighbourhood designated by the neighbourhood selection anchor, i.e., is within the storage depth provided in the reveal.

In the claim phase, the winner node is required to submit a claim transaction.¹⁶ First, in order to determine the outcome of the Schelling game, one reveal is selected from the reveals submitted during the reveal phase.¹⁷ The selected reveal represents the truth; the set of applications that agree with the selected one represent the honest peers of the neighbourhood, the ones who disagree are considered liars, while those committers that did not reveal or revealed invalid data are classified as saboteurs. Honesty is incentivised by the fact that liars and saboteurs get punished. In what follows we introduce staking that is needed for both the selection processes and the punitive measure.

3.4.3 Staking

Neighbours with shared storage

In order to provide robust protection against accidental node churn, i.e., to ensure the retrievability of chunks from a neighbourhood in the face of some nodes being offline, the swarm requires a number of independent storers in each neighbourhood to physically replicate content. If payout was given to each node that shows proof of entitlement, operators could exploit the system by creating spurious nodes for the sole purpose of applying for the reward. Although measures can be implemented to enforce that these spurious nodes must be operating on the network, operators may still opt to run several nodes yet share their storage on a single hardware. The incentive system must ensure that storage providers do not adopt this strategy. To this end, we introduce staking.

Stakes serve as weights used by the contract to determine the true reserve commitment (truth selection) and to identify the winner among honest nodes (winner selection). The probability of winning is determined by peers’ relative stakes, making stake an additive factor. Consequently, operators’ profits depend solely on their total stake within the neighbourhood. Given the costs associated with running a node, operators will have no motivation to divide their stake among multiple nodes sharing the same storage hardware.

Committed stake and stake balance

When registering in the staking contract, stakers commit to a stake denominated in rent units called committed stake. The committed stake amount must have a lower bound.¹⁸ The amount sent with the transaction is recorded and serves as collateral called stake balance. Stakes can be created or topped up at any time, but the update time and amount are recorded. Participation is limited to peers whose stake has not changed recently, ensuring that stakes cannot be altered after learning about the selected neighbourhood. When querying the stake of a node, the contract returns the absolute committed stake in BZZ. This value is calculated as taking either (1) the committed stake in rent units multiplied by the unit price of rent or (2) the entire stake balance, whichever is smaller.

Stakes must be transferable between overlay addresses to facilitate neighbourhood hopping in case the distribution of stake per neighbourhood is unbalanced.

Withdrawability of surplus stake balance

The committed stake allows operators to indicate their profit margin together with their time preference for realising that profit. However, since the profit is only transparent once the relative stakes within the neighbourhood are known, it may take a while for nodes to discover their optimal stake.

If the BZZ token price increases and the unit price of rent drops, the entry for the node in the stake registry will show an excess balance. This surplus can be withdrawn at any time, allowing stakers to realise their profit from BZZ appreciation.¹⁹

3.4.4 Neighbourhood consensus over the reserve

Peers applying for the round must reach a consensus on which chunks belong to their respective reserves. At a minimum, applicants must agree on their area of responsibility, which can be derived from their storage depth and their overlay address. The consensus over the reserve content is verified through the identity of a reserve sample. The sample consists of the first $k$ chunks in the reserve using an ordering based on a modified hash of the chunks. The modified chunk hash is obtained using the chunk contents and a salt specific to the round.²⁰ It is impossible for any node to construct this set unless they store all (or a substantial number of the) valid chunks, together with their data, at or after the time the salt is revealed.

Recency and sampling

The reserve sample must exclude very recent chunks to prevent malicious uploaders from bombarding nodes in the neighbourhood with a non-identical set of chunks that are going to be sampled, thereby disrupting the consensus about the reserve. One way to guard against this attack is to save each chunk together with its time of storage²¹ in the local database. Pairwise synchronisation of chunks between neighbouring nodes, using the pull-sync protocol, respects this ordering by time of storage. We require that live syncing, i.e., the syncing of chunks received after the peer connection has been started, has a latency no longer than a predefined constant duration called maximum syncing latency (or max sync lag for short). Peer connections that exceed this syncing latency are not considered legitimate storer nodes according to the protocol. This restriction ensures that malicious nodes cannot back-date new chunks beyond the maximum sync lag without losing their storer status.

In order to reach consensus, it is important to ensure that all chunks received by any node in the neighbourhood not later than $l$ are distributed to every node of the neighbourhood before the claim phase. If we set the value of $l$ to be 2 times the allowed sync lag, then every chunk landing first with a node has time to arrive at each node to be safely included in a consensual sample.²²

Storage depth and honest neighbourhood size

In order to decide which reveal represents the truth for the current round, one submission out of all reveals is selected randomly, with a probability proportional to the amount of stake the revealer has. More precisely: based on the amount of stake per neighbourhood size, i.e., stake density. The reserve sample hash and the reported storage depth thus revealed are considered the truth for the current round.

Now we can understand why nodes will report actual storage depth correctly. If a node chooses to play with a larger neighbourhood than the neighbours, it will be selected more often than the others. However, as the committed storage depth decreases as compared to peers, the node’s stake is counted with an exponentially deflated value relative to the peers reporting a deeper storage radius, making such an attack costly.

Overreporting storage depth is possible as long as the the node falls into this narrower proximity of the neighbourhood selection anchor. Therefore, a systematic exploit requires the malicious actor to control a staked node in each sub-neighbourhood of the true honest neighbourhood. Additionally, the winners need to provide evidence that the set of chunks within their storage depth indeed fills their reserve. The actual integer values of the transformed chunk addresses in the sample contain information regarding the size of the original sampled set. By requiring the size of the sampled set to fall within the expected range (with sufficient certainty), a constraint is imposed on the upper bound of the values of the sample. This construct is called proof of density.

Note that the sample-based density proof can be spoofed if the attacker mines content filtering chunks in such a way that the transformed chunk addresses form a dense enough sample, and then uses its own postage batches to stamp them. To enhance security against such attacks, additional measures can be implemented. One approach is to require a commitment to the entire set of postage stamps and prove custody of a sufficient quantity through a randomised sample. By imposing this requirement, fraudulent claimants not only need to generate the content, but also must have enough storage slots to fake the sample. This would require the attacker to purchase postage batches in the magnitude of the entire network or keep track of and store the actual postage stamps existing in the network. The former imposes a prohibitive cost on the attacker, whereas in the latter case, the malicious claimant must bear the risk of depending on honest neighbours for the post-hoc retrieval of the witness chunk data needed for the proof of entitlement.

Skipped rounds and rollover

If no claim is made in a particular round, the funds in the pot simply carry over and increase the outpayment for the next round of the redistribution. This policy is by far the easiest to implement, resulting in the lowest gas expenditures.²³

The eight rules of entitlement

Here we summarise the eight rules of validating a claim, which involve committing and revealing a reserve commitment, as well as submitting evidence as proof of entitlement; see also Table 3.1):

replication

Since liars get frozen, which means they are excluded from the game for a certain period due to revealing a reserve commitment hash or storage depth different from the winner, nodes within a neighbourhood are incentivised to replicate their reserve by synchronising the chunks they store using the pull-sync protocol.

redundancy

The stake serves as weights in determining the probability of a node within a neighbourhood being selected as winner. As a result, there is no advantage in submitting multiple claims. Operators running multiple nodes in one neighbourhood (sharing storage) do not gain any advantage compared to running a single node with the same total stake. Assuming that this disincentive to proliferate is effective, staking can be considered a guarantee for true redundancy.

responsibility

At the time of revealing, it is verified whether the neighbourhood selection anchor falls within the node’s radius of responsibility, i.e., if it belongs to the range of addresses covered by the node, where the proximity of those addresses to the node’s overlay address is not less than their reported storage depth.

relevance

Using a witness proof with the reserve commitment hash as the root, we provide evidence that an arbitrarily chosen segment in the reserve sample packed address chunk corresponds to the address of a witness chunk. A valid postage stamp signed off on this witness chunk address is presented, indicating that storing this chunk in the reserve is relevant to someone (and has been paid for).

retention

A segment inclusion proof is provided as evidence that the chunk data has been preserved with complete integrity.

recency

The salt used for the transformed reserve sample is derived from the random noncs of the current round, proving that the RS must have been compiled recently. The witness and segment indexes are derived from the next game’s random seed, ensuring that no compressed or partial retention of chunk data would have been sufficient at the time of compilation and commitment.

retrievability

The chunk’s retrievability is demonstrated through proximity-based routing, indicating that its address belongs to the range of addresses covered by the neighbourhood. The chunk’s proximity order to the node’s overlay address is not less than its reported storage depth.

resources

Resource retention verifies the volume of resources constituting the reserve by estimating the sampled set size using the density of chunks and postage stamps.

3.4.5 Pricing and network dynamics

In this section, we first contextualise the redistribution scheme within the framework of self-sustainability and offer a simple solution for price discovery.

For Swarm to be a truly self-sustaining system, the unit price of storage rent needs to be set in a way that is responsive to supply and demand. Ideally, the price is automatically adjusted based on reliable signals, resulting in dynamic self-regulation. The guiding insight here is that the information provided by storer nodes during the redistribution game also serves as a price signal. In other words, the redistribution game serves as a decentralised price oracle.

Splitting and merging of neighbourhoods

The storage depth represents the proximity radius within which the neighbourhood’s storer nodes keep all chunks with valid postage stamps and fill their reserve.

If the volume of newly issued storage slots from recently purchased batches (ingress rate) and the volume of expired storage slots (outgress rate) balance out, the storage depth remains constant. However, let’s consider what happens when the volume of reserved chunks increases. As the client’s reserve capacity is fixed, over time, nodes are able to fill their capacity with chunks that are closer to them by at most one proximity order compared to the previously farthest chunks. This results in an increase in their storage depth. Specifically, when the volume of reserved chunks doubles, the storage depth increases by one.

In order to store this excess data under the same redundancy constraints, the network needs double the number of nodes. If all else is equal, double the network-wide reserve, double the postage revenue and therefore double the overall pot that gets redistributed. When neighbourhoods split as they are absorbing the new volume, they simultaneously release the chunks in the PO bin of their old depth, i.e., the chunks now stored by their sister nodes.

Utilisation rate is an organic way to introduce pressure against fully maximising a node’s reserve with critical content and allows for early detection of capacity pressure. This provides a sufficient safety buffer for the triggered incentives to take effect. For instance, if utilisation rate is $1/8$, the storage depth is up to 3 POs shallower than the reserve depth.²⁴ Now the ingress can be really high and bring the reserve depth down to the storage depth. When a narrowing gap between the potential (reserved) and actual (observed) utilisation of the DISC is detected, any changes in incentives will have the buffer to take effect without the target redundancy being threatened.

Number of honest nodes as price signal

Since the storage capacity is maxed out, the ratio of supply and demand is directly seen in the number of honest nodes playing the Schelling game.

We make the assumption that nodes staying in the network for a longer period indicate their profitability. For a stable swarm, neighbourhoods only need a minimum of 4 (balanced) nodes. Assuming equal stake (or more precisely, assuming that relative stake equalises the profitability of node operators) if there are $n$ nodes in a neighbourhood, their long-term profit is equally shared, this amount is optimised if there are exactly four nodes ($n=4$). However, there can be more nodes in a neighbourhood, as opportunistic operators may anticipate a neighbourhood split due to capacity demand and start their nodes accordingly. As these nodes stay in, the same long-term winnings of the neighbourhood get distributed among more nodes than optimal. However, the fact that nodes tolerate this implies that the reward is too much (the price is too high), and the network can tolerate a decreasing price.

On the other hand, if the number of honest revealers is lower than the neighbourhood redundancy requirement, it signals a capacity shortage and therefore requires an increase in storage rent.

Parameterisation of the price oracle

The rule for updating the price from one round to the next is that the current price is multiplied by a value $m$ which depends on the number of honest revealers in the round. Mathematically, $p_{t+1}=m{p}_t$, where $p_t$ is the price in round $t$ (and $p_{t+1}$ is then the price in the following round). We define the multiplier $m$ in terms of the number of honest revealers $r$ and a stability parameter $\sigma$ that governs how quickly the price should increase or decrease, all other things equal.

In particular, we choose $m=2^{\sigma (4-r)}$, and therefore we have $p_{t+1}=2^{\sigma (4-r)}{p}_t$. This expresses how the deviation $4-r$ of the number of revealers from the optimal value of 4 maps to an exponential change in price. The stability parameter $\sigma$ determines the generic smoothness of price changes across rounds, indicating how many rounds it takes for the rent price to double in case of a consistent signal of the lowest degree of undersupply (or halve in case of a consistent oversupply). Figure 3.19 illustrates how the price model works.

Two minor adjustments are applied to this simple model. First, $r$ is capped at a maximum value $r_{\text{max}}$ (chosen to be 8 in our case). That is, $r$ should actually be interpreted as the minimum of the number of honest revealers and $r_{\text{max}}$. Second, the price is never allowed to drop below a predetermined minimum $p_{\text{min}}$. That is, if the price were to drop from one round to the next and reach a value below the predetermined minimum $p_{\text{min}}$, the price will be held steady at $p_{\text{min}}$ instead.

3.5 Insurance: negative incentives

The storage incentives presented so far refer to the ability of a system to encourage the preservation of content through monetary rewards given to storers. This was achieved using an on-chain game which instrumented the fair redistribution of postage payments to storers, thereby providing positive incentivisation at a collective level. Such a system, however, is suspect to the tragedy of the commons problem in that disappearing content (any particular chunk missing) will have no negative consequence to storers (any one storer node punished). The lack of individual accountability renders the storage incentivisation limited as a security measure against data loss. Introducing competitive insurance, on the other hand, adds an additional layer of negative incentives, compelling storers to be very precise in their commitments to ensure reliability for users. Particular attention is required in the design of the incentive system to make sure that failure to store every last bit promised is not only unprofitable but outright catastrophic to the insurer.

3.5.1 Punitive measures

Unlike in the case of bandwidth incentives where retrievals are immediately accounted and settled, long-term storage guarantees are promissory in nature and it can only be decided if the promise has been kept at the end of its validity. Merely losing reputation is not sufficient as a deterrent against foul play in these instances: since new nodes must be allowed to provide services right away, cheaters could just resort to new identities and keep selling (empty) storage promises.

We need the threat of punitive measures to ensure compliance with storage promises. These can be achieved through a deposit system. Nodes wanting to sell promissory storage receipts should have a stake verified and locked-in at the time of making their promise. This implies that nodes need to register in advance, agreeing to a contract and placing a security deposit. Once registered, a node may sell storage promises covering the time period for which their funds are locked. While their registration is active, if they are found to have lost a chunk that was covered by their promise, they stand to lose their deposit.

Let us start from some reasonable guiding principles:

• Owners need to express their risk preference when submitting to storage.
• Storers need to express their risk preference when committing to storage.
• There needs to be a reasonable market mechanism to match demand and supply.
• There needs to be guarantees for the owner that its content is securely stored.
• There needs to be a litigation system where storers can be charged for not keeping their promise.

Owners’ risk preferences consist of the time period covered as well as a preference for the degrees of reliability. These preferences should be specified on a per-chunk basis and they should be completely flexible at the protocol level.

Satisfying storers’ risk preferences means that they have ways to express their certainty of preserving what they store and factor that in their pricing. Some nodes may not wish to provide long-term storage guarantees, while others may be unable to commit large deposits. This differentiates nodes in their competition for service provision.

A market mechanism means there is flexible price negotiation or discovery or automatic feedback loops that tend to respond to changes in supply and demand.

In what follows we will elaborate on a class of incentive schemes we call "swap, swear, and swindle" due to the basic components:

swap

Nodes maintain a quasi-permanent, long-term contact with their registered peers. Along these connections, peers exchange chunks and receipts, triggering swap accounting (see 3.2.1).

swear

Nodes registered on the Swarm network are held accountable and stand to lose their deposit if they are found to violate the rules of the swarm in an on-chain litigation process.

swindle

Nodes monitor other nodes to check if they comply with their promise by submitting challenges according to a process of litigation.

3.5.2 Contracts through receipts

A litigation procedure necessitates that there are contractual agreements between parties, ultimately linking an owner, who pays for securing future availability of content, and a storer, who gets rewarded for preserving it and making it immediately accessible at any point in the future. The incentive structure needs to make sure that litigation is used only as a last resort.

The most straightforward approach to manage storage deals is through direct contracts between the owner and the storer. This can be implemented by having storers return signed receipts of chunks they accept to store. Owners then pay for the receipts either directly or via escrow. In the latter case, the storer is only awarded the locked funds if they are able to provide proof of storage. This procedure is analogous to the postage stamp lottery process. Insurance can be bought in the form of specially marked postage stamps. Statements of custody receipts can close the loop and represent a contract between the uploader and the storer. Outpayments conditional on proofs of custody can be implemented the same way as the lottery.

Even if the consumer attempting to access the chunk was not a party to the agreement to store and provide the requested content, failure to deliver the stored content is subject to penalties. Litigation is therefore expected to be available to third parties who wish to retrieve content.

If the pairing of chunks and receipts is public and accessible, then consumers/downloaders (not only creators/uploaders) of the content are able to litigate in case a chunk is found to be missing.

Registration

Before a node can sell promises of long-term storage, it must first register via a contract on the blockchain we call the SWEAR (Secure Ways of Ensuring ARchival or Swarm Enforcement And Registration) contract. The SWEAR contract allows nodes to register their public key to become accountable participants in the swarm. Registration involves sending the deposit to the SWEAR contract, which serves as collateral in case the terms that registered nodes "swear" to keep are violated (i.e. nodes do not keep their promise to store). The registration is valid only for a set period, at the end of which a Swarm node is eligible to reclaim their deposit. Users of Swarm should be able to count on the loss of deposit as a disincentive against foul play for as long as enrolled status is granted. Because of this the deposit must not be refunded before the registration expires. The expiry of the insurance period should therefore include a final phase during which the node is not allowed to issue new receipts but can still be challenged.

When a registered insurer node receives a request to store a chunk that is closest to them, it can acknowledge the request with a signed receipt. It is these signed receipts that are used to enforce penalties for loss of content. Because of the locked collateral backing them, the receipts can be viewed as secured promises for storing and serving a particular chunk.

3.5.3 Submitting a challenge

If a node fails to observe the rules of the Swarm they swear to keep, enforcement of punitive measures must be preceded by a litigation procedure. The implementation of this process is called SWINDLE (Secured With INsurance Deposit Litigation and Escrow).

When a user attempts to retrieve insured content and fails to find a chunk, they can report the loss by submitting a challenge. This scenario is the typical context for starting litigation. This is analogous to a court case in which the issuers of the receipts are the defendants who are guilty until proven innocent. Similarly to a court procedure, public litigation on the blockchain should be a last resort when the rules have been abused despite the deterrents and positive incentives.

The challenge takes the form of a transaction sent to the SWINDLE contract, in which the challenger presents the receipt(s) for the lost chunk. Any node is allowed to send a challenge for a chunk as long as they have a valid receipt for it (although it may not have necessarily been issued to them). The same transaction also sends a deposit covering the price of the upload of a chunk. The validity of the challenge as well as its refutation need to be easily verifiable by the contract. The contract verifies if the receipt is valid, i.e. 1) authentic, 2) active and 3) funded, by checking the following conditions:

authentic

The receipt was signed with the public key of a registered node.

active

The expiry date of the receipt has not passed.

funded

Sufficient funds are sent alongside it to compensate the peer for uploading the chunk in case of a refuted challenge.

The last point above is designed to disincentivise frivolous litigation, i.e. bombarding the blockchain with bogus challenges and potentially causing a DoS attack.

The contract comes with an accessor for checking that a given node is challenged (potentially liable for penalty), so the challenged nodes can be notified that they must present the chunk in a timely fashion. The challenge remains open for a fixed amount of time, the end of which essentially sets the deadline for refuting the challenge.

Upon verifying the format of the refutation, the contract checks its validity by checking the hash of the chunk payload against the hash that is litigated or validating the proof of custody.

If a challenge is refuted within the period the challenge is open, the deposit of the accused node remains untouched. The cost of uploading the chunk must be reimbursed to the uploader from the challenger’s deposit. To prevent DoS attacks, this deposit should actually be substantially higher than the upload cost in any case (e.g. a small integer multiple of the corresponding gas price). After successful refutation the challenge is cleared from the blockchain state.

This challenge scheme serves as the simplest method (1) for defendants to refute a challenge and (2) make the disputed data available to nodes that require it.

3.5.4 Successful challenge and enforcement

If the deadline passes without successful refutation of the challenge, then the charge is regarded as proven and the case enters into the enforcement stage. Nodes that are proven guilty of losing a chunk lose their deposit. Enforcement is guaranteed to be successful by the fact that deposits are kept locked up in the SWEAR contract.

If, on litigation, it turns out that a chunk (that was covered by a receipt) was lost, the deposit must be at least partly burned. Note that this is necessary because, if penalties were paid out as compensation to holders of receipts of lost chunks, it would provide an avenue of early exit for a registered node by "losing" bogus chunks that had been deposited by colluding users. Since users of Swarm are interested in their information being reliably stored, their primary incentive for keeping the receipts is to keep the swarm motivated to do so, not the potential for compensation in the case they do not. If deposits are substantial, we can get away with paying out compensation for initiating litigation, however we must have the majority (say 95%) of the deposit burned in order to make sure this easy exit route remains closed.

Punishment can entail suspension, meaning a node found guilty is no longer considered a registered Swarm node. Such a node is only able to resume selling storage receipts once they create a new identity and put up a deposit once again.²⁵

Incentivising promissory services

Delayed payments without locked funds leave storers vulnerable to non-payment. Advance payments (i.e. payments settled at the time of contracting, not after the storage period ends) on the other hand, leave the buyers vulnerable to cheating. Without limiting the total value of receipts a node can sell, a malicious node can collect more than their deposit and disappear. Having forfeited their deposit, they still walk away with a profit even though they broke their promise. Given a network size and a relatively steady demand for insured storage, the deposit could be set sufficiently high so this attack is no longer economical.

Locking the entire amount eliminates the storer’s distrust due to potential insolvency of the insured party. When paying for insurance, the funds should cover the total price of storing a chunk for the entire storage period. This amount is locked and is released in installments contingent on the condition that the node provides a proof of custody. On the other hand, since payment is delayed it is no longer possible to collect funds before the work is complete, which eliminates a collect-and-run attack entirely.

3.6 Summary

In the first two chapters of the architecture part of the book, we introduced the core of Swarm: the peer-to peer-network layer described in Chapter 2 implements a distributed immutable storage for chunks, which is complemented by the incentive system described in the subsequent chapter. The resulting base layer system provides:

• permissionless participation and access,
• zero-cash entry for node operators,
• maximum resource utilisation,
• load-balanced distribution of data,
• scalability,
• censorship-resistance and privacy for storage and retrieval,
• auto-scaling popular content,
• basic plausible deniability and confidentiality,
• churn-resistance and eventual consistency in a dynamic network with node dropouts,
• sustainability without intervention due to built-in economic incentives,
• robust private peer-to-peer accounting,
• incentivised bandwidth sharing,
• off-chain micro-commitments with on-chain settlement,
• DoS resistance and spam protection,
• positive (i.e., motivated by reward) incentives for storage,
• negative (i.e., discouraged through threat of punitive measures) incentives against data loss.