Storage market bidding (originally by Menduist)

[dryajov]

(@Menduist I took the liberty to move your issue here from codex-storage/nim-codex#14)

I've been thinking about the storage market, and I think it may not be a good idea to lock up funds to be able to bid.

We aim to have a global market, and who says global market says a lot of competition.
It means that on a given contract, we should have tens, if not hundreds of bids, and only a fraction of them will be accepted.

For instance, let's say on average, 10% of bids are accepted (it seems a lot, but anyway)
It means that as a storage node, I have, on average, to place 10 bids to win one contract.

If I have to lock up funds for each of theses bids, it means that not only I'll have a lot of funds locked up, but also I'll have to pay a lot of fees (to lock up theses funds), without being sure of having any contracts in the end.

But this property also allows us to reduce the probability of "double bidding" (like double spending):
Since I have "low" (10%) probability of actually getting a contract, the probability that I get two contracts is even lower (1%), especially if we have a fast marketplace with bids with little TTL.
(Obviously, in reality, the probability of getting a bid are not random, but depends on your price, but simplifying here)

My idea would be:

• I want a contract with 5 storage nodes
• I create and send that contract
• Get 50 bids
• Choose the 10 best
• Lock up my funds, send the contract + 10 bids to the blockchain
• The 5 first to lock up their fund win the contract

In reality we could probably have less than 50% of "backup bids".
And if I'm very unlucky and end up with less than 5 storage node?
I can recreate a contract, which is not ideal, but should not happen in regular operation

[dryajov]

Good points, thanks for the comments.

I think it may not be a good idea to lock up funds to be able to bid.

I'm not adamant about this and I understand the downsides, but it is easier to reason about it like that and it eliminates a whole slew of issues. For example it would eliminate, "shallow bids" (bidders that weren't serious), "over bids" (bidding without having enough funds to fulfill it), etc...

The problem with open protocols is that there is no external entity that can intervene and control this sort of things, all the security and rules have to be intrinsic to the protocol and having bidders lock their funds up during the bidding process seems like a simple and effective way of dealing with all sorts of issues without introducing any machinery to deal with them individually (e.g. we'd have to implement something to deal specifically with "shallow bids" and something to deal with "over bids", etc...). This was my original reasoning for suggesting it.

One way of dealing with this is arbitrage, but it introduces and even greater overhead and complexity.

If I have to lock up funds for each of theses bids, it means that not only I'll have a lot of funds locked up,

I imagine that the bidding process will be relatively quick, usually seconds, minutes and in extreme cases hours, so I'm not too concerned with locked up funds.

but also I'll have to pay a lot of fees (to lock up theses funds), without being sure of having any contracts in the end.

This is definitely a concern, but it heavily depends on the cost of the transaction which varies from platform to platform. In all honesty, I don't think we can run this directly on top of an L1 due to the current limitations in throughput. For this reason, it makes sense to think about this as either an L2 or running on top of an L2, in which case the transaction cost should be (hopefully) negligible. In general I think transaction cost is something that will have to be addressed in order to make decentralized systems usable in the long run and we aren't the only ones dealing with this, it is an inherent limitation of the current systems and their capacity.

My idea would be:

I want a contract with 5 storage nodes
I create and send that contract
Get 50 bids
Choose the 10 best
Lock up my funds, send the contract + 10 bids to the blockchain
The 5 first to lock up their fund win the contract

What happens with nodes that:

• Shallow bid, they weren't committed enough to ultimately execute on the contract - due to bidding on multiple contracts at the same time for example?
• Over bid, i.e. no enough funds to stake?

How do we prevent that from happening in the future to avoid many failed contracts/attempts to secure storage? Reputation systems -they have even nastier complications.

I'm sure there are more not so obvious issues that will pop up, but at least, from a very cursory view, it seems like they can be addressed by locking up funds.

The way I see this working is:

• Both the user and the bidders lockup funds
  • The client locks up when posting the contract
  • The bidders lockup when they bid (only one bid per contract)
  • We have a deterministic criteria to choose a subset of the nodes from all the bidders
• At the end of the bidding period the best nodes are chosen the rest are rejected and their funds unlocked
• If the client wants to withdraw before the contract ends, it can but it will have to cover the cost of the bidders (+some penalty?)
• If a bidder wants to withdraw, it will have to eat up the cost of the transaction (+some penalty?)

Note that the above model only considers one bid per contract, not multiple bids. The idea behind the contract is to signal storage nodes that there is a client with some specific storage requirements and to allow storage nodes to signal their capability and willingness to fulfil those requirements.

Maybe "bid" is the wrong word here?

[Menduist]

Shallow bid, over bid

I agree that is an issue. For over bid we can check their wallet on the blockchain (I assume that reading the bc is cheap), but shallow bid remain problematic. On the other hand, the only reason to do a shallow bid would be to disrupt the network.

And if you want to disrupt the network, the locking of funds can have a much more profound impact:
If we say that on average, the contracts have a 50x stake factor, and that on average, 20 nodes bids to a contract.
That means that if I create a contract worth 10€, a total of 10.000€ would be locked!
Even if that's locked just for a minute, that's still a super cheap way to lock up a lot a funds maliciously, and because of this multiplication factor, you could almost freeze the network by locking everyone's funds.

The fact that the host has to lock a lot more funds that the client (because of the stake) to participate to a contract indicate to me that they should have the "upper hand" when it comes to creating theses contracts.

As long as you have a majority of honest hosts, shallow bid should not be an issue. And because to bid on a 10€ contract you need to have 500€ in your wallet (with 50x stake factor), it's basically a mini PoS

The client locks up when posting the contract

How much funds? You have no idea how much you'll end up paying. And if you publicly show how much you want to pay, that can have some bad impacts.

[dryajov]

A lot of this has been covered in offline discussions, but I'll try to capture the essence here - the discussion revolved mostly around this same point, so it makes sense to address them individually here as well.

To avoid confusion lets adopt some common terminology, we can change this later but this seems to be as good as any for this discussion:

• Bidding refers to making an offer by a storage provider made to a request for storage (i.e. contract) made by a client
• Contract refers to a request for storage made by a client which storage providers respond to with "bids"

On the other hand, the only reason to do a shallow bid would be to disrupt the network.

Not necessarily. It might be an optimization strategy, if there is nothing at stake what prevents the bidder from maximizing it's chances of entering the best possible contract by bidding on multiple contracts at the same time and only sticking with the best ones?

Even if that's locked just for a minute, that's still a super cheap way to lock up a lot a funds maliciously

True, there is asymmetry in cost when it comes to asking for storage. It should be costlier for a storage provider to bid than for a client to request storage because the storer needs to signal the contract creator that it is willing to take on the risk and has the required resources to store the client's data.

and because of this multiplication factor, you could almost freeze the network by locking everyone's funds.

This is a good point but, there might be pretty straightforward solutions to this. Let's explore a couple here - I'm not saying that they are definitive, but it should help give some perspective on how this can be addressed.

• First, when I say that the client requests storage, I mean that the client locks up the entirety of the funds of the contract up front. The total upfront cost will increase with the size and duration of the contract - the more data and the longer the contract duration the more expensive it becomes.
• Second, the argument around locking up funds revolves around timing and we can address that by adding a fee on top of the stake proportional to the amount of redundancy+time the contract is accepting bids. The larger the amount of requested nodes and the timeframe for which the contract is accepting bids, the higher the fee. We might even make the fee exponential to discourage this sort of attacks.

In general, I think there are enough tools in the incentives toolbox to address this concerns without having to resort to external countermeasures.

How much funds? You have no idea how much you'll end up paying. And if you publicly show how much you want to pay, that can have some bad impacts.

Not sure what negative impacts will this have? It's standard practice to disclose the ask and bid in a two-way-price system. In fact, I tend to think (maybe this is standard, but I'm no expert in the field, so I don't know) of bidding as having two (more?) functions at once; a price discovery mechanism and a signaling mechanism.