How "big" should a network upgrade be?

[arajasek]

Background

As the Filecoin ecosystem continues to grow, the number of FIPs being suggested is increasing, as is the average complexity* of these proposals. This is great, and something we want to be able to support. Various bottlenecks exist in the goal of supporting more overall development -- perhaps the most obvious is the engineering time needed to implement and test FIPs.

Size of a Network Upgrade

One other bottleneck is the "size" of a network upgrade itself. This is a very nebulous and multi-dimensional concept, but, very simply put, we don't want to change "too much" in a single upgrade, because the stability and security of the Filecoin network is of paramount importance. A single network upgrade that, for instance, simultaneously introduced a new proof system, a new VM, and reshaped the entire state tree, would be a very alarming prospect!

To date, we have informally agreed upon what makes an upgrade "too big" and planned accordingly. For instance, I believe we plan on including no other FIPs in the network upgrade that first introduces the FVM to Filecoin (a plan I support). This is somewhat unscalable as more people start proposing complex FIPs, and might cause dissatisfaction with the decision-making process. Some proponents may disagree that a proposal they are championing is "too complex to include" alongside another major change, while others may agree about the complexity, but want their changes to go live first!

Proposed solution

While this post is primarily intended to spur discussion around how we should we be making this decision, I'd like to also propose a solution. We could assign weights to any every along certain dimensions -- for example the FIP that introduces user-defined contracts in Filecoin would be close to a 10 in technical risk (it opens up entire galaxies of possible attacks that don't exits today), but probably a 0 in terms of political contention (does anyone think it's "unfair"?) We then agree on what the maximum risk we can tolerate (per dimension) in a network upgrade. Scoping a network upgrade is then simply the knapsack problem

Feedback welcome!

*complexity is a nuanced term here: some FIPs are technically complex, others open security risks, while yet others are politically contentious

[anorth]

Thanks for opening this discussion.

The other side of this is the frequency of upgrades. For a given rate of change, we can either do fewer large upgrades, or more frequent but smaller upgrades. Different risks are associated with different points on that spectrum.

There are many parties for whom a network upgrade is perceived primarily as a (short-term) cost, and it's hard for them to get excited about the long-term benefits of an increased rate of change. E.g. exchanges don't really care about Filecoin functionality - they mainly profit from volatility. One might hope that storage providers are a bit more enthusiastic about improvements, but many who are operating on short-term incentives may not be.

I posit that we, the implementers, along with long-term aligned providers, clients, and adjacent parties, should aim to maximise the safe rate of change. We want to improve faster, iterate more rapidly, build towards our visions of Filecoin's potential as fast as possible. I take this as a given for discussions like this.

I like the idea behind attempting to quantify risk dimensions. I probably wouldn't up-front "agree" on a maximal value, which would unnecessarily limit our own future agency, and promote gaming and otherwise manipulating the ratings so as to decrease their informative value. And the interactions between changes are an important part of the risk of deploying multiple together – it's not just a simple sum.

I would be very happy if we could move toward more frequent upgrades, and reduce or constrain risk in that way while promoting an increased rate of change. What could we do to improve that? What challenges are there? How could we mitigate them?

[jennijuju]

The community, especially a large amount storage providers, had suggested they would prefer less regular upgrades with some sort of pre-schedule. I’m one of the SPWG call it was suggested bi-annual, or at most quarterly upgrades were preferred. Main reason being that every upgrade means a maintenance window or a down time of their storage service.

[anorth]

I realise that's what they (think they) want. Their desire is based on a short-term incentive to avoid cost/effort. The desire for a higher rate of change is a long term one about building utility, value, and adoption. This is really what invested parties, including large SPs, should want, but it's a benefit that comes in the future, while the cost comes right now, so feels more real. It's a trade-off, we need to find a good balance. But twice a year is 🤯 ridiculous, we're dead if we do that. Quarterly is too slow.

[steven004]

Quarterly is acceptable to me; one upgrading every two months could be also an option I would support.
I would prefer limited changes in each upgrading, and make it quicker upgrading. But there should also be a balance since we need to go through all the process of implementation/testing/calibration etc.