2021 process updates and improvements

[rviscomi]

Opening a thread to discuss ways we can improve the content creation process in 2021. All 2019 and 2020 contributors are encouraged to share their feedback. Let's identify any areas of deficiency so that we can improve contributor satisfaction and productivity.

[tunetheweb]

Yes would really like to hear suggestions on ways to improve!

However, at the same time I’d like to hear what really worked this year as well — to make sure we keep doing that next year! That will also be good to keep a bit of positivity too so we don’t just list the things we can do better on — which can lead to forgetting the good parts.

[rviscomi]

The top issues I'm sensing from contributors are:

1. Not enough time to complete milestones.
2. Not enough contributors to share the workload.
3. Not enough guidance from project leads

1. Not enough time to complete milestones

The obvious answer is to start earlier in the year. We started in June this year, so maybe we should start the content team assignments in April or May to leave more time for planning, analysis, and writing. My hope is that the total number of hours required of contributors doesn't go up, but that there's simply more time between milestones for contributors to work at their own pace.

The milestone that was the most delayed in 2019 and 2020 was the analysis. We need to budget even more time for analysts to query the data. My (incorrect) assumption was that in 2020 we can simply rerun 2019 queries and maybe a few new ones, so the process should be quick, but I was happy to be wrong because that means we're finding new and interesting ways to analyze the data and keep up with the modern web. Unfortunately, that did cost us in terms of having to rush through getting custom metrics implemented shortly before the August crawl and getting metrics analyzed after the crawl was completed.

This year we had a free-for-all "call for authors" issue where people were nominated to participate. Instead, to streamline this process, I would like to see us start by creating the chapter-specific issues first, and having interested contributors reply directly. This would enable us to jumpstart specific chapters that are ready to go without having to wait for all nominations to come in. We can open up the 2021 planning issues any time after the 2020 launch to start gathering interest.

2. Not enough contributors to share the workload

This will always be a problem with a volunteer project. I think for long-term viability, we need to be open to the idea of compensating contributors financially to encourage participation and also—importantly—diversity, by ensuring that people can afford to spend their time on this project away from other important parts of their life. I've been looking into how we can get HTTP Archive on Open Collective so that we can accept contributions from individuals and businesses in the web community for the purpose of supporting the Web Almanac. For example, we could also enable sponsorships through the GitHub platform. We need to carefully think through how compensation would work fairly in practice.

I would also love to establish a more formal relationship with web community expert groups like GDEs. GDEs are proficient in may different aspects of the web, much like our breadth of content, and experienced writers who do community outreach. I think this makes the GDE group a great pool of candidates for authors, and can help sustain the Web Almanac contributions year after year. Please let me know if there are other groups like this that we can partner with.

Having enough contributors will also improve our ability to staff chapters and stay on track for each milestone. We especially need to have a proportional number of analysts to the number of metrics needed by a chapter, given how long analysis takes. Some chapters may only have a dozen metrics, but others may have over 100. There should be enough analysts to get this done comfortably. If not, we need to lower authors' expectations about analyzing more metrics than can be queried.

Specifically, we also need to increase the pool of analysts. HTTP Archive itself needs to be better documented to lower the barrier to entry for new analysts. We can also do more training for people who want to learn SQL and the HTTP Archive dataset. Analysts are critical members of the content team and I'm glad that we took action this year to give them credit in the byline of the chapter. That may also help drive more interest in the role to some small degree.

3. Not enough guidance from the project leads

I'm speaking specifically about my own feelings of not having personally done enough and not putting any blame on any of the other project leads. It's just becoming unscalable for me to directly manage a growing number of chapters each year.

I am really happy with how content teams worked out this year and assigning one author to be the content team lead. Having a single point of contact for each chapter has proven to be an effective way to drive the content creation process across the writing, reviewing, and analysis roles. It empowers contributors to drive the process themselves and make decisions about the content, without getting micromanaged by the project leads.

I'd like to take this idea a step further in 2021 by assigning section leads to be responsible for all chapters under each of the four main parts: content, experience, publishing, and delivery. These leads would regularly check in with chapter leads to make sure everyone is on track and collaborating effectively. Please reach out to me if you're interested in being a section lead in 2021. I think previous authors would make excellent leads, having gone through the experience before.

By coordinating in groups of chapters rather than all 20+ individually, I think it'd be more manageable to do periodic face-to-face syncs via video chat and ensure that everyone is on track. This is something I wish I was able to do in 2020 at the major content planning, analysis, and writing milestones. This will help avoid surprises later in the process too, rather than me assuming everyone is reading all the docs. The section leads can meet with their 4-5 content leads periodically, a size more practical for getting everyone from various timezones in a meeting together. Then content leads can meet with their teams to keep the progress moving.

---

I would love to hear everyone's thoughts on these ideas and anything else you think could be improved.

I'm getting the strong sense that everyone is here because they agree with our mission and they'd like to see the project continue to run year after year. This is a good opportunity for us to fine tune the machine so that it can run smoothly and autonomously.

[rviscomi]

As mentioned in #2147 I'd like to more tightly integrate editors with content teams this year. Editors will review the chapter drafts as they're written and provide incremental feedback, rather than massive rewrites and surprises at the end.

[rviscomi]

Closing this as the 2021 project has just started, but if anyone has feedback please feel free to reach out