Skip to content

Sprint Planning Meeting 2020 04 02

Jump to bottom
Erik Moeller edited this page Apr 3, 2020 · 1 revision

Sprint Planning Meeting, SecureDrop, April 2, 2020

Sprint timeframe: Beginning of Day (PDT) 2020-04-02 to Beginning of Day (PDT) 2020-04-15

0) Retrospective

Team comments and observations:

Pilot release

What worked well:

  • (Erik) Good coordination between PDT/EDT/IST time zones to maximize coverage

  • (Erik) Speedy releases of components + speedy bugfixes (ro)!

    • (Conor) +1 on speedy bugfixes, dang

    • (mickael) we were very quick and confident in releasing fixes due to good test coverage and simple build/deploy process

  • (Conor) The remote-only provisioning plan is remarkably solid! Improvising from when we thought we'd be on-site, we've put together quite a fallback plan.

  • (Jen) The QA process did catch a ton of issues that would have otherwise hit end users. Big big thanks for all the QA folks!

  • (Allie) Functional tests are in place, so that's good

What can be improved:

  • (Erik) Deep investigation of issues at realistic scale was too late, creating need for last minute fixes. Need to ensure we continue to test all changes from here on in production-like configs. +1
  • (Jen) Kind of related to this I think we could have (this is my bad) declared an official freeze to non-bugfixes. We ended up making UI changes/improvements that weren't strictly bugfixes, e.g. the export UI changes pretty late in the game.
  • (Erik) Number of components can make it difficult to have shared understanding of what constitutes a "release". Project version that covers RPMs & DEBs could be useful for major release milestones.
  • (Allie) We need more communication between team members about how we do development and test different datasets/ set up staging etc., e.g. there is a QA loader script, a create-dev-data.py script, and a TEST_DATA_FILE environment variable you can pass to make when building staging.

Kev and Kushal please see https://github.com/freedomofpress/securedrop/issues/5186

  • (Allie) Functional tests are still a bit flaky, we need more functional tests too.

  • (Allie) We could improve our test plans for PRs to try to avoid regressions

  • (Jen) There's a fair bit of manual work involved in creating artifacts for production use (tarballs, debian packages) that we could automate further to free up developer time (this process is automated for nightlies which I think we've got a lot of benefit from imho 😇).

  • (Kushal) Integration testing of the whole workstation to catch things early

  • Test suites for SD core & SD client integration are a bit optimistic about db state. We're learning how to seed db in tests with more realistic battle-tested db state.

  • Test datasets and self-contained environments still required - qubes staging env is a resource hog and a bit brittle, one alternative is to add ability to load datasets and add hidden service config in the dev environment.

    • not disagreeing, but I have been pretty happy with the Qubes staging env. it's more manual but so far I think it's easier to work with than the regular env. Nice to be able to just ssh in instead of having to use molecule; it makes it pretty easy to use qa_loader

    • (Conor) testing client-with-realistic-staging-server (container or otherwise) deserves its own knowledge share, I think we're all doing slightly different things locally

    • (Conor) one thing that'd help me is shared artifacts for db state, e.g. backup tarballs for reliable repro+1

      • (Kushal) +1 to this as I am also following random steps to create better dataaset for my dev environment.

      • (Conor) @Kushal previously i've thought of this as an lfs repo where we encrypt tarballs (if containing creds) and self-serve. if things change, we update the repo and everyone gets the changes downstream

        • +1: extracting a tarball and restarting Apache would be a lot faster than create-dev-data or qa_loader for scale testing

      • (mickael) We can also have another physical instance (or prod vms on a server) for sharing, perhaps with a larger amount of sources. If we aren't making server-side changes we should consider reducing variability

      • @Mickael: that's an interesting suggestion: preprovision hardware with n=50, n=500, n=1000...

  • (ro) Support: I'm worried about our non-Qubes-familiar folks re pilot provisioning and troubleshooting. I think for some folks provisioning will take a full day at least, a few hours longer than we currently anticipate.

What's still a puzzle:

  • (Erik) Qubes VM startup/shutdown behavior not fully predictable (see libxenlight issue), even if we resolve now, need to ensure we have clear resolution upstream.

1) Review important dates and time commitments

2020-04-03              : PTO: Jen
2020-04-06              : PTO: Jen, Mickael
2020-04-07 to 2020-04-08: SecureDrop Workstation installs
2020-04-09              : PTO: Mickael
2020-04-10              : Holiday (Canada): Kev, Mickael, Ro
                          PTO: Erik

After sprint period:

TBD                     : SecureDrop Client 0.2.0 Release
2020-04-15              : QA / String freeze for SecureDrop 1.3.0 (tentative)
2020-04-29              : SecureDrop 1.3.0 (tentative)

Time check: https://docs.google.com/spreadsheets/d/1khpVs6-G2LDmZrAYnorB8In9Z9pG9jtKud3xTQjXOkA/edit#gid=0

2) Agree on goals for this sprint

  • Respond to immediate pilot-related needs (this may result in sprint tasks being deferred)
  • Land critical updates for SecureDrop 1.3.0
  • Land scalability fixes to ensure client is consistently well-behaved with large number of sources

3) Task selection and estimation

https://docs.google.com/spreadsheets/d/1vA_9kwrtktdqq--ps1xt9iHJu7Q-U9Y8Q4xy0HfgI9M/edit#gid=0

Clone this wiki locally