2.3.0 Test Plan

2.3.0 QA Checklist

For both upgrades and fresh installs, here is a list of functionality that requires testing. You can use this for copy/pasting into your QA report.

If you have submitted a QA report already for a 2.3.0 release candidate with successful basic server testing and application acceptance testing sections, then you can skip these sections in subsequent reports, unless otherwise indicated by the Release Manager. This is to ensure that you focus your QA effort on the release-specific changes as well as changes since the previous release candidate.

Environment

• Install target:
• Tails version:
• Test Scenario:
• SSH over Tor:
• Release candidate:
• General notes:

Basic Server Testing

• After installing the testinfra dependencies, all tests in ./securedrop-admin verify are passing:
  • Install dependencies on Admin Workstation with cd ~/Persistent/securedrop && ./securedrop-admin setup -t
  • Run tests with ./securedrop-admin verify (this will take a while)
  • Remove test dependencies: rm -rf admin/.venv3/ && ./securedrop-admin setup
• QA Matrix checks pass

Command Line User Generation

• Can successfully add admin user and login

Administration

• I have backed up and successfully restored the app server following the backup documentation
• If doing upgrade testing, make a backup on 2.2.1 and restore this backup on this release candidate
• "Send Test OSSEC Alert" button in the journalist triggers an OSSEC alert and an email is sent
• Can successfully add journalist account with HOTP authentication

Application Acceptance Testing

Source Interface

Landing page base cases

• JS warning bar does not appear when using Security Slider high
• JS warning bar does appear when using Security Slider Low

First submission base cases

• On generate page, refreshing page produces a new 7-word codename
• On submit page, empty submissions produce flashed message
• On submit page, short message submitted successfully
• On submit page, file greater than 500 MB produces "The connection was reset" in Tor Browser quickly before the entire file is uploaded
• On submit page, file less than 500 MB submitted successfully

Returning source base cases

• Nonexistent codename cannot log in
• Empty codename cannot log in
• Legitimate codename can log in
• Returning user can view journalist replies - need to log into journalist interface to test

Journalist Interface

Login base cases

• Can log in with 2FA tokens
• incorrect password cannot log in
• invalid 2fa token cannot log in
• 2fa immediate reuse cannot log in
• Journalist account with HOTP can log in

Index base cases

• Filter by codename works
• Starring and unstarring works
• Click select all selects all submissions
• Selecting all and clicking "Download" works

Individual source page

• You can submit a reply and a flashed message and new row appears
• You cannot submit an empty reply
• Clicking "Delete Source Account" and the source and docs are deleted
• You can click on a document and successfully decrypt using application private key

Basic Tails Testing

After updating to this release candidate and running securedrop-admin tailsconfig

• The Updater GUI appears on boot
• Updating to 2.2.1 is successful

2.3.0 release-specific changes

Web application

• #6306 Add basic message filtering in the SI

  • minimum message length

    • in the JI, navigate to the Instance Config page via the Admin page
      • verify that the Submission Preferences section includes a "Prevent sources from sending initial messages shorter than the minimum required length" option, currently unchecked and with the length field set to 0.
    • check the "Prevent sources from sending initial messages shorter than..." checkbox but do not set a length. Click Update Submission Preferences
      • verify that the checkbox is unchecked and an error message is displayed asking you to set a length.
    • check the "Prevent sources from sending initial messages shorter than..." checkbox and set a length of between 5 and 20 chars. Click Update Submission Preferences
      • verify that the checkbox is checked, the length you chose is set, and a success message is displayed.
    • Create a new source account on the SI, navigating through to the /lookup page
      • verify that a notice is displayed under the message textfield like "If you are only sending a message, it must be at least N characters long" where N is the length you chose above
      • Verify that if you try to submit too short a message, the submission fails with a flashed error like "Your first message must be at least N characters long"
      • Verify that a message N characters long can be submitted successfully
      • after a successful first submission, verify that the notice under the text field is no longer displayed and you can submit message shorter than N chars
      • in the JI, uncheck the "Prevent sources from sending initial messages shorter than..." checkbox,, click Update Submission Prefs and verify that the length is set to zero and a success message is displayed
      • Create a new source account in the SI and verify that no message length limit is mentioned or set

  • codename messages

    • in the JI, navigate to the Instance Config page via the Admin page
      • verify that the Submission Preferences section includes a "Prevent sources from submitting their codename as an initial message." option, currently unchecked.
    • check the "Prevent sources from submitting their codename as an initial message" checkbox. Click Update Submission Preferences
      • verify that the checkbox is checked and a success message is displayed.
    • Create a new source account on the SI, navigating through to the /lookup page
      • Enter the codename as a message and click Submit - verify that the message is not submitted and an error is flashed like "Please do not submit your codename.."
      • Enter anything else as a message and click Submit - verify that the message was submitted correctly
      • Enter your codename again and click Submit - verify that the message was submitted correctly.
    • Create a new source account on the SI, navigating through to the /lookup page
      • attempt to submit its codename as a message, verify that the codename message is rejected
      • log out from the SI, then log in again as the same source, and attempt to submit the codename as a message. Verify message is accepted

• #6290 Improve Tor2Web detection and handling

  • Find a tor2web proxy like onion.ly, onion.dog, etc. (just an extra suffix to add to the real onion address)
  • JavaScript check:
    • Visit the SI over a tor2web proxy using a clearnet browser (FF, Chromium) with JavaScript enabled. Verify you are redirected to the /tor2web-warning endpoint.
    • Verify the warning contains the real onion address (no mangling by the proxy)
    • Visit the SI over a tor2web proxy using Tor Browser with JavaScript enabled. Verify you are redirected to the /tor2web-warning endpoint
    • Verify the warning contains the real onion address (no mangling by the proxy)
    • Visit the SI over a tor2web proxy using Tor Browser with JavaScript disabled. Verify you are not redirected to the /tor2web-warning endpoint
  • Server-side check:
    • Visit the SI over a tor2web proxy using Tor Browser with JavaScript disabled. Click "Get started". Verify you are redirected to the /tor2web-warning endpoint.
    • Verify the warning contains the real onion address (no mangling by the proxy)
  • No indexing:
    • Visit the SI over a tor2web proxy using Tor Browser with JavaScript disabled. Examine the HTML source and verify <meta name="robots" content="noindex,nofollow"> is in the head.
    • Visit the SI over a tor2web proxy using Tor Browser with JavaScript disabled. Manually navigate to /robots.txt. Verify that all indexing is disallowed.

• #6336 Prevent viewport jumps when there's flashed messages

  • Visit SI index with Tor Browser, JavaScript disabled
  • Inspect the html element with the Web Developer Tools. Using the layout tab, make sure that the content size of the html element is 1000x540 (when no scrollbar is visible)
  • Add a new submission
    • Verify browser does not scroll to Success! flashed message
    • While Success! or other flashed message is visible, use tab to navigate site
    • Skip to notification becomes visible above the logo first, Skip to main content becomes visible second

• #6320 Bring Tor Browser security level instructions up to date

  • Set Tor Browser to security level "Standard", visit SI
  • Click on the underlined "Security Level" link in the purple warning on the top of the page
  • Follow instructions, verify instructions match Tor Browser interface and terminology

• #6237 Add "skip to main content" link to all pages

  • Log into the Source Interface:
    • Observe that there is no "Skip to main content link" visible.
    • Tab through the page, with or without a screen-reader running.
    • Observe that the "Skip to main content link" appears when it receives focus.
    • Click the link and continue tabbing through the main content.
  • Log into the Journalist Interface:
    • Observe that there is no "Skip to main content link" visible.
    • Tab through the page, with or without a screen-reader running.
    • Observe that the "Skip to main content link" appears when it receives focus.
    • Click the link and continue tabbing through the main content.

• #6302 Add honeypot to Source Interface to stop very basic spambots

  • Create a new source and submit a dummy message in the SI, observe that it works fine
  • Use the browser's inspector to disable the "display: none", write something in the antispam input field (named text), and submit. You should get a 403 error screen.

• #6301 Fix text overflow in Source Interface replies

  • From the JI, send a source a long reply that is one word, like 200 exclamation marks (e.g. !!!!...).
  • Log in as that source, observe that the long reply correctly wraps and stays inside the red box.

• #6240 Add aria- annotations for WTForms validation errors

  • Log into JI as an administrator
  • Begin adding a user and enter invalid values for each field (too short username, and too long (100+ chars) first and last names)
  • Click "Add User"
  • When the form comes back with validation errors, view source and observe that each invalid field is marked aria-invalid and has an aria-describedby annotation pointing to its validation errors.

Preflight testing

Basic testing

• Install or upgrade occurs without error
• Source interface is available and version string indicates it is 2.3.0
• A message can be successfully submitted

Tails

• The updater GUI appears on boot
• The update successfully occurs to 2.3.0
• After reboot, updater GUI no longer appears