QA Procedures

Release QA

Preflight scenarios can be done on either VMs or on hardware.

[tk]

[tk]

(Optional): Take a backup of your current instance.
(Optional, if reusing existing Admin workstation): Copy the following config and pubkey files from your existing Admin Workstation setup to a new folder in ~/Persistent (eg ~/Persistent/sdfiles):

~/Persistent/securedrop/install_files/ansible-base/SecureDrop.asc (or your instance's Submission Public Key)
~/Persistent/securedrop/install_files/ansible-base/journalist.asc (or your instance's journalist pubkey)
~/Persistent/securedrop/install_files/ansible-base/alerts.asc (or your instance's OSSEC alerts pubkey)
~/Persistent/securedrop/install_files/ansible-base/group_vars/all/site-specific

Then

Delete the old securedrop repo from this Admin USB
Delete ~/.ssh/known_hosts (assuming this Admin USB's SSH key is not being used for anything else)

Download and verify Ubuntu Server and perform a clean Ubuntu install (http://releases.ubuntu.com/20.04/ubuntu-20.04.3-live-server-amd64.iso) on hardware; configure Ubuntu Server as per the standard SecureDrop install process.

cd ~/Persistent
git clone https://github.com/freedomofpress/securedrop

Once cloned, git fetch --tags, verify the tag for the release branch you are testing, and check out that tag.

cd ~/Persistent/securedrop
source admin/.venv3/bin/activate 
cd install_files/ansible-base
ansible-playbook -vv --diff securedrop-qa.yml

OR:

Manually edit install_files/ansible_base/roles/install-fpf-repo/defaults/main.yml to use:

apt_repo_url: https://apt-test.freedom.press
...
apt_repo_pubkey_files:
    - apt-test-signing-key.pub

cd ~/Persistent/securedrop/install_files/ansible-base/ and copy in the required .asc files, and optionally, copy the site-specific file into ./group_vars/all/ (this will pre-populate your previous configuration during setup, and save you some time and typing).

To configure and install SecureDrop, run:

cd ~/Persistent/securedrop
./securedrop-admin setup
./securedrop-admin sdconfig
./securedrop-admin install
./securedrop-admin tailsconfig

Kernel Testing. On app server:

a. install paxtest (available via apt) and run paxtest blackhat The desired results are:
- strcpy and strcpy, PIE will be paxtest: return address contains a NULL byte
- Randomization tests: bit guesses in the high 20s or higher (there may be one or two in the mid-20s, this is still okay).
- memcpy and memcpy, PIE will show Vulnerable: this is expected on Ubuntu 20.04
- Everything else: "Killed"
b. curl -L https://meltdown.ovh -o meltdown.sh to make sure you have the latest meltdown script. Then run sudo bash meltdown.sh -v so that you have the script on the app server. Note: this should never be done on production hardware--this is for QA-only hardware. You should not see any tests that come back as STATUS:VULNERABLE.

ATTENTION: The NUC7i5BNH, NUC7i7DNHEs, NUC8s and NUC10s and generate a false-positive on one particular test (Foreshadow L1TF). This has been noted by our Security Engineer. See https://github.com/freedomofpress/securedrop/issues/5040#issuecomment-559597643.

Note: you will generate OSSEC alerts about grsec when you run these two tests--this is normal/expected.
Follow the QA matrix and the Github release ticket for further tests (end-to-end, acceptance tests, release-specific tests, etc).

Take a backup of your existing SecureDrop installation using the securedrop-admin backup command. You will need enough free space on your Admin Workstation USB to complete this backup.
Either run the ansible QA playbook:

cd ~/Persistent/securedrop
source admin/.venv3/bin/activate 
cd install_files/ansible-base
ansible-playbook -vv --diff securedrop-qa.yml

OR

Connect to each of the application and monitor servers via SSH and follow these steps:

Modify /etc/apt/sources.list.d/apt_freedom_press.list and /etc/apt/security.list to use the apt-test.freedom.press repo instead of the apt.freedom.press repo, ensure the apt-test public key is present, and update the apt cache.
Run sudo unattended-upgrades -d to upgrade your instance. Your SSH session may freeze or drop, but you should be able to reconnect thanks to tmux. You may use apt-cache policy <packagename> to check a given package's version.

Complete steps 6 and 7 of clean install QA instructions (kernel testing, QA Matrix and Github release ticket testing).

Perform a visual review of the code changes.
Check out the pull request branch on your own machine using git fetch <remotename> pull/ID/head:BRANCHNAME (If your freedomofpress/securedrop repo remote is named upstream, and you want to review PR 1337 on a local branch called newfeature, run git fetch upstream pull/1337/head:newfeature, then git checkout newfeature.)
Follow the test plan as documented in the PR, and run the appropriate unit/functional/configuration/linting tests as relevant.

Save a folder in your Tails Persistent directory called something like sdconf that contains: install_files/ansible_base/group_vars/all/site-specific, your Submission Public Key, and your admin/journalist email public key, and copy those into place (the latter two just to the ansible_base directory) on a new QA run to avoid retyping everything during sdconfig.
AppArmor can be checked with sudo aa-status
Kernel can be checked with uname -r
The submissions table is at /var/lib/securedrop/db.sqlite. Use sudo sqlite3 /var/lib/securedrop/db.sqlite to open, select * from submissions to see submission hashes, and so on.
There are some utilities (such as qa_loader.py) that may help you with testing; if their use is required, they should be documented in the test plan