Plan for user testing of sq

I will do some informal user testing of sq. In short, I will watch volunteers use sq to achieve specific tasks that I give them.

The goal of this is to find out pain points when using sq: what is easy and straightforward; what is difficult to understand; what is difficult to do. The testing will cover the sq command line tool and its built-in help, but not any other manuals or materials. The outcome I hope for is a list of proposed improvements to sq. The volunteers will not be judged or graded.

The form of the user testing is a set of tasks given to the volunteer to perform, while I observe via a video conferencing system.

The tasks to perform are:

• Log into the virtual machine.
• Generate a new key.
• Share the certificate for the new key with me.
• Get my certificate.
• Certify my key and share the result with me.
• Encrypt a message using my certificate, and send the encrypted message to me
• Receive a response from me. Verify and decrypt the response.

Test setup:

• I will provide a virtual machine with a specific version of sq installed. This is to avoid “installing sq” being part of the test, and also to make sure the version of sq being used works as expected.
• The volunteer will access the virtual machine over SSH.
• The volunteer will share their screen, or at least their terminal window where they access the virtual machine, over the video conferencing system being used.
• I will destroy the virtual machine after the test.
• Instead of email the volunteer and I will exchange files via a mutually writable directory on the virtual machine.

I will take notes and publish a blog post about what I learn.

Note

This work is supported by a grant from the NLnet foundation from the NGI Assure fund, which is financially supported by the European Council.