Abstract
Secure email is increasingly being touted as usable by novice users, with a push for adoption based on recent concerns about government surveillance. To determine whether secure email is ready for grassroots adoption, we employ a laboratory user study that recruits pairs of novice users to install and use several of the latest systems to exchange secure messages. We present both quantitative and qualitative results from 28 pairs of novices as they use Pwm, Tutanota, and Virtru and 10 pairs of novices as they use Mailvelope. Participants report being more at ease with this type of study and better able to cope with mistakes since both participants are “on the same page.” We find that users prefer integrated solutions over depot-based solutions and that tutorials are important in helping first-time users. Additionally, hiding the details of how a secure email system provides security can lead to a lack of trust in the system. Finally, our results demonstrate that PGP using manual key management is still unusable for novice users, with 9 out of 10 participant pairs failing to complete the study.
Paper
TBA
Usage Policy
This data is intended to be used for usage in academic research. No attempt should be made to de-anonymize users.
Data
tops2018-data_sanitized.xlsx (Replication)
tops2018-data_sanitized.xlsx (Mailvelope)
Surveys
Johnny’s Survey (Replication)
Jane’s Survey (Replication)
Johnny’s Survey (Mailvelope)
Jane’s Survey (Mailvelope)
Recruiting Poster
Poster (Replication)
Poster (Mailvelope)
Screenshots
Pwm
Pwm’s integrated tutorial.
Pwm’s secure composition interface.
Pwm’s secure read interface.
Tutanota
Tutanota’s sign up page.
Tutanota’s secure compose interface.
Tutanota’s password-encrypted email.
Tutanota’s password-encrypted email’s password entry interface.
Tutanota’s secure read interface.
Virtru
Virtru’s integrated tutorials.
Virtru’s secure composition interface.
Virtru’s integrated secure read interface.
Virtru’s depot-based encrypted email.
Virtru’s depot-based secure read interface.
Mailvelope
Mailvelope’s button to enable secure compose. Found in the upper right corner of the webmail provider’s compose interface.
Mailvelope’s secure composition interface.
Mailvelope’s public key selection interface.
Mailvelope’s secure read interface.
Mailvelope-encrypted message.
Mailvelope’s private key password entry interface.
Mailvelope’s encrypted message in the webmail provider’s compose interface.