I’m worried about password strength meters.
In March 2015 I tested five popular password strength meters in a simple experiment that was designed to show if they could actually spot weak passwords. They all failed.
It’s been almost eighteen months since my original test and during that time password cracking has moved on, authentication standards have moved on and password best practice has moved on.
I wondered if password strength meters had too.
The problem
There is a gap between what password strength meters tell us and what we need to know.
On the face of it, password strength meters seem like a great idea – when a user needs to create a password for a website, the meter can tell the user how strong their choice of password is and, most crucially of all, help them steer clear of really bad passwords.
The trouble is that most password strength meters don’t actually measure password strength at all.
A strong password is one that is highly resistant to attempts to crack it with online or offline dictionary attacks. The only good way to measure the strength of a password is to try and crack it – a serious and seriously time consuming business that requires specialist software and expensive hardware.
So instead of measuring the thing we really care about, password strength, most meters actually measure something that’s easy to figure out: password entropy.
A password with a lot of entropy should be hard to crack by brute force (guessing) but that’s a password cracker’s technique of last resort. Their first line of attack is likely to be based on dictionary words and rules that mimic the common tricks we use to di5gu!se th3m. Measuring entropy doesn’t tell us anything about that.
The test
In both tests I used the same five terrible passwords, passwords that would fail a genuine cracking attempt instantly, and then ran them through five popular password strength meters.
The premise of the test is simple; password strength meters should dismiss all of the passwords out-of-hand so a failure to dismiss any password is a failure of the whole test.
Rejecting all of the passwords doesn’t prove that a password strength meter is good, but accepting one of the unsafe passwords shows that it’s not up to the job.
Five terrible passwords
The passwords I used in the test are all, deliberately, absolutely dreadful. They’re chosen from a list of the 10,000 most common passwords and have characteristics I thought the password strength meters might overrate:
- abc123 – number 14 on the list, first to mix letters and numbers
- trustno1 – number 29, second to mix letters and numbers
- ncc1701 – number 158, registration number of the USS Enterprise
- iloveyou! – number 8778, first with non-alphanumeric character
- primetime21 – number 8280, longest with letters and numbers
Being on the list of the 10,000 most common passwords is broadly synonymous with being one of the 10,000 worst passwords.
Even if a hard to crack password got on the list by accident it would instantly become a weak, easy to crack, password because it’s on the list. Password crackers seed their dictionary attacks with lists of common words and passwords they think people are likely to use. If your password is on that list, it’s toast.
To check my assumptions, I ran the five passwords through John the Ripper and cracked them on my laptop using its out-of-the-box settings. They all fall in well under a second.
The meters
The meters were chosen by googling ‘jQuery strength meter’ and picking the first five that came up. This is is the kind of thing a web developer would do if you asked them to add a password strength meter to your website.
Two of the five meters under test, the jQuery Password Strength Meter for Twitter Bootstrap and Strength.js were also in the first five results in 2015.
- jQuery Password Strength Meter for Twitter Bootstrap
- Strength.js
- Mato Ilic’s PWStrength
- FormGet’s jQuery Password Strength Checker
- Paulund’s jQuery password strength demo
The ringer
This year I added a ringer to my tests; zxcvbn. It’s a sophisticated, open source password strength meter used by Dropbox and WordPress that’s been rigorously tested.
I added it to the test so that it’s clear what a website password strength meter of proven quality does when faced with this test.
The results
My table of results below uses the same colours and words (sometimes abbreviated but with misspellings faithfully reproduced) that the password strength meters use:
| Password | 1 | 2 | 3 | 4 | 5 | ZXCVBN |
|---|---|---|---|---|---|---|
| abc123 | Weak | Week | Very weak | Weak | Weak… | Very weak |
| trustno1 | Normal | Week | Very weak | Good | Make it… | Very weak |
| ncc1701 | Medium | Week | Very weak | Weak | Make it… | Very weak |
| iloveyou! | Medium | Week | Mediocre | Good | Weak… | Very weak |
| primetime21 | Medium | Medium | Weak | Good | Make it… | Very weak |
The result, sadly, is exactly the same as 2015. They all failed.
The ringer, zxcvbn, identified the five passwords as very weak but none of the first five password strength meters I plucked out of Google did.
Just as they did in 2015 the meters also muddy the waters with misleading or ambiguous terminology and colours – what is a medium or mediocre password?
Recommendations
If you’re a website user
- You can’t trust password strength meters on websites
- Watch our video on how to pick pick a proper password
If you’re a website operator
- If you want a password strength meter for your website don’t guess, use zxcvbn
- Use two-factor authentication so that hackers can’t get into your site with just a cracked password
- Reduce the danger of bad passwords by locking users out after a few failed login attempts