Update 1: The list has been slightly improved and also added examples of how the test should be applied.Update 2: Using pornography websites, there is clever technique leveraging humans that works well in defeating CAPTHA’s (A comment on my last post found an early reference). An attacker offers a free adult website granting access to any visitor who fills out CAPTCHA images. The website, acting as a CAPTCHA proxy, downloads the obfuscated image from the target then redisplays it to the visitor. Once the visitor fills out the image, or two, or three they are granted access. The attacker is then free to perform their intended action. Effective, simple, and what caused me to add #4 to the CET.
CAPTCHA
"Completely Automated Public Turing Test to Tell Computers and Humans Apart"
Just about everyone on-line has seen one typed in one of these by now, even if they didn't know exactly what it was for. CAPTCHA's are designed to prevent automated account registration, blog spam, BBS spam, whois DB lookup, login brute-force, password recovery, etc. People have attempted all sorts of strange and interesting methods to stop the bots. The obfuscated-text-in-an-image variety is the one most commonly used. The problem is not all CAPTCHA systems are created equally. Some are superior to others, but its difficult to tell exactly why. What us web application security people need is a methodology to measure the effectiveness of a CAPTCHA system. I first wrote about the CAPTCHA Effectiveness Test just over a year ago and promised to eventually make an update.
CAPTCHA Effectiveness Test
1) Test should be administered where the human and the server are remote over the network.
2) Test should be simple for humans to pass.
* Humans should fail less than 0.1% on the first attempt.
3) Test should be solvable by humans in less than a several seconds.
4) Test should only be solvable by the human to which it was presented.
5) Test should be hard for computer to pass
* Correctly guessing the answer should be less than 1 in 1,000,000, even after 24-hours of analysis.
6) Knowledge of previous test questions, answers, results, or combination thereof should not impact the predictability of following tests.
7) Test should not discriminate against humans with visual or hearing impairments.
8) Test should not possess a geographic, cultural, or language bias.
Applying the CET.
Given that the implementation is secure (many or not).
obfuscated-text-in-an-image
1) Pass
2) Pass
3) Pass
4) Fail
5) Pass
6) Pass
7) Fail
8) Pass
Hot Captcha
1) Pass
2) Fail
3) Pass
4) Fail
5) Pass
6) Fail
7) Fail
8) Fail
Still work work in progress...
7 comments:
How do you think?
http://news.com.com/2100-1023-5207290.html
*Humans should fail less than 0.1% on the first attempt.*
- I don't think so. it's flawed. Bots can now read the images. Thought this might help you.
That's where #5 comes in.
5) Test should be hard for computer to pass
* Correctly guessing the answer should be less than 1 in 1,000,000, even after 24-hours of analysis.
Hi Jeremiah,
What do you think about the reCAPTCHA (http://recaptcha.net/) service. It will at least pass test 7) over a simple image-text thing.
It would probably fall into the "obfuscated-text-in-an-image" from the above. With the right implementation, it should probably do well in most categories.
1) Pass
2) Pass
3) Pass
4) Fail
5) 50-50, with some of the new OCR and analysis technology, its hard to know for sure what this metric actually is, but its certainly called into question.
6) Pass
7) 50-50, since it has an audio portion, its not an our right failure, it getting better here. Should you be both hearing and visually impaired, then there's a larger problem.
8) Pass
What about paid humans? $3 for 1000 captchas is a welcome avocation for people in less affluent parts of the world.
@Dashmesh, this doesn't impact the test criteria because captcha's only try to determine a human from a bot. And if it takes humans to beat the system, the CAPTCHA and the test has done its job. However, if the humans now are economically viable, we have a whole other problem to tackle.
Post a Comment