CUDA NTLM multi-hash bruteforcer

[Bitweasil]

Threw this together because I didn't want to do real work.

It takes a list of up to 1000 hashes, supports arbitrary character sets, and tests through length 8 right now.

On an 8800 GTX, with a big hashlist, it gets checkrates up around 20B or a bit higher, depending on the length.

It's compiled for 64-bit Linux. I have no 32-bit development boxes, as the remainder of my code laughs at 4GB. I also don't have any Windows boxes. So don't expect a Windows version any time soon. BarsWF should support it soon enough.

Enjoy. Let me know if you run into any issues.

//EDIT: New release! See page three, or download here:
32/64 bit Linux (32-bit is untested right now - feedback welcome)
http://www.cryptohaze.com/releases/NTLM ... .5.tar.bz2

Support forum at http://www.cryptohaze.com/forum/

[BarsMonster]

Nice one, looks like it is time to spend another 3 hours to set display drivers for my nVidia card under Ubuntu

[Bitweasil]

Nice one, looks like it is time to spend another 3 hours to set display drivers for my nVidia card under Ubuntu

Download nVidia binary driver. Compile against kernel. Insert module. Install CUDA SDK and toolkit. Run code. Should take 20 minutes, tops. If you've got a GUI up, it'll probably run into the 5 second watchdog timer. I don't assume video output, and my code runs straight through until it finishes.

[BarsMonster]

Nice one, looks like it is time to spend another 3 hours to set display drivers for my nVidia card under Ubuntu

Download nVidia binary driver. Compile against kernel. Insert module. Install CUDA SDK and toolkit. Run code. Should take 20 minutes, tops. If you've got a GUI up, it'll probably run into the 5 second watchdog timer. I don't assume video output, and my code runs straight through until it finishes.

Well, probably for you, not for Linux newb
Anyway, I have to do it anyway someday

[Spaztikdude]

On an 8800 GTX, with a big hashlist, it gets checkrates up around 20B or a bit higher, depending on the length.

Am I reading that right? 20 BILLION hashes per second? You should only be doing like... ~400 million, then again, that's with BarsWF's 0.1 tool...

It's compiled for 64-bit Linux.

Aww shucks. Sure I have a C2D which has 64bit extensions, but... I don't feel like a 2nd install of Ubuntu.

[Bitweasil]

BarsWF does around 400M, that's correct. My single hash brute forcer for NTLM was around 435M, but that could be optimized to somewhere around 550-600M depending on the hash reversals - I've not played with that yet.

I said my effective check rate was around 20B, and that's correct. If there are a lot of hashes, yes, it slows the step rate down, but it's checking all the hashes. And a step rate of 20M with 1000 hashes is an effective hash check rate of 20B hashes per second.

I expect Bars to be faster when he gets it done. Mine's not an ideal implementation, just a functional one. It's a tiny component to a far more interesting project.

[TrashCanMan]

Hashes loaded (4 hashes)
1E0E867AB8043BBBA9EF4639DBDF562E
3F839BC6E10BA548FF6FD2F9F6209EAD
83E27A37914E49EE2E433877AF5CF35B
17C0D5F216E931A6A48EBE311E0A05CC
CPU: Searching length 1, charset length 26, 4 passwords.
Compute done: Reference time 0.041 ms
Stepping rate: 0.6M MD4/s
Search rate: 2.5M NTLM/s

CPU: Searching length 2, charset length 26, 4 passwords.
Compute done: Reference time 0.206 ms
Stepping rate: 3.3M MD4/s
Search rate: 13.1M NTLM/s

CPU: Searching length 3, charset length 26, 4 passwords.
Compute done: Reference time 5.406 ms
Stepping rate: 3.3M MD4/s
Search rate: 13.0M NTLM/s

GPU: Searching length 4, charset length 26, 4 passwords.
Blocks, threads: [16 x 384]
Compute done: Reference time 5.861 ms
Stepping rate: 78.0M MD4/s
Search rate: 311.9M NTLM/s

GPU: Searching length 5, charset length 26, 4 passwords.
Blocks, threads: [32 x 320]
Compute done: Reference time 126.809 ms
Stepping rate: 93.7M MD4/s
Search rate: 374.8M NTLM/s

GPU: Searching length 6, charset length 26, 4 passwords.
Blocks, threads: [32 x 320]
Compute done: Reference time 3238.162 ms
Stepping rate: 95.4M MD4/s
Search rate: 381.6M NTLM/s

GPU: Searching length 7, charset length 26, 4 passwords.
Blocks, threads: [32 x 256]
Cuda error: the launch timed out and was terminated.

Quad-core Intel w/ a single NVIDIA 8600 GT, all drivers, toolkits and SDK installed and operable. OS is Xubuntu 8.10

[Bitweasil]

Are you running with X11 installed/running?

My code does NOT pay attention to watchdog timers. If there's a watchdog timer killing the process after 5 seconds or so (which appears to be the case, as len6 took around 4 seconds), you will get errors like that.

Kill X11, SSH in, and try again. I run my code on headless servers, so I don't care to limit my kernel execution to 5sec segments.

[TrashCanMan]

Kill X11, SSH in, and try again.

That did the trick, thanks. I'll have to research how to turn off watchdog times in gdm, if that's possible.

[Bitweasil]

No idea how to actually do that. I suspect it's a lower level than GDM, either at the base X11 level, or in the display drivers.

If you figure out how to do it, let me know & I'll update the readme for future releases.

[huhwha]

src plz?

[Bitweasil]

The source is still in flux, and right now is beyond ugly. I'm working on making it function with a GUI (where it doesn't run into watchdog timer issues) currently.

[Bitweasil]

I've got a new release of this that I'm finishing testing & packaging.

I will be releasing versions for both 32-bit and 64-bit Linux initially, and am working on learning how to compile this stuff for Windows (where I will be releasing 32-bit binaries).

The new version does not interfere with the GUI - you can use it on a system with a GUI display, and it does not interfere with doing work while the cracker is in use (though you can adjust the per-step timing to improve performance, if you wish).

It supports cracking up to 2000 hashes in parallel, and supports lengths 0-16 (though 16 is still beyond the realm of crackable for anything but very small charsets).

Right now, it only supports a single GPU. Multi-GPU support may come at some point, but I will be working on other projects before multi-GPU support.

I will also be releasing the code once it is completed and cleaned up, for those curious.

[neinbrucke]

I will also be releasing the code once it is completed and cleaned up, for those curious.

awesome i bet i can use that to when i decide to go and learn cuda

[synthesis]

Even if the project name is "NTLM Multiforce" do you plan to support other algorithms or make similar programs to crack md5, *nix md5 etc? It would be great to study these sources...

[Bitweasil]

Even if the project name is "NTLM Multiforce" do you plan to support other algorithms or make similar programs to crack md5, *nix md5 etc? It would be great to study these sources...

Yes, I intend to include support for other algorithms.

They're likely not going to be nearly as optimized as BarsMonster's code, as I just don't have the time to spend on algorithm optimization, but they're still a lot faster than single hash brute forcing for hash lists.

Working on the final bits today/tonight, should have something released for Linux tomorrow, and then Windows support will come as soon as I get a Windows dev environment set up & figure out how to make things compile for CUDA in Windows.

[Bitweasil]

Found more performance, need to write a chunk of benchmark code (as right now it's tuned for things like a GTX260, and won't play nice with a GUI on anything much slower than an 8800 - testing on my 8600 was somewhere between funny and painful with the GTX260 block/thread counts).

I'm running about 560M MD4/s on a GTX260 right now with a single hash, which is definitely slower than BarsWF (especially considering MD4 is only 75% the work of MD5), but on very large hash sets, it scales to around 25B checks per second.

I'm going to finish the benchmarker, release it, then work on hash reversal (unless anyone else wants to implement it, since it will come with the code...)

[synthesis]

So I will change my Geforce 7950 on my Gentoo desktop pc with a 8xxx one to try your code

[Bitweasil]

So I will change my Geforce 7950 on my Gentoo desktop pc with a 8xxx one to try your code

Are you running a 32-bit or 64-bit install? If it's 64-bit, I could send you some code to play with currently - it's not quite done, but it's very usable right now.

[synthesis]

It's 32 bit one
BTW my brother probably has a Geforce 8600GT (not that powerful, but nice for some tests)... I will install SDK, Toolkit etc. and then try your 32bit code or binary when it will be ready. Thanks again and good work! I'd like to help you but my knowledge about this area is not enough

[Bitweasil]

I really don't understand why people still insist on 32-bit OSes...

I also need to finish my benchmarking code to tune block size, as I suspect my current GTX260 optimized settings would not work well on an 8600.

[BarsMonster]

I really don't understand why people still insist on 32-bit OSes...

I also need to finish my benchmarking code to tune block size, as I suspect my current GTX260 optimized settings would not work well on an 8600.

Well, why don't you like to do a small benchmark, or guess block size based on SP count?

[Bitweasil]

I'm going to benchmark - I just haven't finished that block of code yet.

[Bitweasil]

New version released:

32/64 bit Linux binaries (with launcher):
http://www.cryptohaze.com/releases/NTLM ... .5.tar.bz2

Support at http://www.cryptohaze.com/forum/

I'll write up more docs later - going bowling now.

Performance is right around 25B hashes per second on a GTX260 with a hash list of 185 hashes.

[Bitweasil]

And... found a bug.

I wouldn't use a non-multiple-of-8 charset right now

I'll fix it and release a new version this evening.

... and maybe release a MD5 multiforcer while I'm at it.

Still working on the Windows code - Visual Studio 2008 doesn't like my C code and doesn't include a lot of headers I include.