Distributed crypo-network discussion

[BarsMonster]

oh btw, i have 3MB per core, so it's 12MB... you probably have the same, but dual core?
i have a Q9450, running it at 3.2 GHz. For you information: I get like 200-210 Mhashes/s with BarsWF.

brute forcer: http://blog.distracted.nl/2009/10/emdeb ... ihash.html (this version actually has a couple of bugs, but it finds hashes on windows )
i explain most of the stuff i use here: http://blog.distracted.nl/2009/09/multi ... pport.html
partial reversing still gained some speed over full md5... most speed is probably in the use of bitmaps for initial hash comparison, i used bitweasils implementation and optimized it.

How many hashes are in your hash-list?

[neinbrucke]

With a hash list of 200 000 hashes EmDebr is just falling back to around 119 Mhashes/s

[BarsMonster]

Awesome idea with 2x bitmaps...
I see, I am on 460'000.

Actually, killer solution is to fetch a required line from L2/RAM in advance, and while it's loading do some rounds of next MD5 key.
But that's gonna cause kinda scary code

[neinbrucke]

hehe, i'll leave such scary code up to you (or your intel compiler )
hmm... no idea if intel compiler will actually improve speed of my multi hash version... maybe i'll need to go and grab myself a trial version again

[BarsMonster]

hehe, i'll leave such scary code up to you (or your intel compiler )

Intel C++ is not gonna prefetch that, unfortunately (that would be too good to be true, but CUDA does that)

hmm... no idea if intel compiler will actually improve speed of my multi hash version... maybe i'll need to go and grab myself a trial version again

Let us know results
I bet it will

[LordMike]

Why not let microprocessors do the job!?!..

Just kidding anyhow.. Learnt they work at 1 MHz at school last friday
Got 160 pages datasheet on microprocessors

[neinbrucke]

fyi: opencrack uncracked list of something like 460k hashes drops to around 113 Mhashes

[BarsMonster]

oh btw, i have 3MB per core, so it's 12MB... you probably have the same, but dual core?

Well, you have 6Mb for cores 1 and 2, and another 6Mb for 3 and 4
Yes, I have the same but 2 cores (C2D E8400@3.86)

[schwarzwaldhacker]

most speed is probably in the use of bitmaps for initial hash comparison, i used bitweasils implementation and optimized it.

I will disclose a somewhat similar idea I got, when I started to write a DES Beta:

* Before I start the Kernel routine, I calculate hashes of all the hashes(comp2 is where the Hashes are stored, after having being read from file. In DES, the two first characters are the salt and I didn't include them in order to speed up things):

Code: Select all

for (int i=0; i<PassCountr;i++)
{
int testsumme=0;
for (int j=2; j<13; j++)
{
testsumme=(abs(testsumme-comp2[i*13+j])+j)*2;
}
comp3[i]=testsumme;
}

For 768 hashes, the results were from like 2 to around 150 000(Good formula, because the results are "evenly split", "well spread" or something. And in Cuda Kernel, you can use "mul24" and "sad", which is quite fast).

* Then I sort them from the smallest to the biggest.
* Now, every time Cuda bruteforced a new combination, I created a Hash of it("zwischenergebnis"):

Code: Select all

	#pragma unroll 11
	for (i=2; i<13; i++)
		{
		c=0;
		#pragma unroll 6
		for (j=0; j<6; j++)
			{
			c<<=1;
			if (bb[y] & u) c|=1;
			u>>=1;
			if (!u)
				{
				y++;
				u=0x80;
				}
			}
zwischenergebnis=__mul24((__sad(zwischenergebnis,cov_2char[c],i)),2);
		}

* And I compared this Hash("zwischenergebnis") to the list of "pre-calculated" Hashes(compd3 is the sorted list of comp2 Hashes in the Kernel, "dcount" the number of compd3 hashes):

Code: Select all

while ((zwischenergebnis>=compd3[i])&&(i<dcount))
{
if (zwischenergebnis==compd3[i])
[...]

Benefits:
* On average, only around ((50%)*Passwords) "simple tests" are being made("simple tests", because only the Hashes are compared, no full hash calculations).
* And usually, in millions of checks, only the bruteforced combination is being fully calculated(For instance, the DES cracker calculates the DES Hash for "aaa"+the hash of it). If no equal compd3-Hashes are found, no other Hashes are getting fully calculated(If no other Hashes have equal Hashes to the "calculated Hash", no further tests are being made).

[Crucifix]

Man, I'm dying of waiting for this thing. It's gonna FLY through that 400K hash list.

Also, is all this coded in C++ ? I only know Java, a lot of the operators and reserve words look similar but some look very different.

[BarsMonster]

Man, I'm dying of waiting for this thing. It's gonna FLY through that 400K hash list.

Also, is all this coded in C++ ? I only know Java, a lot of the operators and reserve words look similar but some look very different.

Yes, C++ is the only way to reach top performance, so both CPU & GPU parts are in C++.

[BarsMonster]

bars, how much cache is on your cpu?
cause i 'only' dropped to like 70-75% with multihash support (125M) on sse2 md5. Of course i started out with a somewhat lower single hash speed (175M), but still

p.s. i have 3MB L2 cache.

Thanks to your heads up, I found a nice bug caused my bitmaps not working at all (and I had disabled them at all thinking they are not effective)
Now I am at 63%, which is 87Mhash for 2 cores @3.86Ghz. (And I was too lazy to reverse 8 steps with that hashlist rebuild - too much burden when you have to take care of GPUs too)

Error was kinda simple:

Code: Select all

	int aa = (a)&((1<<CPU_HASH_BITS)-1);
	if((g->a->cpu_bitmask_a[aa>>5]) & (1<<(aa&31)) == 0)
		return;

Spoiler Alert: After checking disassembly found out that I forgot that == have higher priority than &

Meanwhile, sent another bug report to Intel, fortunately not a showstopper for BarsWF/Salty cracker

[LordMike]

Simple?..

[K.9]

Let's start the beta test now?

[DarkPrince]

what is happening?

1½ year?

[_haxxor_]

i think he's lazy

[BarsMonster]

what is happening?
1½ year?

That sucks really hard

i think he's lazy

Also, need to earn less

[Bruter]

EnFuzion could be useful in the distributed password cracking.

Benefits of EnFuzion is support of varied Operating Systems :

Windows on x86 , 32-bit and 64-bit
Linux on x86, 32-bit and 64-bit
Mac OS X on PowerPC
Sun Solaris on Sparc
IBM AIX on PowerPC
HP HP-UX on PA-RISC
SGI Irix on MIPS
HP Tru64 on Alpha

http://www.axceleon.com/
http://www.axceleon.com/install/
http://www.axceleon.com/specs.html
http://www.axceleon.com/features.html
http://forum.ru-board.com/topic.cgi?for ... ic=46276#1

[Rolf]

EnFuzion uses CPUs only as I've found out.That's old.

[Bruter]

You are wrong!
EnFuzion is a Pipeline Manager, Job Creator and Job Scheduler, so it uses what you give him.

[Rolf]

Prove me wrong.
I've found nothing there about CUDA/CAL/OpenCL.

[_haxxor_]

Commercial license. not for us

[vampyr]

Well, considering the fact that I've already put a cracker WITH network support out there;)
Sure, the networking code isn't all that pretty, but it works. And it has a fairly low bandwidth cost.
Seriously tho, php or any web-scripting language isn't going to cut it for millions of hashes. Might as well use c++ and save some time.

[LordMike]

Well, considering the fact that I've already put a cracker WITH network support out there;)
Sure, the networking code isn't all that pretty, but it works. And it has a fairly low bandwidth cost.
Seriously tho, php or any web-scripting language isn't going to cut it for millions of hashes. Might as well use c++ and save some time.

Why would you code a website in C++?.. Or an application in php?..
I know that the latter is possible.. But isn't it a little ... Over-exceeding?

[vampyr]

Well, i wasn't talking about coding the whole WEBSITE in c++, just the server distributing the tasks. 'cause php is too slow for those purposes.
Oh that, and that sending hashes in text instead of in binary form wastes bandwidth.