[phpBB Debug] PHP Warning: in file [ROOT]/phpbb/session.php on line 580: sizeof(): Parameter must be an array or an object that implements Countable
[phpBB Debug] PHP Warning: in file [ROOT]/phpbb/session.php on line 636: sizeof(): Parameter must be an array or an object that implements Countable
3.14.by forum • Saving the keys/hashes?
Page 1 of 1

Saving the keys/hashes?

Posted: Thu Aug 19, 2010 5:01 pm
by cr4ck
Hey guys,

I must say BarsWF is very quick. I get an average of 200M keys or whatever per second. Anyway, would it be possible to develop something that can either
a) Generate a text file full of keys (aaaaa, baaaa, caaaa, etc.) and then hash them all or
b) Generate a text file full of keys and hashes?

Sort of like a generator. Then it could be used as a rainbow table...

Just an idea and it'd be useful.

Re: Saving the keys/hashes?

Posted: Thu Aug 19, 2010 5:38 pm
by Bitweasil
1. You don't understand rainbow tables if you think they're pass:hash pairs.
2. Have you looked at the size requirements for your proposed system with any reasonable password coverage? There's a reason it's not done.

Re: Saving the keys/hashes?

Posted: Thu Aug 19, 2010 7:09 pm
by Sc00bz
You want a LHT (Lossy Hash Table) but these can only be made with key spaces of at most a few trillion. 4.8 trillion is the largest and that's on 50 TB. It can be done with 16 TB and each lookup cost 2 hard drive seeks and about 160,000 MD5s. Add or subtract 600 GB each time you double or half the number of MD5s.

Re: Saving the keys/hashes?

Posted: Sat Aug 21, 2010 4:34 pm
by LordMike
Sc00bz wrote:You want a LHT (Lossy Hash Table) but these can only be made with key spaces of at most a few trillion. 4.8 trillion is the largest and that's on 50 TB. It can be done with 16 TB and each lookup cost 2 hard drive seeks and about 160,000 MD5s. Add or subtract 600 GB each time you double or half the number of MD5s.
Ooh.. That's nice :P
My webhost gives unlimited space - anyone up for it? :P

:P

Re: Saving the keys/hashes?

Posted: Sun Aug 22, 2010 6:50 am
by Sc00bz
Use more than 100GB and you'll get kicked out. If you don't get kicked out for disk space you'll get kicked out for using 100% CPU for a few days straight. TMTO.org has been generating a 36.4 billion password LHT on 7 computers for the last few months (there was some down time and I think he regenerated part of the table).

Also you'd have to write it all in PHP or something which means it will be a lot slower to generate and use more resources. Also the server probably has a max run time on PHP scripts.

You could upload it to the server but they'll probably say you're using it a file back up or something which is normally against their TOS.

Re: Saving the keys/hashes?

Posted: Sun Aug 22, 2010 11:13 am
by blazer
how do lossy tables work? its not hashing plains and storing part of the hashes is it?

Re: Saving the keys/hashes?

Posted: Sun Aug 22, 2010 9:50 pm
by Sc00bz
Hash table:
5f4dcc3b5aa765d61d8327deb882cf99 => password
5f4dcc3bnotarealmd5blahblahblahb => bob
6104df369888589d6dbea304b59a32d4 => blink182
d0763edaa9d9bd2a9516280e9044d885 => monkey

Lossy hash table:
5f4dcc3b => passwor, bo
6104df36 => blink18
d0763eda => monke

So for "5f4dcc3b5aa765d61d8327deb882cf99" you'd search for 5f4dcc3b and get "passwor" and "bo" then you brute force the missing part of the password for each of the password ranges.

It's more commonly done like this where aaaa-azzz is "0" and zaaa-zzzz is "25" and you only store 5 bits per password range.
5f4dcc3b => aaaa-azzz, caaa-czzz
6104df36 => faaa-fzzz
d0763eda => kaaa-kzzz

There are very clever ways for indexing. Also storing part of the hash is optional but leads to less optimal tables.

Re: Saving the keys/hashes?

Posted: Sun Aug 22, 2010 10:48 pm
by blazer
Oh i see, this is very interesting indeed. So it seems you generates lots of hashes and search for 4 byte collisions store it and also part of the plain.
i get why you need to have lots of processing power for this to work now, since it needs to be big to be effective.
Does that mean you can save on some MD5 rounds since you only need the first couple of bytes?

Re: Saving the keys/hashes?

Posted: Mon Aug 23, 2010 1:35 am
by Sc00bz
Yeah but it only saves:
3 out of 64 steps for MD5 if you use the first 32 bits or less or
2 out of 64 steps for MD5 if you use 64 bits or less (the first 32 bits and last 32 bits)
You can save 1 more step if all the passwords are less than 36 bytes, but then you need to use bits 32-63 or bits 0-63.

BTW it's not always 32 bits indexed. Also it's not always going to be a nice range like aaaa-azzz it's more like aaaa-avdg. This is so you can have 2^(password range bits) password ranges and in this case it's 32 password ranges.