Saving the keys/hashes?

Moderator: BarsMonster

Post Reply [phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable
cr4ck
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Saving the keys/hashes?

Post by cr4ck » Thu Aug 19, 2010 5:01 pm

Hey guys,

I must say BarsWF is very quick. I get an average of 200M keys or whatever per second. Anyway, would it be possible to develop something that can either
a) Generate a text file full of keys (aaaaa, baaaa, caaaa, etc.) and then hash them all or
b) Generate a text file full of keys and hashes?

Sort of like a generator. Then it could be used as a rainbow table...

Just an idea and it'd be useful.

Bitweasil
Posts: 110
Joined: Fri Nov 07, 2008 6:50 am
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Re: Saving the keys/hashes?

Post by Bitweasil » Thu Aug 19, 2010 5:38 pm

1. You don't understand rainbow tables if you think they're pass:hash pairs.
2. Have you looked at the size requirements for your proposed system with any reasonable password coverage? There's a reason it's not done.

Sc00bz
Posts: 136
Joined: Fri Oct 03, 2008 8:28 am
Contact:

Re: Saving the keys/hashes?

Post by Sc00bz » Thu Aug 19, 2010 7:09 pm

You want a LHT (Lossy Hash Table) but these can only be made with key spaces of at most a few trillion. 4.8 trillion is the largest and that's on 50 TB. It can be done with 16 TB and each lookup cost 2 hard drive seeks and about 160,000 MD5s. Add or subtract 600 GB each time you double or half the number of MD5s.

User avatar
LordMike
Posts: 184
Joined: Tue Feb 10, 2009 8:34 am
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Re: Saving the keys/hashes?

Post by LordMike » Sat Aug 21, 2010 4:34 pm

Sc00bz wrote:You want a LHT (Lossy Hash Table) but these can only be made with key spaces of at most a few trillion. 4.8 trillion is the largest and that's on 50 TB. It can be done with 16 TB and each lookup cost 2 hard drive seeks and about 160,000 MD5s. Add or subtract 600 GB each time you double or half the number of MD5s.
Ooh.. That's nice :P
My webhost gives unlimited space - anyone up for it? :P

:P

Sc00bz
Posts: 136
Joined: Fri Oct 03, 2008 8:28 am
Contact:

Re: Saving the keys/hashes?

Post by Sc00bz » Sun Aug 22, 2010 6:50 am

Use more than 100GB and you'll get kicked out. If you don't get kicked out for disk space you'll get kicked out for using 100% CPU for a few days straight. TMTO.org has been generating a 36.4 billion password LHT on 7 computers for the last few months (there was some down time and I think he regenerated part of the table).

Also you'd have to write it all in PHP or something which means it will be a lot slower to generate and use more resources. Also the server probably has a max run time on PHP scripts.

You could upload it to the server but they'll probably say you're using it a file back up or something which is normally against their TOS.

blazer
Posts: 14
Joined: Sun Oct 05, 2008 8:10 am
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Re: Saving the keys/hashes?

Post by blazer » Sun Aug 22, 2010 11:13 am

how do lossy tables work? its not hashing plains and storing part of the hashes is it?

Sc00bz
Posts: 136
Joined: Fri Oct 03, 2008 8:28 am
Contact:

Re: Saving the keys/hashes?

Post by Sc00bz » Sun Aug 22, 2010 9:50 pm

Hash table:
5f4dcc3b5aa765d61d8327deb882cf99 => password
5f4dcc3bnotarealmd5blahblahblahb => bob
6104df369888589d6dbea304b59a32d4 => blink182
d0763edaa9d9bd2a9516280e9044d885 => monkey

Lossy hash table:
5f4dcc3b => passwor, bo
6104df36 => blink18
d0763eda => monke

So for "5f4dcc3b5aa765d61d8327deb882cf99" you'd search for 5f4dcc3b and get "passwor" and "bo" then you brute force the missing part of the password for each of the password ranges.

It's more commonly done like this where aaaa-azzz is "0" and zaaa-zzzz is "25" and you only store 5 bits per password range.
5f4dcc3b => aaaa-azzz, caaa-czzz
6104df36 => faaa-fzzz
d0763eda => kaaa-kzzz

There are very clever ways for indexing. Also storing part of the hash is optional but leads to less optimal tables.

blazer
Posts: 14
Joined: Sun Oct 05, 2008 8:10 am
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Re: Saving the keys/hashes?

Post by blazer » Sun Aug 22, 2010 10:48 pm

Oh i see, this is very interesting indeed. So it seems you generates lots of hashes and search for 4 byte collisions store it and also part of the plain.
i get why you need to have lots of processing power for this to work now, since it needs to be big to be effective.
Does that mean you can save on some MD5 rounds since you only need the first couple of bytes?

Sc00bz
Posts: 136
Joined: Fri Oct 03, 2008 8:28 am
Contact:

Re: Saving the keys/hashes?

Post by Sc00bz » Mon Aug 23, 2010 1:35 am

Yeah but it only saves:
3 out of 64 steps for MD5 if you use the first 32 bits or less or
2 out of 64 steps for MD5 if you use 64 bits or less (the first 32 bits and last 32 bits)
You can save 1 more step if all the passwords are less than 36 bytes, but then you need to use bits 32-63 or bits 0-63.

BTW it's not always 32 bits indexed. Also it's not always going to be a nice range like aaaa-azzz it's more like aaaa-avdg. This is so you can have 2^(password range bits) password ranges and in this case it's 32 password ranges.

Post Reply
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Who is online

Users browsing this forum: No registered users and 1 guest