BarsWF 1.0 network compilation

Moderator: BarsMonster

User avatar
BarsMonster
Site Admin
Posts: 1118
Joined: Wed Oct 01, 2008 7:58 pm
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

BarsWF 1.0 network compilation

Post by BarsMonster » Fri Oct 03, 2008 1:32 am

To implement new cool features in BarsWF I would need dynamic CUDA core recompilation. I am starting to think that the best way of doing this is to have a compilation server, which will provide CUDA kernels for all clients. What does that mean to end users?

When you set some parameters, and launch BarsWF it connects to compilation server, sends info like salts size, number of hashes, and downloadods 10kb cubin code which will be executed on the videocard. No x86 code downloaded. Cubin code is cached so that if you run several times with similar parameters, it won't connect again.

Will that scare you?

Another possiblity is to bundle compilation tools with BarsWF, but that would be pain in the ass to support all operating systems in that case. (Cubin code is the same for Win32/64, Linux, MacOS)

User avatar
the_drag0n
Posts: 217
Joined: Thu Oct 02, 2008 6:48 am
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Re: BarsWF 1.0 network compilation

Post by the_drag0n » Fri Oct 03, 2008 5:43 am

hmm that would be ok for me as long as it doesnt send any other data ;)

User avatar
BarsMonster
Site Admin
Posts: 1118
Joined: Wed Oct 01, 2008 7:58 pm
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Re: BarsWF 1.0 network compilation

Post by BarsMonster » Fri Oct 03, 2008 6:06 am

Because I need different cubin file for different password length, salt length, number of hashes being cracked.
Universal cubin would be ~twice slower.

We can have all info to be sent unencrypted, so that anyone would be able to check using sniffer that nothing non-necessary is being sent.

I can compile it on your computer BUT it would require CUDA sdk to be installed (60Mb) and probably C++ compiller.
It is possible to generate ptx code which is easier to compile (compiller weight 1.1Mb). Unfortunately it is harder to write ptx code, which means less new features / day

User avatar
the_drag0n
Posts: 217
Joined: Thu Oct 02, 2008 6:48 am
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Re: BarsWF 1.0 network compilation

Post by the_drag0n » Fri Oct 03, 2008 6:18 am

if you think you can handle the traffic and are willing to send everything uncryptet i would be fine with that.


i think what budden means is simply saving all possible cubin files on your own pc.
if the files are only 10kb each you should be able to save them on ones pc

User avatar
BarsMonster
Site Admin
Posts: 1118
Joined: Wed Oct 01, 2008 7:58 pm
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Re: BarsWF 1.0 network compilation

Post by BarsMonster » Fri Oct 03, 2008 6:27 am

Yes they would be saved and will not be requested again. But there are too much combinations to compute them in advance:
My rough estimation is ~32*32*32*11 - 360k.

User avatar
the_drag0n
Posts: 217
Joined: Thu Oct 02, 2008 6:48 am
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Re: BarsWF 1.0 network compilation

Post by the_drag0n » Fri Oct 03, 2008 6:32 am

hmm that would be 3.6Gb. well if anyone doesnt like the feeling he could still download all the files and the server would never be requested right ?

User avatar
BarsMonster
Site Admin
Posts: 1118
Joined: Wed Oct 01, 2008 7:58 pm
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Re: BarsWF 1.0 network compilation

Post by BarsMonster » Fri Oct 03, 2008 6:56 am

the_drag0n wrote:hmm that would be 3.6Gb. well if anyone doesnt like the feeling he could still download all the files and the server would never be requested right ?
I guess not. The problem that these files would be generated on the server, and will require ~5 seconds of CPU time for each 10kb :-) That would take AGES to generate them on server :-)

User avatar
the_drag0n
Posts: 217
Joined: Thu Oct 02, 2008 6:48 am
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Re: BarsWF 1.0 network compilation

Post by the_drag0n » Fri Oct 03, 2008 6:58 am

well then every client should generate it itself so you can avoid server load and traffic.

User avatar
the_drag0n
Posts: 217
Joined: Thu Oct 02, 2008 6:48 am
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Re: BarsWF 1.0 network compilation

Post by the_drag0n » Fri Oct 03, 2008 7:21 am

well if the distributed project comes you could prebuffer the files youll need.
i think you will still be able to do f.e. md5 hashes offline as now.
maybe bar will release some extra offline versions for certain hash algos...

kiando
Posts: 64
Joined: Thu Oct 02, 2008 7:30 am
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Re: BarsWF 1.0 network compilation

Post by kiando » Fri Oct 03, 2008 8:27 am

First of all: It does scare me. but I'm a coward. so I guess it doesn't say anything.

The network thing could also have the advantage of doing distributed bruteforce. it just needs a feature to define the space for each client.
for instance:

charset:ABCDEFGHIJK; len:1-8

machine #1: A - GDBKHFDB
machine #2: HDBKHFDB - CIDAGBID
machine #3: DIDAGBID - FFCAEIBG
machine #4: GFCAEIBG - ICBACEGI
machine #5: JCBACEGI - KKKKKKKK

bias_resistor
Posts: 18
Joined: Fri Oct 03, 2008 10:07 am
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Re: BarsWF 1.0 network compilation

Post by bias_resistor » Fri Oct 03, 2008 10:17 am

doesn't CUDA support external modules? like DLL files in windows?
you could use 7zip bzip2 or zlib compression for the cubin files.

the way i think about it, you would have a limit of how many computers can be connected at one time.

MASTER NODES distribute jobs to SUB NODES and these SUB NODES distribute to clients
Each sub node would support 255 clients for example, like on network 192.168.1.1 - 255

User avatar
BarsMonster
Site Admin
Posts: 1118
Joined: Wed Oct 01, 2008 7:58 pm
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Re: BarsWF 1.0 network compilation

Post by BarsMonster » Fri Oct 03, 2008 10:30 am

The network thing could also have the advantage of doing distributed bruteforc
This is exactly how this would work. Each client will get his own key space to check.

Ofcourse CUDA allows to load cubin from external sources. The only problem is to get this cubin file. I cannot ask every user of BarsWF to install CUDA SDK & Visual Stuido to be able to generate cubin on the client computer.

Some cubins might be bundled, but even for unsalted md5 there will be like 20 cubins for different password lengths

Distributing work between clients is not a problem. ALso, there will be no "subnodes" - just 1(or few) central servers. Each client will send/receive data each 10 minutes or so, so even 1 server can handle thousands of clients.

User avatar
the_drag0n
Posts: 217
Joined: Thu Oct 02, 2008 6:48 am
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Re: BarsWF 1.0 network compilation

Post by the_drag0n » Fri Oct 03, 2008 10:33 am

and we will earn coins per hour ?
and how much coins would you need for submitting a hash or a salted hash.
or would it cost refering to the time it takes to crack?

and btw i agree with you because i was never able to compile any cuda program out of some weird reasons.
it keeps complaining about my version of cl.exe :/
i tried to install pyrit but i failed :wall:

kiando
Posts: 64
Joined: Thu Oct 02, 2008 7:30 am
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Re: BarsWF 1.0 network compilation

Post by kiando » Fri Oct 03, 2008 10:38 am

the_drag0n wrote:and we will earn coins per hour ?
I'd prefer to be able to join a kind of "project". e.g. "md5" or "sha1" instead of computing tons of parts for algorithms that I have no hash to check for.

User avatar
the_drag0n
Posts: 217
Joined: Thu Oct 02, 2008 6:48 am
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Re: BarsWF 1.0 network compilation

Post by the_drag0n » Fri Oct 03, 2008 10:41 am

why that ?
the princip is you crack hashes of any kind for other persons and gain coins for it.
then you can use these coins to submit new hashes which are cracked by other persons.

User avatar
BarsMonster
Site Admin
Posts: 1118
Joined: Wed Oct 01, 2008 7:58 pm
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Re: BarsWF 1.0 network compilation

Post by BarsMonster » Fri Oct 03, 2008 10:45 am

the_drag0n wrote:and we will earn coins per hour ?
and how much coins would you need for submitting a hash or a salted hash.
or would it cost refering to the time it takes to crack?

and btw i agree with you because i was never able to compile any cuda program out of some weird reasons.
it keeps complaining about my version of cl.exe :/
i tried to install pyrit but i failed :wall:
1 point would mean 1 computing hour at 1 teraFLOP (4 example).
So you earn points, and when you spend them, you will have just the same amount of work, but in a short time.
So in theory, if you have 1 point you may have 1 minute of 60-node supercomputer (provided that speeds are the same)
Also, remember that for unsalted hashes you would get 1 hour for just 0.05 points or so :-)

So it does not matter if it is your algorithm or not. Today you crack someone else's hashes, tomorrow someone else will crack yours. And tomorrow you will be able to spend these points very fast by loading the whole cluster:-)

PS. You may send me by mail your error message with versions of software you have installed. We can try to work it out :-)

User avatar
the_drag0n
Posts: 217
Joined: Thu Oct 02, 2008 6:48 am
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Re: BarsWF 1.0 network compilation

Post by the_drag0n » Fri Oct 03, 2008 10:51 am

BarsMonster wrote: 1 point would mean 1 computing hour at 1 teraFLOP (4 example).
that would be a bit mouch.
1TFLOP would mean youd need multiple gpus to do 1 real hour
in my case id need about 9hours for that...

i think 1 point per hour @ 200GFLOP or sth like that would be better.
or youd simply let the project run a while and see what the average is...

User avatar
BarsMonster
Site Admin
Posts: 1118
Joined: Wed Oct 01, 2008 7:58 pm
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Re: BarsWF 1.0 network compilation

Post by BarsMonster » Fri Oct 03, 2008 10:57 am

Well, I am not saying "you will get nothing until you will have 1 point". You might get 0.07 points per hour, but this might look frustrating. Probably we will scale it a bit, so that 1teraFLOP would mean 100 points :-) And points would be fractional even in this way. I will not let anyone loose 0.5 points every hour :-)

User avatar
the_drag0n
Posts: 217
Joined: Thu Oct 02, 2008 6:48 am
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Re: BarsWF 1.0 network compilation

Post by the_drag0n » Fri Oct 03, 2008 11:01 am

well i think we'll find a way ;)
but before that you'll still have quite sth to code i guess...

kiando
Posts: 64
Joined: Thu Oct 02, 2008 7:30 am
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Re: BarsWF 1.0 network compilation

Post by kiando » Fri Oct 03, 2008 11:09 am

agreed. the credit system is for later.
so, BarsMonster if you need anything tested, tell us.

User avatar
the_drag0n
Posts: 217
Joined: Thu Oct 02, 2008 6:48 am
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Re: BarsWF 1.0 network compilation

Post by the_drag0n » Fri Oct 03, 2008 11:18 am

we might konsider using irc so we wont spam the board to much ;)

User avatar
BarsMonster
Site Admin
Posts: 1118
Joined: Wed Oct 01, 2008 7:58 pm
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Re: BarsWF 1.0 network compilation

Post by BarsMonster » Fri Oct 03, 2008 3:01 pm

the_drag0n wrote:we might konsider using irc so we wont spam the board to much ;)
Forum have better memory :-)

User avatar
the_drag0n
Posts: 217
Joined: Thu Oct 02, 2008 6:48 am
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Re: BarsWF 1.0 network compilation

Post by the_drag0n » Fri Oct 03, 2008 3:01 pm

my log has quite a good mem as well ^^

User avatar
styxx
Posts: 5
Joined: Sat Oct 04, 2008 9:56 am
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Re: BarsWF 1.0 network compilation

Post by styxx » Sat Oct 04, 2008 10:59 am

You might get 0.07 points per hour, but this might look frustrating. Probably we will scale it a bit, so that 1teraFLOP would mean 100 points
I believe this is the best idea:
Gain 1 Points for 100 teraFLOPs AND Gain 0.1 Points for one hour running the software on one client.
(any additional clients will gain only Points for teraFLOPs, not for running the software, to prevent abuse)
(the amount of points is only an example and needs of course a lot of testing.)
kiando wrote:agreed. the credit system is for later.
Well said. If we work out any detail that should be included in the software now, there won't be any software ;)

DarkPrince
Posts: 148
Joined: Wed Oct 15, 2008 5:50 pm
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Re: BarsWF 1.0 network compilation

Post by DarkPrince » Sun Oct 26, 2008 7:17 am

and if you computer is then one that break the hash you get a few extra points, its like a lottery. :joy:

Post Reply
[phpBB Debug] PHP Warning: in file [ROOT]/vendor/twig/twig/lib/Twig/Extension/Core.php on line 1266: count(): Parameter must be an array or an object that implements Countable

Who is online

Users browsing this forum: No registered users and 1 guest