CUDA Kernel Error: invalid device pointer

[nubcake]

Hello everyone,

please apologize my bad english (since i'm german and still learning).

i encountered this strange problem from one day to another..
First barswf(0.8 cuda32) was running fine on my machine with 2 gpus (8800gt + 8800gts @ C2D 2,66), but right now it just dies and gives me that error.
The only thing i did (which i can remember) was changing the video drivers from 179.x[CUDA] to 180.x[CUDA] and then the error occured for the first time.
So i thought i should just uninstall the new (180.x[CUDA]) drivers and reinstall the old (179.x[CUDA]) ones to get it back working.
But yeah.. it didn't work, that's why i'm asking you guys for help if possible.
//edit:
System Spec's:
Intel Core2Duo E6700 2,66ghz
2gb ddr2-ram (800mhz fsb)
Asus P5N32-E SLI
XFX nVidia Geforce 8800GT 512mb
XFX nVidia Geforce 8800GTS 640mb
running Windows Vista Home Premium32
nVidia Cuda Driver 179.x
(did i miss something?)

thanks in advance

nubcake

//pic attached

[nubcake]

*bump*
anyone please?
it would be awesome to get my 2 graphic cards running again
thanks in advance

[BarsMonster]

That's a known issue, and I hope CUDA 2.1 would be able to solve it.
In the worst case there will be -gpu_mask command line option to disable on of GPUs.

[nubcake]

so you would suggest me to upgrade to cuda 2.1?
i'll try that one later on today

thanks for your help

[BarsMonster]

so you would suggest me to upgrade to cuda 2.1?
i'll try that one later on today

thanks for your help

Well, I guess recompiling BarsWF against 2.1 would give better chances, but still worth trying to replace DLLs from the barswf distribution with the 2.1 ones.
Let us know if it would fix the problem.

[nubcake]

Well.. after playing with different dll's from different CUDA versions, i somehow figured, that the problem only occurs, if i have physX enabled on my 2nd video card (8800gt) in the nVidia control panel.
If i disable physX completely, it is working fine, but i have only 1 gpu showing up in barswf.
If i turn physX on for my 1st video card (8800gts), it's working fine, but i still have only 1 gpu showing up in barswf.
So the dll switching doesn't really get me a solution.. but playing around with physX settings doesn't get me a solution either :/

[d0PING]

Michail:why are you using limitations on max. working cores(for nvidia)???Max-80!!!That's to small value for me..., hadn't you seen PC case for 40 or higher amount of PCex slots!!!

[BarsMonster]

Michail:why are you using limitations on max. working cores(for nvidia)???Max-80!!!That's to small value for me..., hadn't you seen PC case for 40 or higher amount of PCex slots!!!

No, limit is higher

[d0PING]

GetSystemInfo()?-is this how it knoes how much proc cores!!!Stupid API function.now i got why it shows 20 proc. cores on my northwood.
sofe API was prepatched.what about cpuid?

[d0PING]

>>>>>No, limit is higher
00439AF8 /$ 83EC 0C sub esp , 0Ch
00439AFB |. 68 D8484700 push 004748D8h
00439B00 |. E8 211B0000 call 0043B626h ; <JMP.&cudart.cudaGetDeviceCount>
00439B05 |. A1 D8484700 mov eax , dword ptr [004748D8h]
00439B0A |. BA 80000000 mov edx , 00000080h
00439B0F |. 3D 80000000 cmp eax , 00000080h<-------
00439B14 |. 0F4FC2 cmovnle eax , edx
00439B17 |. A3 D8484700 mov dword ptr [004748D8h] , eax
00439B1C |. 85C0 test eax , eax
00439B1E |. 7E 14 jle 00439B34h ; bars.00439B34
00439B20 |. 33C9 xor ecx , ecx

[d0PING]

and some question on realization, please:
how you're playing without drivers 5 sec. shit, cause i'm playing(patching) nvidia's driver->>> it (your prog) works forever-while "reset" Some timers?But how without perfomance falling.
pass ranges and theirs check!!!How...(checks) without perfomance falling?

[d0PING]

>>>>>No, limit is higher
00439AF8 /$ 83EC 0C sub esp , 0Ch
00439AFB |. 68 D8484700 push 004748D8h
00439B00 |. E8 211B0000 call 0043B626h ; <JMP.&cudart.cudaGetDeviceCount>
00439B05 |. A1 D8484700 mov eax , dword ptr [004748D8h]
00439B0A |. BA 80000000 mov edx , 00000080h
00439B0F |. 3D 80000000 cmp eax , 00000080h<-------
00439B14 |. 0F4FC2 cmovnle eax , edx
00439B17 |. A3 D8484700 mov dword ptr [004748D8h] , eax
00439B1C |. 85C0 test eax , eax
00439B1E |. 7E 14 jle 00439B34h ; bars.00439B34
00439B20 |. 33C9 xor ecx , ecx

at the end of working day my mind in outer space: dont be afr. :80h-128 signed short.

[BarsMonster]

GetSystemInfo()?-is this how it knoes how much proc cores!!!Stupid API function.now i got why it shows 20 proc. cores on my northwood.
sofe API was prepatched.what about cpuid?

Well, hopefully you will be able to see any number of cores via -num_cpu 99

[Sc00bz]

So why is there a limit when you can just dynamically allocate enough threads?

[BarsMonster]

So why is there a limit when you can just dynamically allocate enough threads?

I do dynamically allocate most of stuff I need. Only few things left which are static.
With the amount of refactoring I do every week, this static stuff gonna die soon

[d0PING]

>>>So why is there a limit when you can just dynamically allocate enough threads?
what limits are you talking about, cuda kernel execution?
I guess it's obviously what in good code(without high memory needs) it's better to have number of working threads the same as shaders processors for (ATI and CUDA) and the same for CPU for cpu threads, isn't? but with high memory occupation it's more interesting, better got higher threads number than max SP (due to fetching for MC, from MC to mem. chips, back, and to the thread what it needs).
but with new cards with new sp coutns it should be dynamic (i guess no need to explain) maybe put some check on cudaGetDeviceProperties() for sp count check, or in some cases make some check on MHashes/s per thread changing threads count?

[BarsMonster]

>>>So why is there a limit when you can just dynamically allocate enough threads?
what limits are you talking about, cuda kernel execution?
I guess it's obviously what in good code(without high memory needs) it's better to have number of working threads the same as shaders processors for (ATI and CUDA) and the same for CPU for cpu threads, isn't? but with high memory occupation it's more interesting, better got higher threads number than max SP (due to fetching for MC, from MC to mem. chips, back, and to the thread what it needs).
but with new cards with new sp coutns it should be dynamic (i guess no need to explain) maybe put some check on cudaGetDeviceProperties() for sp count check, or in some cases make some check on MHashes/s per thread changing threads count?

He was talking about CPU threads dedicated to each GPU.
Number of threads inside GPU is determined at runtime in some sub-optimal way based on cudaGetDeviceProperties.

[d0PING]

>>>Number of threads inside GPU is determined at runtime in some sub-optimal way based on cudaGetDeviceProperties.
right choise,had you experimented with gpu threads, was that always equival. count of threads and sp of gpu shown hightest perfomance?
Cause in theory it has to be so, but what in practise?on my job stuff it's not always so...
and you implementation of gpu kernel:are you running gpu kernel on one flash (with all sp count threads, comparing and returning to cpu code) or for some time, some password part... ----> how are you playing with watch dog?

[BarsMonster]

>>>Number of threads inside GPU is determined at runtime in some sub-optimal way based on cudaGetDeviceProperties.
right choise,had you experimented with gpu threads, was that always equival. count of threads and sp of gpu shown hightest perfomance?
Cause in theory it has to be so, but what in practise?on my job stuff it's not always so...
and you implementation of gpu kernel:are you running gpu kernel on one flash (with all sp count threads, comparing and returning to cpu code) or for some time, some password part... ----> how are you playing with watch dog?

Of course work is divided on small subtasks, and I can send right amount of work to each GPU so that kernel works close to gpu_time milliseconds.

[d0PING]

>>>>Of course work is divided on small subtasks, and I can send right amount of work to each GPU so that kernel works close to gpu_time milliseconds.
I know -on small subtask, but how exactly small (just one pass->one thread for sp multiplied on sp count and returning to cpu code) or (firstly found how much it takes: test one pass.execution time after divide miliseconds from gpu_time on this value and find how much pass could be checked before watchdog ) or there's hardly written pass ranges (so small what it won't touch watchdog) and after theirs check returning to cpu code???
moreover on small password count it takes more time to run kernel on gpu than doing that on cpu, so i guess there's no sense to create any kernel for 4-(maybe)5 long pass. better just run them on cpu thread.
what about any dividing pass for multiply gpus: divide them in hard-written mode or depending on it's speed(sp count)...
p.s.:too much

[BarsMonster]

>>>>Of course work is divided on small subtasks, and I can send right amount of work to each GPU so that kernel works close to gpu_time milliseconds.
I know -on small subtask, but how exactly small (just one pass->one thread for sp multiplied on sp count and returning to cpu code) or (firstly found how much it takes: test one pass.execution time after divide miliseconds from gpu_time on this value and find how much pass could be checked before watchdog ) or there's hardly written pass ranges (so small what it won't touch watchdog) and after theirs check returning to cpu code???
moreover on small password count it takes more time to run kernel on gpu than doing that on cpu, so i guess there's no sense to create any kernel for 4-(maybe)5 long pass. better just run them on cpu thread.
what about any dividing pass for multiply gpus: divide them in hard-written mode or depending on it's speed(sp count)...
p.s.:too much

"firstly found how much it takes: test one pass.execution time after divide miliseconds from gpu_time on this value and find how much pass could be checked before watchdog" - that one.
But not till watchdog, but till gpu_time parameter, which defaults to 500ms.

[d0PING]

...and after that you're multiplying this(max pass. count per sp untill deadline) and sp count of gpu core if possible multiplying numbers of gpu cores (for dual core it's easier?), for multygpu you must do that of time gpus count???? ---and that's yours range for only gpu;
meanwhile it has finished and return to cpu code into superwisor thread(not doing sse2), working cpu thread still working
or i have sequential password ranges cpu then gpu, cpu then -gpu,.....,while found
or cpu.....,gpu..(till deadline),gpu(till deadline)...

[BarsMonster]

...and after that you're multiplying this(max pass. count per sp untill deadline) and sp count of gpu core if possible multiplying numbers of gpu cores (for dual core it's easier?), for multygpu you must do that of time gpus count???? ---and that's yours range for only gpu;
meanwhile it has finished and return to cpu code into superwisor thread(not doing sse2), working cpu thread still working
or i have sequential password ranges cpu then gpu, cpu then -gpu,.....,while found
or cpu.....,gpu..(till deadline),gpu(till deadline)...

I do benchmark for each GPU core. For CPU it's not that important, so there is fixed amount of work.
There are sequential ranges served asynchronously for any core finished their work.

[d0PING]

man, that's the biggest sse2 code in one function i'd ever seen ( with pc gamess as the exeption)
i cann't even believe everything that's could work on so high speed.on my northwood it shows ~11-11,5 mh/s@3,8GHZ&HTT(aka Prescott).
...core 2duo with its 5 decoders really works fast...
so much code on one page; is that realy less eq. than code L1?
competition what's that:

00438F69 |. 66:0F6FC4 |||movdqa xmm0 , xmm4
00438F6D |. 66:0F72F4 06 |||pslld xmm4 , 06h
00438F72 |. 66:0F72D0 1A |||psrld xmm0 , 1Ah
00438F77 |. 66:0FEBE0 |||por xmm4 , xmm0

[BarsMonster]

man, that's the biggest sse2 code in one function i'd ever seen ( with pc gamess as the exeption)
i cann't even believe everything that's could work on so high speed.on my northwood it shows ~11-11,5 mh/s@3,8GHZ&HTT(aka Prescott).
...core 2duo with its 5 decoders really works fast...
so much code on one page; is that realy less eq. than code L1?
competition what's that:

00438F69 |. 66:0F6FC4 |||movdqa xmm0 , xmm4
00438F6D |. 66:0F72F4 06 |||pslld xmm4 , 06h
00438F72 |. 66:0F72D0 1A |||psrld xmm0 , 1Ah
00438F77 |. 66:0FEBE0 |||por xmm4 , xmm0

rotate?