last day (15 days later) » 

19:25
0
A: programming issue with openmp

pyStarterIn case of an parallelization using OpenMP, you will need to specify the number of threads your program is to use. You can do so by using the environment variable OMP_NUM_THREADS, e.g. calling your program by means of OMP_NUM_THREADS=5 ./myprogram to execute it using 5 threads. Alternatively,...

@Rain That is indeed possible. You should watch out for the result of omp_get_num_procs(). Afaik, you should only use this command within a omp-environment if you'd like to obtain reliable results.
@Rain If I understand you correctly, you already do set the number of processes (but it does not seem to work)? Have you checked the other two hints in my posting?
I set the number of processes as modified in the main context, and checked that when the program is running, the system monitor shows 4 threads being used by the program
@Rain I assume you only have 4 CPUs? The problem with using the command the way you do is that you never actually assign a number of threads to your program.
I have 2 CPUS on my laptop. Intel has this hyperthreading technique which will split it into 4. I don't understand your answer as to my problem with programming. Is it that the do-loop is not split evenly to the four threads?
@Rain Thought so.. you should not use the command omp_get_num_procs() and set the number of threads explicitly to 2 (everything else will jam your CPUs). Avoid setting the number of threads the way you do it right now. A non-equal splitting of the do-loop is additionally possibly, that is a matter of optionally using schedule(...).
19:25
I solved the issue of data accuracy, turns out I forgot to declare private variables for some internal parameters... The program is still quite slow as compared to serial code. I suspect it has to do with putting the parallel code in a subroutine that is called frequently. What do you think?
@Rain I'd need to see some more code to be able to answer this. Maybe start a separate/new question on this? As soon as you achieve 20 reputation, we can also enter a chat and discuss this issue in detail.
There we go...
Hi first of all thanks for the help!
I pushed your existing questions for you to reach the mark of 20.. ;)
sure no problem
So now everything is up and running with the number of threads you choose?
yes, it is a Quantum Monte Carlo simulation
ok
so you have a loop and call a subroutine within?
19:27
the subroutine which I placed the parallel code in is very low-level
but still Fortran?
in the sense that it is called about ~1000 times per Monte carlo iteration
yes still Fortran
is it pure?
yes, inside the parallel code, is matmul(A,B)
hm.. and all the variables within are declared to be "private"?
19:29
no. A,B are matrices that also carry indices of the outer loop
so there's no need to declare private
for example: do i=1,N, matmul(A(i),B(i))
..and the results are just like if you run the program sequential?
only that the program seems to be slower (than if run sequential)? -- this is what I understand so far
hi Alex :)
I am not sure yet. Each MC iteration takes up like 40seconds with two threads, while 30 with one thread
Hi
I suspect that the results will be the same though
parallel as compared to serial
if the results would deviate this would give a hint toward unexpectedly shared variables
good point, just a sec
19:33
how large are matrices btw?
some 64x64
I get the code running some 20 iterations to see if the results are consistent with the serial one
How large are deviations? Keep in mind that running code through OpenMP is non-deterministic - it's not executed in the exact order the serial code would. In floating point arithmetics this will lead to deviations in the last few digits...
Alex, I made sure that the parallel code are pure functions, in the sense that non depend on the other
feel free to do whatever for the moment, I'll post the results in 2min
Well, that's not exactly my point... even a simple do loop for 1...N would be executed in "random" order!
kk
good point, Alex^^
19:38
I'm still confused. why would the "randomness" affect the numerical accuracy?
Consider a matrix-vector multiplication.. there are 2*N elements to be multiplied and added up.
Now, since we are on a binary system and may use floating point numbers, it actually does matter in which order we add up the N resulting product.
*products
If you add two floating point numbers, the result is rounded to the next number that can be represented by a float again. So, due to different rounding operations taking place, A + (B+C) is not (A+B) +C
Not exactly, at least ;-)
Ah, you misunderstood my point. when I say do i=1,N C(i)=matmul(A(i),B(i)), I am not adding up ,say, C(i)+C(j)
19:43
Sure, but you multiply A(i) and B(i) (how are these matrices, btw?) which feature columns/rows that are multiplied and summed up to yield C = A*B.
let's say real A(1)={1,2;3,4}, B(1)={3,2;4,1}. then C(1)=A(1)*B(1)
I understand openmp as putting this operation into one thread,
So... Your multiplying scalars?
no, C(1) is a 2x2 matrix
so C(1)(i,j) is a scalar
aha.. :) that may be what is causing the slow-ness..
How would you distribute a 2x2 matrix across 4 threads..? ;-)
I'm confused how computing C(i) in random sequence could result in difference as compared to computing C(1) C(2) C(3)...
oh it is not distributed
it is within one thread only
the 4 threads deals with C(1) C(2) C(3) C(4) respectively
each thread calls the function matmul to compute a matrix product
19:50
Hm.. I am not sure if matmul will stick to using only the thread that called it. Alex, do you know more on this?
So, before I get totally lost... You have 3D matrices A, B, C, and want to perform
yes
do i=1,N ; C(:,:,i)= matmul ( A(:,:,i) , B(:,:, i) )
enddo
yes
Ah, I see... And your shapes are 2x2xN?
19:51
yes
sorry for the confusion
@pyStarter I'm not sure about that, but I would think so... I'll try it out.
btw the results for serial and parallel are out. The output data are consistent (physically, I am a physicist!). However the time cost for parallel code is 10min for 20iterations, while 6.39 for the serial code
wait a sec...
sorry
How do you measure the time?
that's my point...
let me check
Damn, jjust a second faster ;-)
19:54
Hehe ;-)
I used call cpu_time(clock_start); call cpu_time(clock_finish), time=clock_finish-clock_start
Hehe..
yup
And we have a winner ;-)
Try using omp_get_wtime()..
that will not sum up the time of all processes.. ;)
with that regard you have approx. 10s*0,6*0,6=3,6s vs 6,4s
Or, simpler, use system_clock()
19:58
^^
Aha
Thans guys!
no problem :)
Don't forget to close your question (accept the answer).. that'd be more I can ask for ;)
sure I'll edit it ;)
20:00
Awesome
I'm off... Good night (or whatever timezone you're on)
Yup.. that'd be Europe here ;) Good night
Goodnight ;)

last day (15 days later) »