last day (15 days later) » 

23:16
0
A: c++ <queue> memory leak?

andrereplace: queue<Graphnode> *preload1 = new queue<Graphnode>; queue<Graphnode> *preload2 = new queue<Graphnode>; with: queue<Graphnode> preload1, preload2; There really is no need to allocate on the heap. try to do the same with slaveQueue and if possible slavero...

so I don't need the delete either?
will it free my memory if I don't use new and delete and just put it on my stack? This is the code in main
@weeo, yes andre posts better use of queue than yours, you don't new then you don't delete. I profiled your code, no memory leaks found. But you really don't need pointer here as you actually copy objects around.
As you use linux, could you try valgrind --leak-check=full program to profile memory issue?
@billz Does it work on MPI programs?
I am not sure MPI, you didn't mention it in your question.
@billz SO I should try not to use new and delete in the same function?
23:16
You don't gain any benefit from new/delete but more complicated code to maintain
@billz After I did what you said, it doesn't help, it is still increasing constantly. I tried to comment dfs function call, it wouldn't increase, maintaining 100M all the time. Any suggestion?
can you show me dfs function code?
also you make unnecessary copy to dfs function
void dfs(Graphnode _root,std::tr1::array<unsigned char, 16> goal,unsigned char _totalDepth,queue<Graphnode> _preload1, queue<Graphnode>_preload2,std::fstream &_checkfile)
can be:
void dfs(const Graphnode& _root,
const std::tr1::array<unsigned char, 16> &goal,
unsigned char _totalDepth,
const queue<Graphnode>& _preload1,
const queue<Graphnode>& _preload2,
std::fstream &_checkfile)
pass them by reference to elide unnecessary copy
So it will not create any copy by using constant?
Thank you very much first!!
it elide copy because pass by reference &
void dfs(Graphnode _root,std::tr1::array<unsigned char, 16> goal,unsigned char _totalDepth,queue<Graphnode> _preload1, queue<Graphnode>_preload2,std::fstream &_checkfile)
{
//printf("preload1 %d,preload2 %d\n",(int)_preload1.size(),(int)_preload2.size());
/**
*This is the search algorithm done by slave nodes. It takes starting nodes from the queue generated by the master
*node doing breadth-first search. Each slave nodes get portion of starting nodes. The number of nodes each slave
*node get is depending on how many slave nodes. the queue generated by master node is devided equally to slave
what I did in dfs is just read preload1 and preload2
pop them into the hashset
while(!_preload2.empty())
    {
        _closelist.insert(_preload2.front().state);
        _preload2.pop();

    }
so that's what I did in the dfs function
23:22
that is fine
_closelist is local variable, it's released when dfs returns
yes I commentted dfs function call
and it will always maintain the memory at 100M
memory will never increase again
so I think it has something to do with my way of calling the dfs function
it's not really leaking memory
Actually I need the copy of preload1 and preload2 in my dfs function, So I can not use const. Because the modification in the dfs function for preload1/2 should not affect my globle preload1/2 var
because you pass parameters by value, your program is copying a lot queues
But as the function returns it should free all of them?
23:27
yeah
all the var in dfs are local, it doesn't modify any global variable
pass _preload1 and _preload2 by reference, not const reference
Ok let me try
void dfs(const Graphnode& _root,
const std::tr1::array<unsigned char, 16> &goal,
unsigned char _totalDepth,
queue<Graphnode>& _preload1,
queue<Graphnode>& _preload2,
std::fstream &_checkfile)
But if I pass them as reference they will modify the global var, I don't want that happen
23:29
I see what you mean.
_preload1/2 will be pop()ed in dfs function
I see what you mean
So it will modify the global var
I think you are right..there's something wrong when I call the function
But as local function, after return, all memory should be freed...but it's not....
do you have to use queue?
yes
I need to implement the breadth-first search that kind of thing
so stack/queue is must.....
23:35
how about vector?
you can still go through each element with random access
and it doesn't need to move any elemnts
I need a first-in last-out container
when I don't have the preload thing implement, the memory is ok-constant taken
when taking the idea of preload, it is increasing
so I think it should be the way I pass the preload queue
hmm, I can't help much. you need to use valgrind to profile it
Can I use that on MPI jobs?
It is weird...
I am not sure, I never used MPI before
local var should be freed after each iteration
right?
23:37
yes
let me try dfs function with valgrind
Yes, it shouldn't be the problem in dfs
it should be the head of dfs or the function call
because if I comment the dfs line in my main
memory doesn't increase
So I checked the queue pop() function will free the memory it take
So every thing seems to make sense...but it just couldn't free the memory..
let me try dfs function with valgrind
thank you very much...
hmm, valgrind doesn't detect any leaks
==11620==
==11620== HEAP SUMMARY:
==11620== in use at exit: 0 bytes in 0 blocks
==11620== total heap usage: 1,182 allocs, 1,182 frees, 579,711 bytes allocated
==11620==
==11620== All heap blocks were freed -- no leaks are possible
==11620==
==11620== For counts of detected and suppressed errors, rerun with: -v
==11620== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
so it should be how i call the function?
it's really weird.......
even though dfs has memory leak, it shouldn't affect since it is local function...right?
23:49
yeah
how long do you run your code, how much do you think it leaks
I wonder if i can use valgrind on mac?
do you know top doesnt' report real memory usage?
I think yes you can use valgrind on mac
wow.....
but after my memory is gone
it starts page in
so I assume it doesn't free my memory?
My code used to free the memory..but after I put preload there. It's increasing 10M every second on top
it should stabilize after a while
I also run it on the cluster. It will increase to take memory forever
So I think it wouldn't stabilize.....
the cluster has 128G memory
23:52
hmm, that's strange. you may leak it somewhere.
it doesn't leak when I comment the dfs call......
you may need to profile your own code
thank you very much!!!!!!!
really
maybe it doesn't trigger something else?
I appreciate !!
23:53
you are welcome
I've been working on it for 2 days...
bye
narrow down to the function call....
but then...no idea any more
valgrind should help you
thank you!
have a good night!

last day (15 days later) »