last day (19 days later) » 

4:30 PM
0
A: executing a code of webservice in another child process in node.js

kailniris fork some workers at the begining of the code in your code write seperate worker tasks and master tasks 'if (cluster.isMaster) {...} else {...}' send the data which needs to be processed to the worker with IPC messaging or redis pub/sub messaging, after the worker processed the data it can send ...

 
Kailniris, Thankyou for the explanation, can you please provide some example like how to fork some workers in a webservice, and write separate workers task or which you have stated in your third point "you can also create 1 worker which handles http requests and 3 workers which processes data eg. if you have 4 processor cores", any example will give me more clear picture. Thankyou
 
i will, but keep in mind its a bit overcomplicated approach and without redis, its even more complicated, do you consider using redis or simply IPC messaging?
 
I have considerd using redis but not at this stage, I am very much new to redis and IPC and in this clustering/forking. It would be nice if you could provide me with an example. Thankyou once again :)
Kailniris one more thing, does processing of a data in a for loop really blocks the event loop? what if I send multiple requests will it not work?
 
well in asynchronous javascript environment especially in node.js doing something not I/O will block the event loop. like doing something like array.push('asd') will block the event loop for very small amount of time you won't notice but if you create a for loop or any kind of loop which will do array.push('asd') lots of time i mean thousand of billions of times the app basically will freeze busy doing the foor loop. In node.js this will basicaly block all I/O so all the request will time out. In client no user interaction or other javascript will work.
so no, no multiple request can work parallal if you use sync functions like JSON.parse like in your example. JSON.parse is pretty fast in V8 but iterate over 2000 json objects can be time consuming. This does not mean loops are inherited bad you just have to be careful how much data you process with it. In large data sets instead of loops you can use recursive function , each function called with setImidiate so each function will be put after the next event loop IO. It will be slower but will not block the event loop so multiple request can be processed simultaniously
 
Thanks Kailniris. What i do currently I do JSON.parse for just one time only, actually its an array of objects and then I pass that array have objects in async.eachSeries()
but yes iterate over 2000 will be a time consuming will consider using recursion
 
 
3 hours later…
7:20 PM
Did you meassured time? Can you provide your code? I will build cluster model around it.
 
8:05 PM
yes I can provide my code
should I post here? its a huge one
or anyother way?
time for 1500 was 2 minutes and 40 seconds
should I send you a text file with that webservice? or anyother communication medium?
 

  last day (19 days later) »