last day (14 days later) » 

2:40 AM
At long last... Erlang has a room on SO!
Not that its much of an achievement...
 
Hi zxq9
 
Hi there.
Are you wondering how to block on finalization of your signal cluster's job with a timeout condition?
 
I was thinking of that but I felt that should be a last resort. I wonder if there are events send by the Erlang scheduler to notify if the children are idle( and no more messages in the message queue)
 
Its not really a message queue issue, though.
In Erlang anything can send anyone a message, at any time.
Without that sane clustering would be impossible.
How are you to know if the messages you are checking for and seeing are the "right" ones?
Instead, use a reference block to guarantee that you will only respond to receipt of a message that carries that reference.
Check the code of b_manager/1 here again: stackoverflow.com/questions/27504378/…
The call to make_ref() returns a unique identifier, which is guaranteed to be unique across the cluster. The next thing the b_manager does is wait for a message carrying that reference (which is why we go to the trouble of passing it around within the B signal cluster).
 
I was going to implement the reference block that you had earlier suggested(and its a great answer, a blog post of its own) but the problem I thought it would not solve is this: a) b_manager will 'unblock' and accept the next message from its client only when it get's a message carrying that reference. But what if there are multiple messages from its children? Then b_manager/1 should ideally wait for all messages from it's children.
In the current b_manager/1, it accepts the next message as soon it gets the first messages from its children
also what if the children don't send any message at all?
 
2:55 AM
Right. So we could make multiple "modes" for b_manager (fsm, right?)
How many responses from the B signal cluster are supposed to be valid?
Or we have no way of knowing?
in other words, what is the criteria for signal processing response validity?
 
perhaps, I dont understand the validity question. all messages send to b_manager are valid
i.e. from it's children processes
b_manager is a fsm with two states. busy/open
 
You are saying that the b_manager may receive multiple responses from the children, and more than one may be valid?
 
yes, because it is technically possible for a child to send multiple events out to the world. b_manager will know who in the world to send it to
 
So a signal split is valid.
If that is the case, then you are multiplexing, and this should be another layer in your signal path.
 
I actually don't know signal processing terminology :). just give me a second to read about them
Let me send you what I am trying to achieve
one sec
 
3:00 AM
I mean, you might split a signal, and that's OK -- but it is an independent task within the system.
 
I am trying to implement an event reactive flow based programming model
Can you see the diagram I sent you?
 
Yep.
 
That is a dataflow diagram. I will be adapting it to be reactive
so in my case each box is a fsm or a composite which may contain other composites or fsms
 
Sensible.
 
It's not my idea. I learned this from a master programmer and I am just trying to understand it by implementing it
 
3:04 AM
So you are wondering how to combine the signals (collate) and then split again at the other end?
In any case, you will have to declare a complete success and a complete failure condition for each step along the processing path.
 
So, in the box 'Process merged stream', there are two output ports. Imagine that talks not to the two boxes but to a parent process(the .png file is the composite :))
 
That's what I was getting at with "validity criteria". If you don't know how many responses a manager might get from its workers, then you will not be able to set a timeout or any condition that represents completion of processing.
You must be able to tell some manager somewhere that "processing is complete" or "processing completely failed".
The details of collating/splitting are trivial. Its figuring out what success and failure mean that is important here.
Since we don't know the details of the problem, we can abstract that simply by saying "receive {success, Ref, Result} -> yay(); {failed, Ref} -> oh_no() after Timeout -> oh_no()"
or something along those lines.
Below that, in a subordinate process, is where you split or merge the message.
That's why you see so many Erlang stdlib functions return things like {ok, Foo} or {error, Reason} so that you can write an assertion that will crash your process if the result isn't "{ok, _}"
 
Error handling in FBP is done by a :error port
 
If all the processes in a signal processing cluster are linked -- a crash in one kills them all. Any process monitoring them can interpret that as complete failure.
 
I think you are two steps ahead of me. I haven't considered error yet in my mapping of FBP to Erlang
 
3:12 AM
If you are worried about blocking being sort of a funky fit for this, then the common reaction in Erlang is to spawn a process (or cluster of processes) per job and let them die natural deaths when they complete their task.
 
Ya, I was considering that...
 
Errors are easy in Erlang.
The monitor spits out an error report when something its watching dies. That's all there is to it, actually.
Its a lot easier than doing massive "try .. catch .. finally"
 
I know :). My questions are more related to normal functioning(I was going to add the erlang error processing gradually)
 
everywhere, in every possible place your code might die.
I would just write up a variation of the b_manager &co. code I put in the previous answer, but make it spawn a cluster per job.
And play with it from there.
 
There is a similarish implementation called fractalide.com which combines erlang with event based FBP
erlang error handling
 
3:17 AM
Its pretty easy to move between synch/asynch and pool/independent/singleton type workers without changing much code. So you can experiment with different approaches.
 
yes, I was thinking of that (spawn cluster per job) but I thought of asking you and the community first
ok, I will try that approach
Thank you so much for taking the time
 
The standard conversation about that goes like this:
"What did you try?"
"Nothing yet, I wasn't sure about performance/behavior/etc."
"You'll never know until you write something."
And... that's about it.
Genuinely concurrent systems are not easy places to predict emergent behaviors. And performance, resource usage, even simple message ordering in a 3-way system are all emergent behaviors in Erlang.
Luckily, Erlang is pretty easy to hack around on, so its not hard to try 5 or 6 different implementations of something in a few hours.
 
You are right. I'll hack something up and reach out
 
The Fractalide thing is sort of funny... its based on a very similar premise to what I'm doing with business data (we're ultimately trying to avoid ever having processing occur at the data center, or holding any transparent customer bits).
Btw, here is an example of a linked, spawned process used to represent a "hand" moving something from the game environment to inventory, or vice versa: github.com/zxq9/erlmud/blob/master/erlmud-0.1/…
The details of the tuples aren't important, The "H" and "R" in HPid/HName stand for "Holder" and "Recipient". The hand function is called by the functions take, drop and give.
Its one example of using a "third process" that acts as a processing passthrough (not unlike your signal clusters) to avoid the case where the Holder and the Recipient are the same type of process, and would hit deadlock if they both sent the same synchronous message to each other at the same time.
Btw, the v0.1 version of that codebase is all raw Erlang, so don't use it as an example of OTP best practices -- that is what the rest of the project is for later on.
 
Cool, I think I'll read it slowly (as I am still new to Erlang syntax) and get back to you if I have any questions. I don't understand the application context.. game environment and inventory seems to me be two diferrent domains.. Is this the code for your business data?
Alright Craig, it was great talking to you. I have to head to bed
if you ever come to Toronto, do let me know. I would love to meet up.
 
3:36 AM
Stay in touch.
I'm curious about your use case. Let me know how things go: zxq9@zxq9.com
 
Sure,will do
 

last day (14 days later) »