« first day (1725 days earlier)      last day (3223 days later) » 

9:00 PM
so the server times out the client when it hasn't answered the heartbeats in a given time
Probably formulated that wrong, sorry
 
@BenjaminGruenbaum Because we have multiple sources of data in multiple threads, all of which write into a single queue. Can't use multiple queues because they don't support select
 
var socket = require('socket.io')({
  // options go here, like heartbeatTimeout (check property naming on server)
});
!!afk 5pm
 
is it the same way there? heartbeatTimeout = value?
 
@copy what does "they don't support select" even mean?
 
@BenjaminGruenbaum As in go
Blocking read a value from the first queue (of multiple) that gets one
 
9:03 PM
@copy ah. "select as in go" explains a lot.
@Zirak you should have probably started with that :D
 
user1596138
@SomeGuy have any of you read the Metro books? There's games based on them... Might interest some people. I liked them
 
the FBI raided Jared(subway guy)'s home. It might be connected to child pornography. Let's all speculate.
 
I will hit you with sharp things
 
Async iterators make sense here, you just get a promise for the next value for the type.
 
9:03 PM
@NickDugger zip zop zoobity bop
 
@BasvanderPloeg Welcome to the JavaScript chat! Please review the room pseudo-rules. Please don't ask if you can ask or if anyone's around; just ask your question, and if anyone's free and interested they'll help.
 
@BasvanderPloeg Nice name
 
How do I do this
It's retarded but hopefully you get my point - I need to listen to two completely separate data sources at the same time
 
9:07 PM
awesome doc, @MadaraUchiha
 
@Zirak I still like my original suggestion
 
@copy Remind me again please?
 
Any way of calling a function when a socket/client is dropped serverside with socket.io? for example when it times out?
 
@Zirak Write everything into a single queue that yields (type, value) pairs, and implement your abstractions on that
 
I have a question but its not ensert yet...
:(
 
9:08 PM
@copy That's what the first code block is, isn't it?
 
@Zirak so just an event emitter?
 
Each of them (map on one type, map on multiple types, filter, etc.) is only a few lines of code
 
> I want to be able to play with potentially several input sources at once (mix and match, like having one wait for another to appear before doing some more merging and filtering, potentially arbitrary complexity of processing events based on other events).
^ ^ ^
 
So async iterator
 
This is why I implemented leaned over to observables instead of event machines
 
9:09 PM
queue.on("type") // returns a promise
 
6 mins ago, by Zirak
I will hit you with sharp things
 
Since everything is a promise, you can mix and match easily and so on.
 
If your diet was a promise, would you resolve it, or reject it?
 
How is it fundamentally different than writing thin observables like I did?
 
Because it would be a lot less work to build and it would compose better with native language features.
But fundamentally, an observable would also be OK here, observables are just really hard to get right.
 
9:11 PM
@Zirak lol
 
But you probably don't even need flat_map so there's that.
 
Or rather, a stupid Rx.Source-ish thing with a next and subscribe
(but is subscribe sync or async!? shoots self in head)
 
@Zirak clearly, we should just listen to you since you know all the answers :D
 
well not to good :p I have a question but its not ensert yet...
:( may i post a link of the question?
 
@rlemon Any way of calling a function when a socket/client is dropped serverside with socket.io? for example when it times out? Not always the disconnect listener is triggered when a socket "dies"
 
9:13 PM
@Zirak that'd be a Subject.
 
@KasperRT rlemon is afk: 5pm
 
@CapricaSix ye, just messaged so he'll get it when he returns
 
Wait, no, I lost it. How will continuously returning Futures solve things? It's still data over time
 
(I have a related question when this debate is over)
 
@Zirak it's async iteration, next returns a promise for a value.
 
9:14 PM
(keep going, I'm reading and learning)
 
@SomeKittens you can ask it's ok
 
@BenjaminGruenbaum Can you write my example magical code assuming this async iteration is implemented?
 
I have a question but its not ensert yet...
:( May I post a link of the question? plz help...
 
ensert? what?
 
answered*
 
9:16 PM
@Zirak depends if the observable is hot or cold! (well, sorta)
Took me a sec to realize I was reading Python and not ES6
 
@BasvanderPloeg you already asked like 10 lines above
 
what the shit does javascript:void(0) do?
 
@BenjaminGruenbaum hrm, I'll think about it, thanks
 
@GarrettKadillak nada
 
never seen that before
 
9:19 PM
Sure
 
Wait will that work?
What's run?
 
@ssube why use it then?
 
1
Q: Get jquery ui sortable data from list after sorting

Bas van der PloegI'm having a problem with getting the sorted data form the id="#playlist_save". id ="#playlist_save" starts with an empty <ol></ol> list. Then the user can sort the items they need form: <ol class="simple_with_animation vertical"> $videos </ol> To the list: <div class="group"> <ol...

 
No no no, you're trying to cheat me
 
@GarrettKadillak to prevent an event from actually doing anything
it's used by links especially so they don't go anywhere
 
9:20 PM
Both yield values.on('money') and yield values.on('drugs') need to run at the same time
 
@Zirak the pump, what you said you know how to write already.
 
Will they?
 
Yes, it's nonblocking, they'll run concurrently not in parallel.
 
@GarrettKadillak though just use preventDefault now
 
9:21 PM
# @BenjaminGruenbaum in other words:
handle_money()
print('hi mom')
# will it immediately print "hi mom"?
 
@BenjaminGruenbaum But reading from a queue is blocking
 
@somekittens javascript:void(0) is old and crusty?
 
@GarrettKadillak yarp
 
@copy right, but it would return a promise and not the value, reading the result of the promise is done via yield and is non-blocking.
 
@GarrettKadillak in some cases.
There are still a fair number where you want links not to do anything.
 
9:22 PM
@Zirak assuming you meant run(handle_money) then it would print "hi mom" before it would run the function.
 
user1596138
@GarrettKadillak It's not Angular 2.0
 
@BenjaminGruenbaum I honestly don't understand how this works
 
But...how can you have two runs at the same time?
 
@jhawins ?
 
@copy let me find you that presentation.
 
9:22 PM
I'm really confused
 
@Zirak it's concurrent, it's an event loop.
 
@Zirak do you know what an ETL system is?
I'm facing almost exactly this problem in another project
 
you're just a step or two ahead of me (as usual)
 
cc @Zirak
 
9:23 PM
It's like you say "sure, so you're having problems running two things at the same time? X solves it", but then like, "I've assumed you can run two things at the same time"
 
@Zirak no, I didn't assume you can run two things at the same time, this is evented concurrency, you have threads that push to the queue, the queue is thread safe, after you're left with nothing but platform code the queue is flushed and so on.
This is, fundamentally what libuv does.
 
lmfao @ this death metal
 
He is doing the weird (yield) thing in the presentation and is losing on a lot of expressiveness for no good reason (missing on bidirectionality where it matters a lot) but it's a good presentation and it's fairly old in internet terms.
 
I'll think about it again
 
9:27 PM
Read the presentation, it's pretty good.
 
I've got a very similar problem, but in Node
 
@SomeKittens what is it?
 
I'm building an ETL system (Extract, Transform, Load). tl;dr, pulls data from points A, A', B, B''', etc, transforms it a bunch, and loads it into point Q.
There are many out there, but none that quite fit my specific needs (Node.js, realtime input)
an ETL is represented as a DAG, with the extract nodes being the roots, and the loads being the terminal nodes
 
Oh, like a feed-forward neural network
 
I can easily hack together some observables to make this happen, but I need a way to track dependencies to build that DAG
@BenjaminGruenbaum I have no idea what that is, but sounds about right.
 
9:31 PM
when reporting to close a question that should be moved to another stack exchange why doesn't it list all the other networks to suggest where it should be moved to ?
 
@Dave because in 99% of cases moving is wrong
 
This is typically used in migrating data from a prod database to a data warehouse for business intelligence
 
@SomeKittens why do you need the actual dag?
 
well it was regarding game optimisations and i think gamedev exchange would be better
 
What's the best place to auction a domain?
 
9:32 PM
The problem is that I'm building it all in observables and they don't lend themselves to declaring deps very well.
 
@taco oh my fucking god lol
 
grease vid?
 
@BenjaminGruenbaum Fault tolerance, rerunnability, cool graphs. If the process dies halfway through, I know where to pick it up.
 
im cracking up
 
I'm not even sure this is possible.
yesterday, by SomeKittens
it's been a very long time since I've been this over my head
 
9:34 PM
@SomeKittens hmm, you kind of lost me at "the process dies halfway through", are there multiple machines at play here?
 
@BenjaminGruenbaum TBs of data
 
@SomeKittens as in single?
 
and yeah, multiple machines, sources can be diverse as databases, APIs, log flatfiles, mongoDB
 
Oh, so you want a way to orchastrate it?
 
right.
 
9:35 PM
There is a company that does this in particular if you care.
 
There's plenty of them, link?
 
Dato
wait no, they focus on ML
 
Yeah, this isn't ML
 
but maybe they fit, they have a drag and drop interface for connecting data sources and sinks :P
 
ETL & Data Warehousing are their own particular brand of Big Dataâ„¢
@BenjaminGruenbaum Yeah, they look somewhat like what I'm talking about
 
9:37 PM
I've only done really local (as in code that runs on a machine) or really big (as in, code that runs on Titan). I don't have a lot of experience in in between, that said to be fair it doesn't sound too hard. You just need the APIs for whatever you're using for VMs (I assume a cloud provider).
 
@BenjaminGruenbaum The slide on "Coroutines vs. Objects" shows pretty much exactly what I've done on the Object side of things
 
@BenjaminGruenbaum This isn't me trying to do the transforms. I could hack a naive script together to make that happen easily
 
@Zirak I'm glad you're finding it relevant, when you're done with it we can talk about what I disagree on :D
@SomeKittens hmm. How many machines are there?
 
I want to build a system akin to Spotify's Luigi that lets other coders fill in the cracks.
I really like Luigi, but there's three problems with my current situation:
- It's in Python
- It's run-once (not realtime)
- Silly name
(I can't provide a proper explanation for those constraints currently. Sorry.)
@BenjaminGruenbaum Assume n > 3
 
That's ok.
Are you declaring the dependency graph in advance? Can it change in real time? Do you need it to generate the graph for you out of existing (and changing)? How are machines communicating anyway?
 
9:42 PM
Yes
No (neat idea, but waaaaay in the future)
Yes
// TODO
 
What about existing ETL solutions?
There is a bunch I heard of, but like I said - no domain experience.
 
@BenjaminGruenbaum Spent a few nights researching (up to and including implementing Luigi myself). None fit the requirements
Learned a lot about the options an ETL system can have
they range from the very specific to the incredibly broad.
 
Well, I'm sorry I don't have any good answers, your specification is probably a lot more specific than anything I can help with anyway.
 
ah, well
 
I don't understand the domain or what you're collecting very well. I know fault tolerance, and DAGs but I don't know what data you're collecting and so on - and it seems you understand it a lot better than I do (the domain) already :)
 
9:46 PM
that's a first
I'll keep banging my head against this particular wall and we'll see what breaks first
Annoyingly, this is easy in Python
in Luigi, you declare deps by overriding the dependencies method of the Task class. Done.
 
That's not a Python thing, you can have a Task class with overridable dependencies just fine in JS
 
Why is that difficult in js?
 
but nooooo, we're functional style! (Which does make the deterministic nature easier)
 
@BenjaminGruenbaum Yep, that was the first iteration of my system
 
9:48 PM
Elaborate please?
 
it's all in TS 1.5 because I need another layer of bleeding-edge insanity
@Zirak It's a tradeoff. The entire event-based idea is "I can emit an event and anyone who cares will listen", right?
 
Rx?
 
@Luggage Yes, this is using Rx
 
Ohh, I was suggesting Rx as a 'solution'.
 
9:51 PM
If you need explicit dependencies and all that, then Rx is probably not what you want, the whole theory says "up the tree" shouldn't be visible because of security, that's true for promises too.
If you want "up the tree" and dependency, you want something that goes two ways - like an actor or agent.
Or an actual graph where objects have references since you do care who is listening in your case and who they are listening to.
 
@Luggage Yeah, I looked at that
landed in "maybe" territory
 
@SomeKittens It sounds a bit like you found a design pattern (although it's not a design pattern) and try to apply to a given situation
Like, "How do I apply Singletons to this"
 
@copy What would be the design pattern in this case?
 
Magic.
 
9:55 PM
@SomeKittens I would love to tell you, but I don't know
 
@copy ah. Still, good question to ask myself.
 
you want continuous, right?
 
I am jamming "ETL" and "real-time" together here.
 
Just out of curiousity: how are you generating those events?
reading files, selecting from a db?
 
@Luggage yes, plus API, etc
 
9:57 PM
I'd think dealing with "wher did I leave off" and duplicates form those sources would be a much bigger problem.
The rest sounds like just a bunch of streams plugged together
pipes and tees and reduces
(I could very well be oversimplifying)
 
Yes, if it wasn't real time it would essentially be a build script, same issues.
 
I've found that sometimes you need a working knowledge of something just to find out it's not what you need & end up using it for something else down the road
 
@taco I do that all the time
"sure this tool was overkill for that problem, but now I know it"
 
Yeah like for example, Javascript. So useless
 
@Luggage one of the transform tasks is parsing out dupes
 
10:00 PM
@SomeKittens I'm in the ignorance crowd, but imagine each workload being done was a web worker, does it help?
 
but point taken
@Zirak conceptually, yes. I've been trying to figure out how to turn that into useful code, without much success
 
@Luggage Yeah like we weren't using Docker, but I ended up learning it anyways. I'm now looking at roles where that knowledge will help
 
Maybe I stay class based, with each class having an input and output property that's an observable.
so the transport mechanism is the observable, but each node is easily declared.
 
@SomeKittens What if they were tasks?
 
Docker is on my list of things to learn even though I won't get a lot out of it.
 
10:02 PM
Still doesn't help much with my DAG
@Luggage It's neat. Deterministic environments are very useful.
@Zirak /me reads
 
I'm interested in the next level of Docker. Spinning up a load-balanced cloud app using tools such as Mesosphere. I've not gotten that far yet. Docker is also not the best at networking, so there is a tool or two I wanted to look at for that
 
@KasperRT socket.on('error'.. and socket.on('disconnect'... seem to be serving me fine.
 
@SomeKittens VMs are represented by an Agent class, Agents talk to each other via message busses (these can be observables), each agent knows other agents by having references to the.
Graph is simply the agent VM graph and the links.
 
@BenjaminGruenbaum That sounds about right.
Got any reading material?
 
Introduction to multiagent systems by Wooldridge is pretty good but it's a whole book.
Basically, you want to model your machines as autonomous systems that get and send messages and think about them conceptually this way.
Each agent is a "representative" for a VM.
 
10:06 PM
I'd like to pick at a detail, if I may. You said that you have a transform that filters out duplicates. Does this persist such that you can restart the service and it'll still handle a cross-session dup?
 
It's ultra imperative, it's like a slap to the face of functional programming, it's OOP++ :D
(Not in a bad way)
 
goto: message_received;
 
TIL generic wildcards in java
Interesting
 
@SterlingArcher it's interesting, but really confusing.
 
Do you get much Java in your CF, @SterlingArcher ?
 
10:09 PM
@BenjaminGruenbaum the book isn't going too deep into it. From what I'm seeing, it's just a way to make sure a generic is at least a subset type
@Luggage not really, if anything <cfscript> is the closest it gets to Java syntax, but we don't really use those much
 
ohh, I liked <cfscript> syntax much more. Later versions of CF let you do more with it and not tags.
I mean, they made cfscript versiosn of many of the tags
 
Yeah one of my old questions ended up being a cfscript solution
 
It was close to (but frustratingly inferior to) Javascript.
 
4
Q: Importing Excel Spreadsheet Into SQL Database (Coldfusion 9)

Sterling ArcherI'm writing a script to import a large excel spreadhseet into an SQL Server 2008 database. Everything is working fine except for one minor detail: If a cell in the sheet has multiple options (like a <select> dropdown menu), only the selected option gets grabbed. I want to grab every possible opt...

 
@Luggage I don't quite have one, but that is a traditional task for a Transform node in an ETL system
@BenjaminGruenbaum I can dig that.
also allows for much easier scaling
shove it in Docker!
 
10:21 PM
> "Sure, I'll get right on that!"
Translation: "You are an utter buffoon with no grasp on reality."
"That's a really great idea!"
Translation: "I wasn't listening, but we both know you were talking to yourself."
"I'll follow up with you in an email."
Translation: "You're an idiot, but I need a paper-trail to prove it."
"I'm excited about this new initiative."
Translation: "I am literally updating my résumé as we speak."
"Our company's vision is both exciting and fulfilling."
Translation: "You're a snitch to upper management, and I know about it."
 
who here uses bower?
 
10:45 PM
@Luggage what do you use this noflojs for?
 
I don't. I was once interested in the UI (which is a separate library).
The drag and drag, attach things ui.
 
@phenomnomnominal why gross?
 
@GarrettKadillak because npm does everything it can do. Multiple package management systems just fragments the ecosystem.
 
@phenomnomnominal very true. I've always wondered why there are so many package managers
 
Are you sure npm nd bower do the same thing?
Because they have pretty different paradigms
 
10:56 PM
the community can't seem to make a decision and are happy to move on to the next thing
 
similar.
 
npm does flat dependencies now
 
then bower offers nothing.
 
10:57 PM
but I still run across libraries that are published in bower and not npm
 
then why does it exist?
 
if nothing else, at least it helped get flat dependencies into npm
 
of the npm version is old or someone else
 
@Luggage I use nada for those cases
 
Oh if npm does flat dependencies now then ignore me
 
10:57 PM
!!xkcd standards
 
bower was created to be "like npm, but for client-side dependencies"
 
@rlemon That didn't make much sense. Use the !!/help command to learn more.
 
Bower still has a cute mascot though
 
@phenomnomnominal nada?
Also.. i forget i can install from a git repo.
 
10:59 PM
@Luggage sorry npmjs.com/package/napa
 

« first day (1725 days earlier)      last day (3223 days later) »