« first day (2023 days earlier)      last day (2930 days later) » 
00:00 - 20:0020:00 - 23:00

Ben
12:08 AM
I'm looking for cloud servers for computationally heavy tasks. Is AWS easy to setup or should I go with another platform? Either Azure, TensorFlow, or a data science site that accomplishes the same thing?
 
what kind of computation ?
tensorflow is not a cloud server
you have aws gce digital ocean and other cloud service providers
 
Ben
i'm working with NumPy arrays and Pandas df's. Trying to get lots of cores for parallel processing
 
they have normal high mem and high cpu categories for you to buy
you need cores thats fine
but unless you have run into bottleneck there is no point going for it first
sometimes theres lot of data to be in memory
so
and python takes a ton of memory
 
Ben
agreed on the memory usage. Incorporated gc.collect and deleting vars where I can
 
dont obsess about cores for now just buy standard one
 
Ben
12:17 AM
bottleneck is def cores
 
see how it goes then change
how do you plan on scaling
 
Ben
buy a standard one? Meaning A cloud service provider?
 
standard as in normal ratio of cpu to mem
of normal/high mem/high cpu
 
@user568109 are you sure about the memory thing, with numpy?
 
Ben
i have a throttle param that limits the memory usage per job. I need about 2GB RAM per multiprocess
memory thing is with pandas :)
 
12:20 AM
ah
 
Ben
job management will be via a queue so ideally several servers (each with say 32 cores) can do their thing
 
what do you use
 
Ben
but i'll start with one server then scale up. So in terms of ease of use, ease to setup. AWS? DigitalOcean? Other?
 
for me google compute engine takes the cake
 
Ben
checking it out now
cool, signing up for google's compute trial. ty
I was using dominodatalab before. Also came across ufora, have you used that? Not sure if it's applicable
 
Ben
12:38 AM
tonight's proj is google compute - thanks again @user568109
 
 
4 hours later…
4:19 AM
cbg
 
 
4 hours later…
Hey guys! Quick question - why is numpy array so memory hungry?
This is the size of my numpy array (549, 256, 256, 4), and this is what it said in the console of the size:
float32 575668224
 
>>> 549 * 256 * 256 * 4
143917056
>>> 549 * 256 * 256 * 4 * 4
575668224
your array has 143917056 elements
and each of these is a float4, using 4 bytes
thus your array size is 575668224
what did you expect?
 
Ohh shoot, I see - I miscalculated. I thought that dimension was 10x smaller than it actually was. My bad!
2 am looking at #'s ; _ ;
 
yeah. Put a postit above your screen saying "at 2 am, probably PEBKAC" :D
 
fair enough of a criticism LOL
 
8:03 AM
funnily, numpy does not seem to count the sizeof PyObject into that.
or ah that wasn't sys.getsizeof
 
Yep, it doesn't. ^
 
it was just the ndarray.nbytes
 
yeah, we can fix any problems with your code but the one big problem remains unfixable ;)
 
@AnttiHaapala Hey - it's not unfixable! I just need some coffee and I'm good to go(maybe)
Decided to downsample my data to 64x64, don't know if this is going to bite me hard ruh roh
 
8:11 AM
by us
sleep could also work :P
usually sleeping 8 hours helps more than staying awake those 8 hours :D
 
True true, but as a freshman college kid I dont understand the definition of sleep
 
yeah you need to graduate.
 
That would be optimal yeah
 
as they say in Finland, the difference between an engineering student and a graduate is that the latter knows that you are not obliged to consume all of the free liquor you're offered.
 
wait - you mean at this moment?
Ah, I'm allergic to liquor so I wouldn't consume it in the first place :'(
 
8:19 AM
@Ffisegydd what this here? A sacriledge!
 
Shhh noone must know
 
Cbg
I just noticed this PCG question in the HNQ sidebar: Compute the Adler-32 checksum... and look who's posted a Mathematica answer: Mark Adler himself! :)
 
@PM2Ring upvoted :D
I think Adler deserves more of those "facts" than Jon Skeet.
Jon Skeet didn't change my life in any way.
Adler has :D
 
8:37 AM
He's a bit funny in the comments :)
 
@AnttiHaapala Sure. Zillions of bytes all around the world have been compressed and checksummed with Adler's algorithms. That's a pretty fine accomplishment for a coder, IMHO.
 
Jon Skeet has mostly annoyed me on StackOverflow
I've seen his answers occasionally in , and usually they're not that good.
and that's it.
 
8:52 AM
He's good on C# and Java
 
yeah but he hasn't affected my life in any way
 
That is annoying :)
 
never ever read a Jon Skeet answer that was useful to me
but those autogenerated avatars are in PNG...
and most of the http in this chat is probably content-encoding: gzip.
so Adler is with us as we speak :D
wish more people were like him, actually following discussions about their area of expertise
 
You're annoying because you also haven't done that
 
haven't done PNG? Haven't read a Jon Skeet answer, haven't changed your life? :D
 
8:59 AM
Any reason you consider JS annoying :)
Why didn't you happen to be involved in an algorithm that really took off?
 
well, as I said, I've never seen an useful answer by Jon Skeet
 
Such an annoying guy.
 
I am not annoying by that metric.
I've tried to upvote my own answers.
 
Surely you didn't find your own answers useful? You already knew them!
 
FWIW, I answered this SO Meta question a little while ago: meta.stackoverflow.com/questions/322096/…
 
9:05 AM
"Ah, this guy's not telling me anything I don't know already!"
 
@RobertGrant that is exactly what happened
I was like "oh, now this is a useful answer, gotta keep this in mind", then hit the upvote button and was greeted with "you cannot upvote your own answers"
 
@Natecat eval and exec should generally be avoided because they can be a security risk. For details, please see Eval really is dangerous by SO veteran Ned Batchelder.
In that particular situation the security risk is minimal, but it's still a Good Idea to avoid using exec when you don't really need it. At the very least, there's a speed penalty. FWIW, Antti has written a great answer about exec here
 
@Natecat in that case you do not need to use exec.
You can use globals() to access the functions as global variables
globals()['doCoolStuff_%s' % system]()
this cannot do anything evil. It just fails with KeyError if a function for the given platform does not exist, and that's it.
However I'd not use globals() for this either...
instead I rather prefer using a registry + decorate pattern.
@PM2Ring my answer does not really answer when to use exec, only what are the differences of exec and eval.
 
9:50 AM
@JRichardSnape I think I fixed my problem. Flask-principal (or something else) seems to have slightly changed how it works, so an attribute user_authenticated is now a method user_authenticated()
Don't know why it's still working on my local machine though
 
10:10 AM
@RobertGrant I see that you're using Falsk instead of Pyramid... >:D
 
Yeah I converted because I couldn't figure some stuff out
I will try Pyramid again, but only when I've got a bit more Python experience under my belt
 
the pyramid authz/n sucks :(
 
Yeah that was one sticking point that took me ages
Another was at the time sqlalchemy-continuum didn't work with pyramid
Not sure if it does now
 
it does
I am using it
but it is shit
everywhere, pyramid or not
 
:)
Yeah flask-security is okay, but still, the docs are split over like 4 different plugins
 
10:26 AM
I'm guessing you're not being paid by the module's author for their marketing? :p
 
@AnttiHaapala Sure, which is why I prefaced that sentence with "FWIW". I just figured it was worth mentioning it since it has good info for people like Natecat who are interested in exec.
 
11:08 AM
cbg
 
anyone interested in doing something fun with neural networks?
 
@Christoph Miles Dyson won't be involved will he?
 
@JonClements he's on board
 
Bit worried if it gets confused - Jon Clements and John Connor might get mixed up and I fancy a fairly lazy day... not running for my life...
 
Kam wid me if you want to live
 
11:17 AM
@AndrasDeak do you want to make sth with neural nets?
 
I'd love to! Where do I download it?
 
@AndrasDeak what?
 
Well I assumed it's in the cloud, I shouldn't need to have a hard drive mailed to me or anything.
 
@AndrasDeak I meant we could program sth together
 
Oh, programming. I don't like that, sorry.
 
11:25 AM
Laurel
 
Hi... I'm creating command line cli for everyday use. It's bit of file based project. Initial coding done. But I wanted to write a test case for all. What method should I follow ? 1) Import exiting function into the test.py file and give values and get the output or 2). write a few need functions inside the test.py file and run it..
 
You know you generally write the tests first, right? :p
 
yes that's right...
so after that I will right the function.. cool....
 
11:45 AM
So @ChristophBühler is there any money involved? I'm only asking to determine whether you are a spammer, or you just sound like one.
 
@AndrasDeak Well.. it's my passion to program. If you can't understand that, maybe you should hang out in tumblr or sth.
 
Or maybe he should hang out here
 
@Christoph no need to get offended - keep calm and civil please
 
Indeed, @Christoph, I didn't intend to offend you (as long as the answer to my previous question is "no")
 
@AndrasDeak there is no money involved if that is the question.
 
11:50 AM
@ChristophBühler also - posting the same question is multiple rooms can be considered spamming - so tread carefully - users might raise a complaint about that
 
I tried to tell you that your method of communication can be seen as off-putting, so to speak. If you mean well and just want to have fun, you should reconsider asking in a more specific way to avoid being confused with a spammer:)
 
I'd like to do something with image manipulation or image processing in python
 
Personally, your "anybody want to make sth with neural nets" question reminded me of "want to make huge $$$ easy?" and "top 10 clickbaits people find annoying" kind of stuff. Again, since this is not a case, I'm just telling you this so that you can see why I asked what I asked, no offence meant:)
And if you haven't seen it yet, the tensorflow playground is awesome.
 
ok, well that looks nice
I made this a while ago: norizon.li/repo/neural-net
 
11:55 AM
Greetings, @ChristophBühler. Sorry if you feel offended or insulted by the response you're getting. We appreciate your enthusiam to code interesting stuff, and to collaborate. However, we don't know you, and it's a bit weird for a stranger to come into a space and basically say "Who wants to collaborate with me?"
 
@PM2Ring Okay, well. I like Python and wanna get into it. So.. I'll be here more often probably.
 
Excellent!
 
12:36 PM
cbg
@PM2Ring FWIWWINMAA
 
1:06 PM
@RobertGrant pleased to hear it. I could log in now too :)
 
1:17 PM
@MartijnPieters @TigerhawkT3 Please see meta.stackoverflow.com/questions/322096/… :p
 
@AndrasDeak That is a brilliant tool.
 
dupe stackoverflow.com/questions/36955069/… Anyone game to hammer this? :)
 
haaaaammmmmmmmmmmaaaaaaaa.
Though honestly Martijn's answer is better.
It explains all the ways to do it, explains why they're bad, then brings out the deque.
 
1:33 PM
@Ffisegydd True. It condenses all the information into a single excellent answer. It's still a dupe, though. :)
 
I love the guy on the original question who suggests using numpy for it
 
cbg guys
 
Hello.
 
FWIW, I've seen Martijn provide a better answer like that, and then dupe-hammer it himself.
@Natecat Well, that'd be appropriate if the list is large, but it's mental if the list is small and you're not already using Numpy.
As someone said the other day, Numpy is the jQuery of Python.
2 days ago, by davidism
Because "use numpy" is the Python equivalent of "use jQuery".
 
Ughn, another Ignacio documentation link for an answer..
 
1:40 PM
@MartijnPieters which one?
 
@MartijnPieters This repeated pattern bugs me also. I do recall that particular poster getting a bit stroppy when I mentioned it once. I think you were quite right to provide an expanded answer on the newer duplicate (fine it is a duplicate)
 
@MartijnPieters Ok. I agree that that accepted answer isn't great, and that your new answer is top-class. And as I said in my Meta post "Sometimes it's actually quite ok to post an answer to a duplicate question".
TBH, I did look at a couple of other dupe targets because I was underwhelmed by that accepted answer by Ignacio. But I figured it was a reasonable dupe target since the other answers there covered the various approaches, and the question is linked to 20 other questions, making it a reasonable hub for future readers with similar questions.
 
user559633
1:59 PM
cbg
 
For the last 4 hours I've been listening to a radio special on soul singer Billy Paul, who died last Sunday.
Billy Paul - Me And Mrs. Jones Soultrain (1972)
 
user559633
this is pretty great
 
2:23 PM
Here's some more soul, Dutch style. Sax player Candy Dulfer's band featuring Roger Happel and Moon Baker on vocals covering Marvin Gaye's Let's get it on. I love the energy between the vocalists on this clip. Either they're merely very good performers, or they're about to rip each other's clothes off the moment they get off-stage. :)
 
@PM2Ring: I wasn't actually commenting on the dupe voting here. Just a gut-reaction to any such an Ignacio-answer.
 
@MartijnPieters Understood.
 
@PM2Ring Candy Dulfer is excellent!
Note: female drummer!
I still have some drum sticks from her from a concert I visited somewhere in the 90s or so.
 
@MartijnPieters so? :)
 
@RobertGrant We don't have enough of those. She's great.
It was perhaps a bigger deal in the 1990s.
 
2:33 PM
Okay :)
 
The whole band is excellent. But I agree, female drummers should definitely be encouraged.
Speaking of female drummers, I posted this clip a week or two ago, but I guess it's worth repeating: all-girl Latvian band Red Bee covering Hendrix's version of Hey Joe
The excellent lead guitarist of Red Bee is Antra Lante.
 
3:16 PM
omnomnom ice cream float.
 
user559633
Describe it to me.
 
I put ice cream in cream soda then drank it.
 
user559633
Root beer soda or cream soda?
 
Cream Soda.
 
user559633
Vanilla ice cream? Paint a picture with words.
 
3:18 PM
Vanilla.
All I had in the house. Though I may try another of those wonderful cocktails that my BCFF @Morgan linked.
Ice cream floats aren't really a thing in the UK.
 
Due to our 8 seconds of sunlight a year
 
DSM
Vanilla-flavoured cabbage for all.
 
That sounds...disgusting...
 
3:36 PM
@Ffisegydd I still need to make one of those. Probably tonight.
Depending on hung over I am.
 
hey guys
 
Hey, idjaw. How's things?
 
things are good! Flying back home today. It was a long week. Great week. But started feeling it's length. It's going to feel good to be back home
how about you
I did myself proud eating as much texmex and bbq as I could in a single week.
it was fantastic
 
That sounds fantastic. I'm doing pretty good, waiting for the shop to call me and tell me how much I owe them for my car while I recover from a hangover.
Now I really want BBQ.
 
Hey @MorganThrapp: Some Tedeschi-Trucks + Allman Brothers: Stand Back
 
3:43 PM
Oooo, nice! Thanks!
 
It's great hearing Susan singing those old Allman Bros songs.
 
Yeah, it's awesome.
 
well, just wanted to pop in quickly. Looks like I got the "you have been on SO for too long badge". I think you just got it too Morgan. Fanatic ftw.
 
Nah, I had it restart somewhat recently because I forgot to log in one Sunday.
 
gah...that sucks
 
3:48 PM
Yeah, no kidding. :/
 
Heh. Thought my PC was being a bit slow. Turns out I had 12 different python scripts running in the background that I hadn't killed using about 80% of my resources and maxing out my CPU.
 
DSM
That'll do, Fizzy. That'll do.
 
I'm looking to put the SO data dump into a graph database.
The current unencrypted dump (including all tags) is about 132 GB.
Makes my desktop sad.
Gonna subset it down to Python tags only though.
 
DSM
4:06 PM
132GB! How long did that take to download?!
 
For some reason I said encrypted, I meant zipped.
Anyway, the zipping is really efficient.
Only took an evening.
 
DSM
What sort of things are you hoping to look for?
 
30GB zipped maybe.
I'm hoping to make some nice visualisations using it, but also going to be looking for relationships between users.
I've got no clear use cases yet, just a few random ideas.
 
Most CPU-chewing force directed graph ever, I hope
 
Fizzy, have you ever represented graphs using a Hyperbolic tree? Eg Walrus. I just did a quick Google for a Python package and found github.com/buzzfeed/pyh3
 
4:10 PM
Posssssibly I could maybe even look at something like the famous SO community visualisation
That one that looks like towers.
 
DSM
I wasn't in there because I've never asked a question. :-(
 
It was just before my time.
I've looked into hyperbolic trees, but never actually used them.
Would be interesting to look at something similar (even if not so grand in scope) showing the different communities within Python.
 
DSM
True. I'm kind of curious about python subcultures that I might not even know about because I only care about numbers.
 
I've never used hyperbolic trees either, although I have done a little bit of stuff with hyperbolic tessellations. But I thought it could be useful for that SO data, since its good for showing large numbers of nodes.
 
sq.ro has got some awesome visualisations
 
DSM
4:15 PM
@PM2Ring: I may forward that to a friend at my old firm. We had some pretty complicated hierarchies to show. So, thanks. :-)
 
@Ffisegydd Wow!
 
Shame a lot of the prints aren't available.
I've often wondered how he did that towers visualisation.
 
@DSM Cool! I always thought hyperbolic geometry was cool, but a bit too theoretical, so it's nice to have practical applications for it.
 
Massive AD forests would look cool
 
And in the centre of the visualisation, the spider at the centre...Bobby G...
 
4:23 PM
Welcome to my world-wide web, puny humans
 
DSM
It's fun when you see a question in another language (in this case, about why two equal objects are both showing up in a set) and you can answer based entirely on Python knowledge (you didn't specify a hash).
 
I'm hoping I'll be able to open the db to the wider world via the web ui for people to have a look round.
 
Graphics master Jos Leys has some hyperbolic stuff in his gallery
 
DSM
4:45 PM
At one point I was optimistic about computer graphics and music technology, because I thought they would unleash my inner Rembrandt and Mozart, on the grounds that the only thing holding me back was the fact I couldn't draw and can't play. It turns out that you still need talent, more's the pity.
 
wtf is that answer
Might be a member of those voting rings Brad mentioned yesterday
 
5:01 PM
@vaultah Weird. And it won't even run, due to that useless x parameter.
 
Dom
Hi folks, can anyone tell me why this would work correctly only once and incorrectly the second time?
        player_wins = 5 // Pulled from DB via query just simplified
        player_losses = 4
        player_draws = 2
        player_score = player_wins * 10 + player_losses * -10 + player_draws * 5
came out with the correct score the first time and then 0 the second?
 
Whatever it is won't be in that
 
no repro, the code works correctly
 
Yeah your simplification has hidden the issue
 
Dom
5:16 PM
        player_wins = GameResultModel.query(GameResultModel.player == player.key).filter(GameResultModel.result == 'win').count()
        player_losses = GameResultModel.query(GameResultModel.player == player.key).filter(GameResultModel.result == 'lose').count()
        player_draws = GameResultModel.query(GameResultModel.player == player.key).filter(GameResultModel.result == 'draw').count()
        player_score = player_wins * 10 + player_losses * -10 + player_draws * 5
 
You must be doing something later to mutate the db
 
Dom
I just make an object out of it and send it down the pipe as json
 
So why is everyone using qt designer instead of hand coding these days. seems every pyqt question is something like "I was using qt designer to make a gui" and then the person often has no idea how to code. Has it always been like this, but I just recently started paying attention?
 
Dom
the numbers pulled from the db come out correctly
only the player score doesn't
{
"player_results": [
{
"draws": "0",
"losses": "0",
"score": "30",
"username": "DomTheBomb",
"wins": "3"
},
{
"draws": "2",
"losses": "5",
"score": "0",
"username": "DomTheBiggedyBomb",
"wins": "4"
}
]
}
ignore the usernames lol
 
Yeah those are right
 
5:19 PM
lol
Have you actually calculated what it should be?
 
Dom
ohhh shiznit
face/palm
my bad thanks for letting me use you as a calculator
 
No worries, Dom the Biggedy Bomb
4
 
Dom
I hope that doesn't stick
 
The Biggediest Of Bombs.
 
Dom
Test data guys!!!
painful lesson learned - don't automatically think 0 is a failure
I will retreat and start a new profile now....
 
5:27 PM
Probably for the best
 
@vaultah That deleted answer is unfortunate, but the other answer that just contains hints is ok, IMHO. But I won't upvote it, since that question deserves to get Roomba'd, unless the OP posts a serious code attempt... but of course that's highly unlikely.
 
5:45 PM
i don't like answers that only consist of hints :/ I wouldn't even call them "answers"
not enough to downvote that particular answer, though
 
DSM
In ancient times I think SO used to be more tolerant of hinty answers. We've moved away from that, especially after we lost the homework tag. I think it's for the best.
 
6:13 PM
things that are only hints should be downvoted, right?
as in "not helpful"
or "not an answer"
or "I hate your rep whore guts"
 
@AnttiHaapala evening
 
6:50 PM
so silent
lmaoed at a "classical" music concert this evening
or should it be said l'edmao
 
user559633
i need to get more into classical music
 
user559633
interesting
 
and not only were they playing and conducting and riverdancing and kungfuing etc well, but they were actually joking in Finnish as well. So kudos.
ah forgot singing
 
Kudos? More like kiitos *bam*
 
6:58 PM
@AndrasDeak lol.
 
Hello all, my question got pushed to the next pages on the questions and I was wondering if I could get help with it?
 
the only problem is I didn't have any cash on me so I couldn't buy their CD which they were also signing
 
@tristan I really enjoy it, mostly enjoy instrumentals.
In short: if it's more than 1-2 days old, then link it.
 
user559633
@Ffisegydd Same. I'm a classical music peasant, so I end up just listening to postrock insead
 
Otherwise, no.
 
user559633
 
@Ffisegydd Yep thanks. Didnt know that!
 
user559633
@ZackTarr weekends are a bit slower for answers, but i'd be surprised if you don't get at least a comment on your question before tomorrow
 
@tristan youtube.com/watch?v=Ov_qBR79e08 here's Igudesman and Joo for you :d
 
user559633
strange.
 
00:00 - 20:0020:00 - 23:00

« first day (2023 days earlier)      last day (2930 days later) »