« first day (1748 days earlier)      last day (3198 days later) » 
00:00 - 15:0015:00 - 00:00

12:00 AM
Is it? Well, there will be other indications one can look for...
 
how they search for an item in a list for example :P
 
usually when I see a double for loop turned into a list comprehension I think someone is progressing
or started in Perl
or Haskell
 
 
2 hours later…
1:41 AM
I know they've moved to the intermediate stage when I see them using set methods
 
set methods as in accessing attributes directly or using properties instead of trying to create/use setter methods?
 
Or methods of sets :)
 
1:59 AM
what Jon said
 
ah, I've always used property decorators as an indicator that those students who are primarily Java programmers are starting to "get" Python - so wondered
 
properties are for attributes that are programmatically determined, as I see it.
hg.python.org is down?
Java programmers will write get_this and set_that...
 
literally getThis or setThat
I prefer the way that Python does it but then I like Python ;)
 
2:18 AM
hey hg.python.org is back up!
yeah, sorry, I used the Pythonic way of writing the Java-ish thing.
 
they must have heard you
 
or maybe they were cycling the server and I got lucky.
 
nah, they heard you
it's the Net man, it's getting smarter
 
2:32 AM
Hey, you ever use Events in threading?
I'm trying to figure it out
threading, module of the week: pymotw.com/2/threading
 
3:26 AM
I think my threading demo worked on the first attempt!
well, technically I had a couple of errors... 3rd attempt?
it felt like the first attempt, though. that's how you know I'm managing my expectations.
sleep result: 54127791
while result: 18861179
sleep allowed 2.86979891342 times work
done relative to while
on my linux machine
sleeping every second
that was python 2.7.6- here's 3.4.0:
sleep result: 47988771
while result: 11852428
sleep allowed 4.0488557281259165 times work
done relative to while
Python 2.7.10:
sleep result: 42420054
while result: 13833122
sleep allowed 3.06655677583 times work
done relative to while
Anyone want to try it on Windows? I'll make a gist...
Here's the gist, if people would run this on systems other than Ubuntu linux, varying Python versions, but especially RHEL and Windows, and post their results, that would be sweet: gist.github.com/aaronchall/1ecb40580e549923cdaa
Anyone around?
I wonder if I should write this up as a Q&A...
I'll wait for a windows and mac demo first, I want to ensure I have consistent results, I'm pretty sure RHEL would be the same, based on my spelunking through the sleep source.
Good night everybody!
 
4:12 AM
Couldn't sleep, had to redo the test switching the threads, but same result! ok, back to bed...
And you can sleep a thousandth of a second instead of a second, and still be more than twice as performant.
ok, bed.
 
 
1 hour later…
5:32 AM
cbg
 
5:43 AM
Cbg @Jerry
 
6:09 AM
Cbg
 
6:27 AM
@AaronHall windows, Python 3.4.3:
sleep result: 47464026
while result: 23136574
sleep allowed 2.051471665597508 times work
done relative to while
 
7:01 AM
morning all
 
7:11 AM
@JonClements cbg
 
7:32 AM
Cbg
 
cbg
@RobertGrant obviously
 
Design question: I'm storing things like annual leave, public holidays, etc, and from that calculating how long projects will take. I'm not storing the end result at the moment, nor the data from intermediate steps that would maybe allow me to calculate other stuff (e.g. for this annual leave, these are the projects affected and by how much). Should I store more stuff, or just calculate off the source data each time?
 
7:51 AM
the answer is: depends
 
Here's a different, more noob question: if I memoise something inside a web application, does the memoisation only stay in effect for the during of that request/response?
Or does it stick around for good?
 
8:08 AM
Cabbage!
@RobertGrant Depends on the web application :P
> For After events, the definitions of Before and After properties are straightforward: Before properties refer to settings that existed before the event occurred, and After properties refer to the settings that exist after the event has occurred. For Before events, however, Before properties refer to current item settings before the event occurs, while After properties refer to settings the item will have after the event occurs.
^ That’s both times the same thing, right?
 
@poke yeah fair enough :) I'm just wondering if I can memoise certain things in the future then if the calculations share the same functions, I can do something like memoisation to cache the functions' results in the future
 
What kind of server do you use? E.g. Flask runs in its own process that persists between requests, so the memory stays intact (some server managers like uwsgi may start multiple application instances though).
 
Pyramid
 
Apache on the other hand will usually launch a new thread for each request, so you would have to use something else there. A common approach there would be memcached.
 
Hm yeah I guess then in that case as you say, I just use an external caching solution
Which might be better anyway, separation of concerns and all that gubbins
 
8:17 AM
From the tutorial it seems that Pyramid also spawns its own server, so memory would be shared between requests.
 
Hm, also interesting
Thanks - that's given me enough to chew on :) I'll recalculate each time, and use some sort of caching to speed stuff up should it become necessary
 
8:36 AM
starts last ML lab
 
8:56 AM
yesterday, by Ffisegydd
I believe in you Bobby
 
Required :)
Although the data set is neural imaging data from a zebrafish, which is cool
Had a good chat with chap from my old company yesterday; they'd like me back over in Blighty
Once again failed to negotiate well, he said a number that I thought sounded reasonable and I couldn't think after that.
So...who knows.
 
That's Number Wang!
 
I could be enjoying the advanced delights of 24/7 electricity and fast internet before long
@JonClements lol yeah, I should've said that
 
@RobertGrant Return to the Promised Land...
 
Also free healthcare for my unborn child, that could be nice
 
9:07 AM
You'd better hurry up while we still have an NHS then Bobby G :)
 
Heh yeah true! has more kids
 
> has more kids
your wife may object if you start that project right now...
 
PSA ;)
 
Meh, we'll adopt. Probably do that anyway; why not speed it up.
 
9:08 AM
obligatory premature optimisation meme
 
9:21 AM
Premature something is the root of all evil?
 
9:32 AM
...something about Number Wang
 
 
1 hour later…
10:36 AM
cabbage
 
11:11 AM
A thing I have just seen: putting "guru" in your self-description on a programming Q&A site. "Machine learning guru", no less. Not a great idea when you then start asking noob, unresearched, non-mcve questions, methinks.
 
I swear, I just saw someone at work wearing something that had the same color as the dress!
 
The dress?
 
not..... the dress.....
 
Oooh. The dress
 
11:12 AM
whispers gold, black, blue, white. Mythological things
 
@RobertGrant Thanks!!!
@JRichardSnape Aspirational title, I suppose.
 
@AaronHall indeed so, indeed so. Do you want results for 2.7 and 3.3 on Windows, or have you exhausted your experiment?
 
I'm now mostly interested in RHEL, if someone has it. I'll do a canonical Windows experiment at work.
for my purposes.
 
def d():
a=set(range[1:1000])

timeit.timeit('d',number=1000)
NameError: global name 'd' is not defined
timeit.timeit('d()','from __main__ import d',number=1000)

TypeError: 'builtin_function_or_method' object has no attribute '__getitem__'
How can I timeit the function d()
 
I'd remove that
do this: range(1,1000)
 
11:20 AM
Mepis 11 Linux, Python 2.6.6
sleep result: 5763522
while result: 2847130
sleep allowed 2.02432695381 times work
done relative to while
 
then do this (hint, d is not a string): timeit.timeit(d,number=1000)
or do the setup, whichever
anyone able to do that on a mac?
 
morning everyone
 
Sorry typo
 
I could do CentOS, which is based on RHEL rather than Debian. Is that a help, or do you need branded Red Hat?
Well - did it anyway
 
yeah, let's try that. :)
 
11:24 AM
[root@v00web00008l ~]# python thread_test_while_versus_sleep.py

sleep result: 57242968
while result: 27955628
sleep allowed 2.04763663331 times work
done relative to while
Python 2.6.6 (standard distribution, I guess)
Results seem pretty consistent TBH
 
people still use python 2.x ?
 
It's a web server box, not using python at all - just whatever was on it.
 
I wonder why ubuntu gave a higher ratio, 3:1 up to 4:1
everyone else is telling me 2:1
 
Was anything else heavy running at the time?
 
@VigneshKalai range[1:1000]gives a syntax error, unless you've used range as a variable name, which would be pretty dumb. :)
 
11:27 AM
Yeah sorry for wasting all your time :P
 
@JRichardSnape Yep. That's what's in (most) Debian-derived distros.
 
user559633
morning everyone
 
cbg
 
@VigneshKalai You might like to look at this code I posted a few months ago that times various implementations of the factorial function.
 
11:35 AM
I have a doubt why is a=set(lst) more faster then a=set() for c in lst:a.add(c) both are doing the same thing only right
 
user559633
@VigneshKalai Does your profiling give you any hints?
 
def d():
a=set(range(1,10000))

timeit.timeit(d,number=1000)
0.5335876876816883

def d():
a=range(1,10000)
b=set()
for c in a:
b.add(c)

>>> timeit.timeit(d,number=1000)
1.5846671363193536
Is it due to increasing memory on the go
 
user559633
@AaronHall is this because the interpreter gets to not check back for a full second as opposed to evaluating pass continuously? (sorry for being too lazy to look at the disassembly, just woke up)
 
es6 is weird. I have no idea how it namespaces stuff
 
I expected that set(lst) would be faster since it could look at the size of the list and perform a single memory allocation for the set, as opposed to add(c) which would require log(len(lst)) allocations. But poking around in setobject.c, it looks like it doesn't actually bother, and set(lst) merely iterates through the list's elements, adding one item at a time.
So I guess the difference in efficiency falls on the Python for statement being slower than C's while loop
 
user559633
11:45 AM
Would would it be log(len(lst) allocations? wouldn't it either do N+1 allocations or do resizing when num elements is approaching the length of the array?
 
user559633
Also, sorry to jump topics slightly, but can someone point me to where list comprehensions are defined in the CPython source? I don't understand how it does the anonymous return ( e.g. [ x for x in somelist if x not == '1' ] does it create a temp var under the hood to append the results onto before returning? )
 
I'm assuming that sets work similarly to lists, in that when they need to become bigger, they double the size of their internal collection, so as to reduce the number of malloc calls the system has to handle. I don't know if that's actually true of sets though, just a guess.
 
Lee
i have been trying to mock a constructor with no luck as per stackoverflow.com/questions/31721503/…;, I got an answer that illustrates the issue I am having, i just cant get the subtlety involved.
`>>> a = mock.MagicMock()
>>> a.return_value = {"spas": True}
>>> a()
{'spas': True}
>>> a.__call__()
{'spas': True}
>>> a.__init__()
>>> a.__call__()
<MagicMock name='mock()' id='139926394332560'>
>>> a()
<MagicMock name='mock()' id='139926394332560'>`
What changed after init was called?
 
sleep is implemented differently for each platform, from my reading of the source. Some platforms are interruptable, some aren't. But it's not equivalent to a high level Python while loop.
 
user559633
@AaronHall yeah, exactly. my understanding is that pass is still handled in bytecode as opposed to sleep which only hangs out on the interpreter for the boundaries (starting/ending sleep)
 
11:52 AM
@tristan I think you want compiler_comprehension_generator. If I'm reading it right, the answer to your question is "yes".
 
user559633
@Kevin thanks starlord, you so dreamy
 
Lee
Never mind, the answer was wrong, the correct answer solves it...
 
user559633
afk, going to get in a run before work
 
good source reference here, I can cite that.
 
@Kevin I'm almost certain that's the main reason for the speed difference
 
11:57 AM
while printing some result I get in this way
3.90449968216e-11
3.11596810767e-10
is there any way to print in % format?
 
Just so you know, 3.9e-11 is equivalent to 0.000000000003%.
 
@Kevin nice link, thanks. I was looking for how with was implemented yesterday. Also contained within that mega-file
 
@Kevin yeah correct, but want to show in such readable format
any lib function?
 
Or something to within a couple orders of magnitude of that, I may have missed a decimal place. It's definitely real small though.
 
@Kevin TTBOMK, Python sets use the same strategy as dicts, since sets are basically dicts with keys but no values. Laurent Luce has a great article on Python dictionary implementation. I read that article a few months ago, but I must confess I don't remember the details. :)
 
11:59 AM
I wonder if str.format has a flag that lets you force non-scientific notation for tiny numbers...
 
yep - one of those numbers is 10 times the other, but for nearly all practical purposes they are both nothing. Still - yes - you can format them as you wish...
 
@JRichardSnape thanks, any specific math function to format it?
 
@nlper check out the decimal module: docs.python.org/2/library/decimal.html
 
Like this?
>>> a=3.90449968216e-11; print('%0.25f' % a)
0.0000000000390449968216000
 
12:05 PM
ok, gentlefolk, I must prepare to go to work, and then go to work. TTFN
 
@nlper If you just want the string representation - use the string format options. e.g. if k is your variable
'{:.11%}'.format(k)
Argh - @pm2 has done a Kevin on me
 
@PM2Ring Skimming through that, it looks like dicts (and consequently, sets) have a resize strategy similar to lists: when the internal table is 2/3rds full, it doubles the size of the table until it's larger than four times the number of currently occupied slots.
So I'd expect number of memory allocations to be approximately logarithmic in that case as well
 
@JRichardSnape :) Your code's more modern than mine. But you probably need a bigger precision specifier.
 
@nlper - string.format documentation : docs.python.org/3/library/string.html#format-string-syntax . %age to arbitrary precision - is found if you search for "Expressing a percentage" on that page
 
@JRichardSnape thanks this helped
 
12:09 PM
@JRichardSnape And since you're printing a decimal as a percentage it should probably be .format(k*100)
 
@PM2Ring thanks for this suggestion too
 
So to be simple it is due to pre-allocation right
 
No, it's not due to that.
 
@VigneshKalai No, it's due to C loops running faster than Python loops.
 
Uhhh now I am confused
I am not that good at c I did not understand the link :(
 
12:13 PM
In hindsight, it would be kind of hard to pre-allocate the size of a set, since you don't know how many duplicates the list contains, so if you just create an internal table of len(lst) items, you might end up with a table that's far too large.
 
Ok so when set(list) is called internally c program while is running right
 
@VigneshKalai You don't need to be good at C. You just need to know that the Python interpreter is a virtual machine that runs much slower than your machine's real CPU. So a for loop written in Python's going to be much slower than a for loop running in native machine code.
 
@PM2Ring Nah - the % in the {} deals with that (you can count the zeros to check if you like ;)
 
Executive summary: set(lst) and for item in lst: my_set.add(item) do the same thing, but the first one does it in C and the second one does it in Python. The number of memory allocations is the same, but the first way is faster, just because it's C.
 
@JRichardSnape Wow! So it does. As you might guess, I rarely use .format(). :)
 
12:17 PM
@Kevin and @PM2Ring thanks now it makes a lot of sense
 
I have one audible credit. What should I get?
 
Darude - Sandstorm
 
What does "imprevisible" mean??
 
Adjective: imprévisible (masculine and feminine, plural imprévisibles)
  1. unpredictable (unable to be predicted)
 
OP is from France, so that seems likely
 
psst. There is a second "imprevisible" in the post.
 
I noticed xD
 
Thanks, Poke.
 
You’re welcome. And sorry.
 
@PM2Ring ah that makes sense - thanks!
 
12:30 PM
@RobertGrant Oh, good. :)
@poke What are you sorry for?
 
My revision comment might have been mean, because I actually just googled for the word and posted the first link :P
 
I don't think anyone reads revision comments anyway :-)
(expected response: "_I_ read them")
Errata: I think most people don't read revision comments anyway.
 
@poke I see. :p FWIW, I did study French in high school, but that was several decades ago, and I don't remember "imprévisible". And I didn't bother Googling, or checking if the OP's location was in their profile. So your cheeky revision comment is entirely justified. :)
 
:D
 
Okay, getting way ahead of myself here, but if I moved to the UK, what car should I buy?
Relatively comfortable for 2 hours of driving a day; got to fit infant in it easily. Preferably no SUVs.
 
12:41 PM
 
@Kevin I hope the OP reads revision comments. And I sometimes read them, especially if I suspect that someone's done a bad edit.
 
12:55 PM
Cbg.
 
Greetings
 
Ahoy
QA Engineer walks into a bar. Orders a beer. Orders 0 beers. Orders 999999999 beers. Orders a lizard. Orders -1 beers. Orders a sfdeljknesv.
 
Saw that this morning. Just terrific :-)
 
God damnit Kevin! Not again
 
@Kevin excellent, thanks :)
 
12:58 PM
It's reddit's fault, Jeff's blog post is #1 in r/programming today.
 
I need to reddit :/
 
On second thought, let's not go there. 'Tis a silly place.
Wow, google delivers:
 
Amazing
 
If you guys don't mind can I ask another stupid question
 
Proceed.
 
1:09 PM
import timeit
f = open('test.txt', 'w')
for i in range(1,10):
	f.write(str(i))
f.close()
def d():
	rf = open('test.txt', 'r')
	a=set(rf.readlines())
def e():
	rf = open('test.txt', 'r')
	a = set()
	for l in rf:
		a.add(l)
print timeit.timeit(d,number=1000)
print timeit.timeit(e,number=1000)
(Canopy 32bit) C:\Users\MOB140003207\Desktop>python test.py
0.0412240925503
0.0319766251574

(Canopy 32bit) C:\Users\MOB140003207\Desktop>python test.py
0.0197786053277
0.0593246171124

(Canopy 32bit) C:\Users\MOB140003207\Desktop>python test.py
0.339515459866
0.306873901767

(Canopy 32bit) C:\Users\MOB140003207\Desktop>python test.py
0.429007384605
0.400539505591

(Canopy 32bit) C:\Users\MOB140003207\Desktop>python test.py
0.197393392876
0.200330646654
Gives me different results
But if I increase the number then my system just hangs
 
Maybe you ran out of file handles. What happens if you do rf.close() at the end of both functions? Will it still hang?
 
this feature is so dumb :| You can load settings, but you can't set default settings, so it's annoyingly unreliable
 
No it dosen't
It is working
(Canopy 32bit) C:\Users\MOB140003207\Desktop>python test.py
3.17401225086
3.68806420872

(Canopy 32bit) C:\Users\MOB140003207\Desktop>python test.py
41.7543653723
36.8978760992
 
@VigneshKalai for the record, next time you decide to post so much code can you use a pastebin instead?
 
FIrst one was for number =10000
 
1:17 PM
You've pretty much filled an entire page with your code :/
 
@Ffisegydd sorry will do that
second one was number=100000
 
I am so p*ssed off with this windows laptop at work I might throw it out of the window rock star style. Something out of the 40 (yes 40) automated updates this month has bust the wireless authentication in some subtle way. Why, oh, why, oh why? hereendeththerant
 
I know that feeling.
 
But still I cannot come to a conclusion which is faster
 
Prepare for sympathy image.
 
1:19 PM
And sorry for asking can the room owners remove the code
 
I do now feel better. Thank you. I am going to recharge caffeine to prevent job-losing vandalism.
 
@VigneshKalai Maybe if the file you read was much bigger, the difference would be more apparent. Try changing the first loop to for i in range(10000):
Ooh, and write a newline in between each number.
As-is, I think your file only contains a single line, so e() only iterates once anyway, making it very similar to readlines
 
Yeah now I can see the difference second one is slow then first
@Kevin thanks again and sorry for wasting your time
 
DSM
'orning 'abbage.
 
It's hard to waste my time, because if you hadn't asked anything I'd just be wasting it somewhere else ;-)
 
1:26 PM
Hey up
 
DSM
So it looks like there's a 50-50 chance that my Python+aurelia approach will be selected to replace a colleague's Java code. "hey what if I this is neat" ideas feel less comfortable when the threat of production-level responsibilities looms.
 
This'll learn you not to do things.
 
@DSM It will be fine, as long as you got all of these right.
 
Turn enough of it into Python and you can get a budget to hire all of us
And make sure the Windows logo requirements are met!
 
DSM
1:42 PM
Everything's running under Linux, so Redmond will just have to learn to like it..
 
Weird. I can't reproduce the OP's bug with .decode('windows-1251'). My code's not identical to theirs, since I'm using Python 2's urllib2 Request, not Python 3's urllib request.Request. But both the .read() methods return byte streams (allegedly). stackoverflow.com/q/31723929/4014959
 
DSM
@PM2Ring: like you, I see the error with utf-8 and do not see one with windows-1251.
 
@DSM Should we close it with "can't reproduce"?
And maybe post a comment to try the 3rd party requests module?
 
@PM2Ring I think what's going on there is that the OP knows the webpage is in windows-1251 from reading the source, but didn't know that could be passed to decode
They don't actually say they hit an exception using .decode('windows-1251'), just that the webpage has that encoding.
 
DSM
@PM2Ring: dunno. Since it's an encode which is throwing the error, if we believe his error message when he tried to decode with 1251, it's probably the print which is throwing, no?
 
1:57 PM
@DSM Good theory. I wonder what's the encoding of their terminal.
 
Hmm Decode throwing error in the programme in the OP, Encode in the comment. I think you're right @DSM. Needs a traceback to isolate the line...
 
DSM
If I encode to cp850, I get
UnicodeEncodeError: 'charmap' codec can't encode characters in position 127-133: character maps to <undefined>
which isn't the same but is close enough.
 
@DSM Are you doing that cp850 encode after first doing .decode('windows-1251')?
 
DSM
Yeah, to mock up the print.
So my working theory is that the original decode failed because it wasn't the right encoding; when the OP used the right decoding, the print failed because the terminal encoding doesn't support it.
 
ok, so now I have to persuade important people that the performance characteristics of sleep make it worth it to do it in the thread. Suggestions?
 
2:08 PM
@DSM Ok. I'll ask the OP what encoding their terminal is using. And I probably should ask what OS they're doing this on. And hopefully we'll get an answer. :)
 
@AaronHall data talks - can you fork the code, change it and record performance of each?
 
DSM
Meeting is about to begin, so I'll leave it in your capable hands. :-)
 
Yeah, but I'm thinking of trying to be as tactful and unabrasive as possible. And you all know how abrasive I can be.
 
@AaronHall I thought your while vs sleep test was just a theoretical exercise. Are you telling us that your colleagues would seriously defend the use of a while busy loop over a sleep call?
 
What if I'm the one being tested?
 
2:14 PM
Being tested about what?
 
It's not paranoia if you think they're out to get you!
 
Just because you're paranoid doesn't mean they're not out to get you.
 
Yeah that's...sort of what I was parodying :)
I should've made it more similar
 
I assumed you were, that was for the benefit of any ? people out there who haven't heard of it.
mutters a prayer for the unwashed masses
 
2:47 PM
Come on people, I'm excited about possible UK job and want to dream about cars :)
 
I created the trap myself. There was a call to sleep in the suggested while loop. :( I don't know how I missed it. :D Well I do know, I saw the comment in a mangled email and didn't look directly at the review request.
 
That's a faboo car.
 
That's more like it
 
Although I think we forgot the "needs to fit a baby in it" requirement :)
 
@IntrepidBrit love the license plate on that one.
 
@MartijnPieters It annoys me a little. Double double oh seven. grinds teeth
 
I was going to say I'm going to save that in OneLote, when I realised there's a much more obvious IT pun involving Lotus and Notes
 
Haha, indeed
xD
I love me a bit of a Lotus
Pity it's not Chapman run any more :(
 
2:51 PM
No no, the owner is trying to imply that he is twice the man Bond is. A double double oh seven.
 
If we're on that sort of car, I like the McLaren 675
Man us Brits can make nice cars. Just don't ask us to mass-produce them or really make half the bits that go in them.
 
This is still my dream car.
 
I like a car with more than a half inch of ground clearance, so I don't scrape my undercarriage on the slightest imperfections on the road, or get wedged on the bottom of a slope.
 
But with whitewall tires and gold trim instead of silver.
 
@RobertGrant Or not make them smell of glue
 
2:54 PM
Yeah, glue is a problem
 
@RobertGrant That be swank
@MorganThrapp What is that?
 
@IntrepidBrit 50s caddy.
 
Although we make the glue out of horses, so that when we drive after foxes we can claim it's foxhunting, and traditional.
 
I can buy a model car at the hobby shop that snaps together with no glue. Why can't we do that with full sized cars?
 
gar... people are so needy :|
 
2:55 PM
@IntrepidBrit Specifically it's the Mary Kay car.
 
@Kevin Let me get right on that. First! I need a bigger 3D printer. Donations welcome.
 
@IntrepidBrit "Please, sir. May I have some glue?"
 
You sir, won a star
 
Aw, thanks. :) It's all part of what I've ever wanted.
 
Oh, I was going to buy you that car. But if you're happy with the star, then that works too I guess
 
2:57 PM
Please, sir. May I have some glue? - surely?
 
@RobertGrant It's been too long since I've seen Oliver Twist.
 
Hey cool - didn't know you could edit it and the star would update
Make sense I guess
 
00:00 - 15:0015:00 - 00:00

« first day (1748 days earlier)      last day (3198 days later) »