« first day (2356 days earlier)      last day (2591 days later) » 

wim
10:09 PM
@nbro don't use assert statements to check inputs
 
I must
since Python is damned slow
and assertions can be disabled
 
wim
if you need maximum performance then don't check inputs at all, but I find it hard to believe this can be the slowest part of any sensible code
 
...that's why they're making it disableable via an assertion
 
that's why I'm using assertions, indeed
 
It's ugly, but assert all(x.shape == (3,) for x in (g1, j1, g2, j2))
 
wim
10:14 PM
g1.shape == g2.shape == j1.shape == j2.shape == (3,)
 
not that ugly ;)
 
52 chars versus 86
 
:D
 
wim's is 59 but perhaps clearer
 
ahah
 
wim
10:15 PM
what is this, code golf?
 
Since the question was "is there a way to shorten this", more or less, yes.
 
I'm never sure if I can do certain things in Python, but usually you can, but I prefer to ask you for a confirmation
 
It's always shorter to just try it yourself
 
I didn't think about those options actually
 
wim
"Python is damned slow" is a dubious claim
 
10:17 PM
if you program in many different languages, you end up forgetting the details
 
Try Python 2. I hear it's faster sometimes. And using it will annoy the room.
 
@wim It's probably the slowest programming language I've ever programmed in, so...
I would like Python compiled, that would be nice
 
There are PyPy and cython.
Have you tried those?
 
wim
it's the fastest, if you factor in development time ... :P
 
yeah, but not all libraries can be used from those implementaitons
 
10:19 PM
But you don't need 'all' libraries. You just need the libraries you need for whatever it is you're trying to build.
 
wim
complaints of python's slowness usually come from bad developers
 
@wim ahah
;)
as you wish
 
Python is the slowest? Have you tried ruby yet?
 
I'm not a perfect developer, for sure, but Python is the slowest programming language I've programmed in (I repeat), mostly because it's interpreted and is too high-level
@KevinMGranger No, but I guess they are at the same level
 
Python is compiled and executed in a bytecode VM, just so you know. If it's too high level, why are you using it?
 
10:23 PM
@KevinMGranger Python source code is not always compiled
 
When is it not?
 
if you execute a single file without dependencies, I guess not bytecode is generated
but I may be wrong
 
you probably are yup
 
I don't know enough about the interpreter
anyway
who cares
 
at least my impression is that the fundamental workings of the interpreter is to interpret byte code
 
wim
10:25 PM
you are wrong
 
Python is slow compared say to C, Java, C++, Scala and even Haskell, etc
 
indeed
I've just checked
sorry, nbro
 
@AndrasDeak What?
 
wim
about that
> if you execute a single file without dependencies, I guess not bytecode is generated
bad guess
 
we're both wrong?
 
10:27 PM
For many cases, sure. Slow enough to be a problem, in most cases? Depends on your workload. If your workload is truly this CPU-bound, there's ways to deal with that in Python, alternative python implementations, native extensions, or going with a closer-to-the-metal language
 
wim
deak you are not wrong, for once
 
cough numpy cough
 
for me it's a joke that Python is one of the most used programming languages in computational science
 
wim
looking at .shape == (3,) i would guess this is already numpy
 
cough numpy cough
 
10:27 PM
@wim let me just print and frame that and hang it above my bed
 
wim
also the comment about pypy and cython hints the same
 
@nbro for computational science it's not slow at all, listen to paul's coughing fit
 
If it's slow even with numpy you're using it wrongly - remember to move the heavy lifting to numpy
 
who said it's slow with numpy?
 
I confess that I'm not really paying attention once it's about "nyeh nyeh python is slow" :P
 
10:29 PM
I sped up a function once by 15(!) times just moving the loop-calculations to native numpy slices + calculations. (So the looping is done in C)
 
What exactly is slow? Where is the actual program you can point to and say 'damn this is slow'.
 
I said it's a joke that an interpreted not so performant programming language is used for computational science
 
wim
your comments are inane
 
lol
 
wim
it's a high level language, and the trade-off for the pleasure of working at a higher level of abstraction is performance
 
10:30 PM
But it is fast - at least fast enough for computational science. (Remember, you hardly ever need real time calculations you CAN wait a few minutes on difficult things)
 
when you encounter someone that is so attached to a programming language the conversations end up being surreal where the defenders always try to support the defects of the mentioned language somehow
it's just a tool
who cares
!!
 
wim
people who complain that it's slow don't know how to write efficient code, and do things the fast way, they don't even know how to identify which parts are performance critical
 
Have you noticed what room you're in lol
 
And for computational algorithms it is even more important to be able to quickly make/adapt your program; the amount of times a function is called is way lower than for application based programming.
 
wim
ergo bad developers
 
10:31 PM
guys
 
my issue is that since you're using numpy, a lot of the code you should be writing would use compiled code under the hood
 
Oh here, I'll just pop in the python room and talk about how bad it is at something tons of people love using it for. Oh, people are disagreeing with me? who cares lol!
 
wim
they write naive code as if python were the same as C or java or whatever language they came from, and then complain that python is slow
 
Like, I don't get how people are surprised when this happens
 
You know: you always have to consider "just calculating by hand" - that's also a valid approach for scientific calculations - if programming is too hard you do it by hand since programming a tool to compute things would be slower.
 
10:32 PM
have you ever implemented a numerical algorithm?!
probably not
so do not even talk
 
-.-
 
Have you ever had to allocate funding to multiple scientists in a time-constrained environment
 
@nbro are you talking to me?
 
1+2+3 is a numerical algorithm. I win.
 
don't flag that
 
10:32 PM
I'm talking to everyone who's never used Python for computational science
 
lol:D
so much for the educated discourse you're missing
 
Nah, "be nice" is a rule of the site. If you can't follow it, get out.
 
wim
@nbro yes, I worked on real time numerical algorithms for many years
 
"a numerical algorithm", that's really broad.
 
@KevinMGranger yes, but flagging for rudeness will lead to a 30-minute ban
 
10:33 PM
@wim yes, where?
 
"get out" is RO stuff
 
@AndrasDeak of me or them?
 
them:D
 
flagging for rudeness?
 
wim
in a previous job
and now I work at a high frequency trading firm, where performance is king
we use python almost everywhere
 
10:34 PM
@nbro if you really want to turn this into an appeal to authority, I'm a computational physicist. Does that count as numerical algorithms?
 
I'm using python daily to calculate things for university work. Things like dynamic stability analysis I often use python for.
 
wim
the parts where python is not fast enough are few and far between, and even then it's just those critical parts that get implemented in C++, with python bindings
 
nbro, if you had just come in asking "my python program is too slow for what I'm trying to do, how can I speed it up?", you might have been greeted more warmly. I suppose it's too late now.
 
But I feel like you are thinking about this from the wrong perspective: python is never "alone": python is "batteries included". You shouldn't consider python on its own for these tasks...
 
@KevinMGranger what? why are you inventing things that I've not said at all?
 
10:36 PM
Well, nbro is not interested in friendly discussion, because they didn't come here to discuss speed. It was just an aside which grew into a "discussion"
considering their stance and attitude this is almost guaranteed to stay on the level of trolling/flaming
 
ok
 
so I suggest we go back to actual, objective python-related questions
 
If you really wish to stay native python, then yes complex algorithms on large datasets is slow: but that's not how anyone uses python.
 
you're so serious
gee
 
10:38 PM
thank you
 
wim
we may browse nbro github here github.com/nbro/ands/tree/master/ands
 
the only thing/facts I wanted to transmit you regarding Python performance is the following: 1. Python is slow compared to many other languages, and because there are many other languages with a lot better performance in most of the cases (which are also high-level languages), for me it was a bad decision to invest on Python in the field of numerical computing and computational science, etc
but it turns out that Python is good enough for many situation
s
happy?!
 
I'm never happy
 
Same, I'm empty inside
 
it really isn't about you saying bad things about python
 
10:41 PM
But I guess that's besides the point
 
lol
 
self.shape == None
 
wim
@nbro much better!!
@nbro also correct!
 
@wim you can browse it, copy it and use it as you wish (MIT) lol
 
wim
bravo, you have seen the light
 
10:42 PM
Yay friendship
 
Though remember, "use what your colleagues use" - Science is about transmitting ideas not about what language or how fast it is.
 
wim
for fast numerical arithmetic stuff in Python, you might be interested to look at github.com/aleaxit/gmpy
 
I'm sure there's a balance of funding, colleague-knowledge, availability of computing resources, and developer time somewhere but that's way too much stuff for me to graph when I'm not even in the sciences
 
wim
it might change your mind about Python being a "bad decision" for numerical computing
 
I'm actually quite happy we moved away from matlab towards python: python allows for much "cleaner" or "completer" code/applications.
 
10:44 PM
well, MATLAB also used to be pretty slow :P
 
I can now use the same language for things that "make life easier" and "compute/plot things for reports".
 
I'd be excited to see Rust catch on more in the scientific community, but it's parallelism story is still a bit young
 
by the way, if some of you is familiar with the numdifftools package and you have time to have a look at this question stackoverflow.com/questions/43104117/…, I would appreciate it ;)
 
@nbro please read the room rules, which among other things asks you not to link your fresh questions here
 
oh, why not?
 
10:53 PM
because it's written in the room rules :P
 
omg :D
 
Mostly because whoever can and will answer your question will already see it on main, and badgering people here is unnecessary. Furthermore, parallel discussion here and there would lead to image fragmentation. At least these are my impressions on the matter. But the bottom line is the rules page.
Did I seriously write "image fragmentation"?
I meant information fragmentation
I am not well-rested today
 
for those of you who are interested in AI, ROS is an "operating system" written in Python to be used in robotics, and is actually also used by NASA, if I'm not wrong to implement their last humanoid astronaut robonaut2: robonaut.jsc.nasa.gov/R2
there's a really interesting youtube video where the fellows at NASA present r2
 
"R2", real original name for a droid
 
lol
 
11:08 PM
S L E N D E R M A N I N S P A C E
 
ahaha
 
slenderman with a snapchat filter
 
wim
11:31 PM
is there a nice way to split a list into n sublists? nice = not itertools
 
that's racist
 
wim
eh?
 
use len(lst)/n, apply the "every m items" trick and zip them together?
or just a list-comp from slices?
 
wim
in code pls
 
bit late for that but I'll try:D
 
wim
11:35 PM
e.g. if len(L) is 54 and n is 4
then we should get chunks size 14,13,14,13 or similar
 
those are not commensurate:P
in that case you should use itertools
my first idea needs a zip_something_something from itertools to work
 
wim
of course they're not, otherwise the zip - splat trick works
 
why do you want to avoid itertools in the first place?
 
wim
no solution with itertools is nice
 
I see
 
wim
11:38 PM
it's a n00b library for making python code unpythonic
</opinion>
 
I'm not sure there's an elegant solution, considering that your sought-for chunk sizes are entirely arbitrary
 
I used Petzold's savage benchmark with 10 million repetitions to test which language implementation is faster.
Python 2.7: 9.7 s
Python 3.4: 12 s
C++ (g++): 3.8 s
C (gcc): 3.7 s
Tcl: 410 s

At least with this example Python is not bad. Tcl, my other favorite general purpose scripting language, is quite slow at this calculation. But Python loses some precision for some reason. I expected the result 9999818.4485011194 but Python gives me 9999817.515947713.
 
wim
@AndrasDeak they are not entirely arbitrary
 
from __future__ import print_function
from math import tan, atan, exp, log, sqrt

def do_savage(n):
	a = float(1)
	for _ in range(n):
		a = tan(atan(exp(log(sqrt(a*a)))))+1.0
	print(a)

do_savage(10000000)
 
wim
it's the best integer approximation of evenly distributed chunks
 
11:47 PM
@wim I know, but still. Lot of room for assigning those
in this case it's easy since you have half-integer congruity
but what if your len was 4.12341245 times the number of chunks?
 
wim
heard of divmod ?
 
how could I not, it's all you speak of :P
 
wim
as long as the longest chunk is at most 1 element longer than the shortest chunk, any distribution is acceptable
 
55
Q: splitting a list of arbitrary size into only roughly N-equal parts

user248237dfsfwhat is the best way to divide a list into roughly equal parts? for example, if I have a list with 54 elements and i want to split it into 3 roughly equal parts? I'd like the parts to be as even as possible, hopefully assigning the elements that do not fit in a way that gives the most equal part...

 
wim
divmod trivially proves that that's possible
nice
I think I have it... we use the skip in a slice, then we use the zip-splat trick
 
11:53 PM
that's what I'm working on right now
 
wim
the trick being that you divide into len(L)/n chunks, and then transpose
 
I'm getting an off-by one, I have 3 chunks instead of 4
>>> t
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53]
>>> list(zip(*(t[k::len(t)//4] for k in range(len(t)//4))))
[(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12), (13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25), (26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38), (39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51)]
there's also the thing with the last 2 elements missing
probably need a ceil or something
 
wim
zip will ensure you have evenly sized chunks
so that's kinda a problem
 
so...itertools.zip_longest?:P
but then need to filter I guess
 
wim
if you say itertools one more time .........
 
11:57 PM
*wonders what will happen*
 

« first day (2356 days earlier)      last day (2591 days later) »