« first day (2018 days earlier)      last day (1217 days later) » 

8:30 AM
has any pythonian here ever used numba.pydata.org
 
 
2 hours later…
10:13 AM
@flawr Yup
 
 
2 hours later…
12:27 PM
@AndrasDeak do you think it could speed up numpy computations? (considering np uses c libraries already)
 
@flawr a single numpy call no. Multiple calls maybe. And allegedly writing naive loops might end up faster with it (I've never tried). I have numpy-heavy code that got faster with it. Also look at pythran: I could replace the numba jit with it and it was even fastee by a bit. But pythran is really annoying.
Like, Titanic quotes with errors. Very cutesy. ":3 uwu" almost
you should separate the heavy bottleneck into a function that takes arrays as inputs
this was pythran:
$ pythran energies.py
WARNING: Compilation error, trying hard to find its origin...
WARNING: Nop, I'm going to flood you with C++ errors!
CRITICAL: Cover me Jack. Jack? Jaaaaack!!!!
[g++ failed message snipped]
reaaally annoying
but once you get it to work it shuts up and it's fast
But it definitely helped that I did the numbification first. They have similar requirements with pythran, but it's a lot more helpful telling you what's wrong.
 
12:45 PM
@AndrasDeak well I guess it is funny for the first two times
 
no, it wasn't funny once :(
but I'm probably too old for this shit
A function made faster by numba/pythran (without distracting comments, mostly just the code lines):
def _compute_energy_change(ki, jijnow, isbij, bijnow, spins, neighbspins, nbulk, specinds, newdir):
    # newdir is shape (3,)

    # get special spin orientation, shape (3,)
    spinnow = spins[specinds]

    # neighbspins is shape (npairnow, 3)
    # jijnow has shape (3, npairnow, 3)

    ki = ki[specinds[2] % nbulk, :, :]

    dE_on = ( np.dot(np.dot(ki, newdir), newdir)
            - np.dot(np.dot(ki, spinnow), spinnow))

    fact_jij = - jijnow.reshape(3, jijnow.size//3).dot(neighbspins.ravel())  # shape (3,)
those .dot calls are not @ to appease numba or pythran, for instance
 
that function was still almost completely made from numpy functions, right?
 
yup, specinds is a 3-length tuple (and nbulk is an int) but everything else are numpy arrays
 
I wonder how it compares to libraries that make use of GPUs
 
And for single expressions there's numexpr which can help a lot by reducing memory. These things often achieve speedup compared to numpy by not creating every temporary array (not sure if numba/pythran are smart enough to do that)
My needs are different so I've never used numexpr
@flawr so I'd just suggest 1. profiling your code to find the actual bottlenecks, then 2. just try and see what numba does with it
 
12:53 PM
@AndrasDeak but that would require work from my side!
@AndrasDeak that'd be interesting, I think there is not much optimization going on there with just python+numpy
 
Oh, and I saw github.com/fluiddyn/transonic being advertised that seems to be a front-end for various acceleration backends (such as numba and pythran). Never tried it.
@flawr Hmm? What do you mean?
res = a * b * c * d will very explicitly compute 2 temporary arrays as well as the final result.
 
oh wait, numexpr is numpy?
 
Not sure about affiliations, but it does numpy
pydata is a separate player that's close to the numpy ecosystem
pydata/sparse is often mentioned in certain contexts, and xarray is also A Thing
 
1:10 PM
I didn't know about pydata!
they do seem to have quite a few other interesting libraries
ok not all of them are very active anymore
thanks a lot for all the suggestions and explanations!!!
I guess my debt is increasing to a coffee and a piece of black forest cake:P
 
1:45 PM
@flawr Sacher or bust
 
this can be arranged
are there any cake delivery services?
I can also make one and mail it to you, but I'm not sure anyone would still want to eat it then
 
Lol
 
 
4 hours later…
5:22 PM
@flawr What do I have to do to get one of those? I can come pick them up myself, without having to quarantine :P
 

« first day (2018 days earlier)      last day (1217 days later) »