« first day (1938 days earlier)      last day (1274 days later) » 

4:44 AM
@CrisLuengo thanks! Although it's not a new job, just something like an internship I do at my spare time
 
 
4 hours later…
9:07 AM
I have a question about MATLAB->python code translation. Suppose I have a 2D cell array of function handles (called calib), and now I want to evaluate all of them for a given input (x(3); the functions accept a single scalar parameter) to get a numeric matrix of the same size as a cell array. In MATLAB I'd do cellfun( @(v)v(x(3)), calib). How should I do this in python?
 
what do you use in place of the 2d cell array?
 
probably with a loop
 
or list comprehension
 
@flawr no idea
there are several problems here: 1) how to store a bunch of "function handles"; 2) how to evaluate them all on-demand
 
the only thing similar to cellfun is map, but other than that python is not very functional
 
9:11 AM
The functions may be hardcoded in the script file... let's call them fun_11,...,fun_mn
 
you can take pretty much any list or list of lists or even a numpy array to store functions
 
yeah but what's the best practice?
 
well I guess the hardcore pythoneers don't like seeing you making collections of functions:)
ah I just remembered that numpy has a nice vectorize
import numpy as np
l = np.array([lambda x:x+1, lambda x:x+2])
out1 = np.vectorize(lambda f:f(3))(l)
print(out1)
which generalizes nicely to multi dimensional np.array()'s
 
I see, that's nice!
 
wait till @AndrasDeak sees it and gets a heart attack
 
9:17 AM
I was fumbling with something like:
x = np.array([0.1, 1, 200])
fun_mat = np.array([np.exp, lambda m:m+2])
y = [ fun(x[2])) for fun in fun_mat ]
but the last line seems wrong
 
that is also perfectly fine, but you probably need two loops if you want a 2d structur of the results in the end
 
why not just reshape it before/after?
 
you can do that too:)
in haskell it would be very easy as it is just colllection_of_funcs <*> collection_of_args :)
 
@flawr Are you trying to sneak in some lambda calculus through the back door again??
 
that I already did in the previous messages, this here is just about my daily ad for haskell:)
 
10:10 AM
@flawr collection of functions is perfectly fine!
there's even a name for a dispatch dict
@flawr it's not nice at all
 
@AndrasDeak is that something I should be using...?
 
if a dict makes sense then yes; since you're fine with array indices a nested list of functions might work
applying them is another matter, of course
I don't think there'd be a better way than a nested list comp: res = np.array([fun(val) for row in fun_arr for fun in row])
 
@AndrasDeak why not?
 
@flawr it fools newbies into thinking that it does anything else than use a python loop. It only makes a nicer API, but it's terribly slow.
when people talk about vectorization in numpy it's absolutely not what np.vectorize does
 
but I don't think it is masively slower than just a loop or list comprehension, no? but I see what you mean
I would not hae expected it to do anything more than a fancy loop, considering how numpy arrays are built
 
10:15 AM
@Dev-iL typical use case for a dispatch dict is something like this: ops = {'+': operator.add, '-': operator.sub, '*': operator.mul, '/': operator.truediv}; res = ops[op_char](*operands)
@flawr it's exactly as slow, unlike real vectorization
@flawr you would not, but most users and most newbies are not you
it's a misleading function
 
well I don't think it is necessarily missleading, I think the key problem is that some people get taught (or maybe it is the only thing they remember) that vectorization is needed at all costs and always better but don't understand why or what it actually means.
 
it actually is always better... if you have the memory :P
 
maybe a np.map() would less misleading
 
@AndrasDeak "SIMD"
 
@flawr but it converts a function into a function
neither map nor apply do that
not sure if np.vectorize_API would be enough to dispel false expectations
 
10:22 AM
calib_at_T = np.vectorize(lambda f: f(x[2]))(calib)
calib_at_T = np.array([fun(x[2]) for row in calib for fun in row]).reshape(calib.shape)
first one seems cleaner, no?
 
yes indeed
 
I'll think if I can come up with something decent
 
@AndrasDeak ain't nothing wrong with functions, yo
 
you can at least replace the reshape if you build a nested list: [[fun(x[2]) for fun in row] for row in calib]
@flawr but neither map nor apply return functions if the operand returns a number
 
@Dev-iL it seems from a speed point of view they are roughly equivalent: stackoverflow.com/a/46470401/2913106
@AndrasDeak just view it as a curried map:)
 
10:26 AM
I'd be tempted to define a class with callable instances that does the second version under the hood
 
@AndrasDeak but considering the speed it shouldn't really matter what you use, so why not use the built in np.vectorize() that allows for concise readable way of expressing it? Wouldn't making an own class just reinvent the wheel?
 
class Calib:
    def __init__(self, calib_functions):
        self.functions = calib_functions
    def __call__(self, *args):
        return np.array([[fun(*args) for fun in row] for row in self.functions])
@flawr depends on how often you want to do this operation. If exactly once then yeah, not much point in defining a class for it. If you want to call these functions in multiple places in the code it might make sense. Calibration functions might also have additional state that could be carried by the class, once you have it.
The one I wrote will also work for array-valued inputs (returning 2+ dimensional arrays). I don't know if and how vectorize handles that; I have no intuition about its API.
(but again you don't need the class for that)
 
Ok so next question
suppose a function requires I pass it a callable, but I need to pass a callable to accept extra arguments... do I wrap it in a lambda?
 
10:42 AM
Yes, or use functools.partial if that works
 
@AndrasDeak if I "freeze" certain inputs, will the partial, when called, redirect inputs to the "free" parameters?
 
no, if you want closure variables you need a lambda or other explicitly defined function
assuming that's the right jargon :D
 
hmm
 
actually, no, I'm being silly there, hold on
there, edited
no, that's not it
let me put together a proper example that runs, just have to go afk a bit
 
so suppose I want to pass somewhere a function inside which calib is evaluated at some x (where x is a variable and calib is constant), I'd do: new_fun = partial(old_fun, calib=calib), then can call new_fun(x)?
 
10:52 AM
No, that would need old_fun to have a kwarg called calib. If that's the case then sure.
 
ok and if calib is a positional argument, should it be the first or the second?
for new_fun = partial(old_fun, calib) to work?
 
You can only fill positional args from the start. But you can access positional args as kwargs unless you explicitly forbid it.
So if you have def old_fun(arg1, calib, arg2) then you can't do partial(old_fun, calib), only partial(old_fun, fixed_arg1). But you can do partial(old_fun, calib=calib).
 
so if old_fun expects 2 positional args (def old_fun(calib,x):, and I create new_fun = partial(old_fun, calib), would new_fun(x) work...?
 
yup, it would be equivalent to old_fun(calib, x)
 
ok, great
thanks for the explanation!
 
10:58 AM
No problem :) But this is also a legit use case for lambdas.
 
btw, @AndrasDeak, do you have any relation to quantum computing?
 
none at all
 
11:27 AM
argh.... when indexing a 2d array, how do I keep the information that I grabbed a column or a row? It seems that I'm getting those generic "1d" vectors, which messes up my concatenation later on
 
@Dev-iL yes, that's how numpy ndarrays work :)
you can either inject a singleton dimension yourself, or use something other than concat, or use the matrix class (no no no don't do this :P)
What is your use case that you can't tell if you're grabbing rows or columns?
 
I know what I'm grabbing, python doesn't
 
I mean you can use something other than concat
 
so let's say I have a 4x4 array, I grab the second column, then need to concatenated a column of -1, then some other column vector
 
>>> x, y = np.random.rand(2, 3)

>>> x
array([0.36686404, 0.63395152, 0.50658574])

>>> y
array([0.32542886, 0.0700521 , 0.43093115])

>>> np.stack([x, y]).shape
(2, 3)

>>> np.stack([x, y], axis=-1).shape
(3, 2)
 
11:30 AM
ok, that looks like what I need
so what is the use case for concatenate?
 
I think it might be possible to do whatever it does with stack, but stack is a lot newer. And there might be subtleties with multidimensional arrays. I'm not very well versed in these things.
 
ok, I'll keep my eyes out
 
11:57 AM
 
@AnderBiguri myea, but walking uphill with such a large rucksack is rather tiring
 
cable car I suposse
they live up there, right?
 
Some do, I live only 50m above the lake, and my work is 50m above the lake on another hill :p
 
paraglider+big fan
 
Or a bicycle
 
12:01 PM
fair, that is your home country scale
 
Except that 50m altitude difference is rather a challenge to achieve on a bicycle back home. ~10m is doable by going on viaducts, dikes etc
 
yeah. I am a bit upset that London is not really bike friendly. Such a huge and flat city...
Lots of people use bikes, but you basically need to go with the cars in the city with the angriest drivers Ive ever seen
 
@AnderBiguri come to Zürich, where everyone seems to be able to spare 150.000$ to buy a ridiculously large SUV, doesn't know what a blinking light is, and hates cyclists. Oh, and there's no cycling infrastructure (lanes, let alone separate) to speak of.
 
yeah in London is not very different. You get bike lanes sometimes, but most of the times is either old canals or just road
I could cycle to work in 30 mins, but I need to take essentially a city highway, which I am not going to do with a bike XD
 
The "Zürih autofree" initiative was blocked by the judge for being too radical to people not living in the city (it was envisioned to essentially bar all private cars from entering the city, thus mainly hitting those not living there and driving in every day), but last month the "For better bicycle infrastructure" initiative did pass! That means that they're now going to build 50km of essentially car-free bicycle fast-lanes.
 
12:08 PM
nice!!
In London the problem is mostly that the city works like 36s different separate towns
 
Which IMO is great for everyone. Great for the cyclists, great for the car drivers who don't have to constantly watch out for cyclists, great for people living on the converted roads, since they will be allowed to use their car there, but will have a generally quieter neighbourhood etc
@AnderBiguri and it's insanely huge
 
each Borough has a say in what happens in their borough, and try to get 36 politicians to work together
most block comes from people from Westminster Borough, because they want to drive their expensive cars without being bothered
@Adriaan yes, but basically you go on a bike lane, and it ends in the middle of a road suddenly, because its the border of the borough
@Adriaan 100% agree. I wish I could go to work by bike (or be brave enough to to it in the mornings)
not that going to work is something that wil happen anytime soon for me :(
 
@AnderBiguri hehe, we're often forced to cycle on the pavement at tram stops. Cars can remain (waiting, if necessary) on the tram lane, but cyclists have to, and are fined if they don't, go on the pavement. Which is a pretty big problem and pain in my behind. For my previous flat I had one of those on a large, secondary railway station bridge. There'd be 80 people or more waiting for their tram, and I was forced to somehow cycle between them, rather than follow the cars over the tramway.
And the truly stupid part is that it really only concerns the tram stop. From ~2m before to ~2m after you should cycle on the pavement, otherwise on the street :s
 
crazy stuff, that is def not bike friendly XD
 
Yup.
Downhill you're usually allowed to stay on the tram lane, together with the cars, but uphill you'd probably slow the tram down too much
 
12:15 PM
@AnderBiguri all I know about london is from this guy youtube.com/watch?v=_DNNIB_PdaA
 
Traffic planning in Zürich goes: Public transport > cars > pedestrians > anything else only if there's space (which there usually isn't)
 
@flawr there is nothing more that you need to know, hes the best explainer
 
@AnderBiguri would be really cool to live somewhere like that:)
 
My feeling about cycling in london is the first few seconds of this video: youtube.com/watch?v=gohSeOYheXg
 
@Dev-iL stack() creates a new dimension (= you need one more index after stacking), while concatenate retains the number of dimensions, and just glues the arguments together along an already existing dimension. If you concatenate two matrices you just get a longer/wider matrix, if you stack two matrices, you get a new dimension: if you index along this new dimension you get your matrices back.
 
12:22 PM
so using hstack or vstackwith 1d vector inputs will always produce longer vectors...?
 
it seems hstack and vstack have additionally a reshapign built in to make 1d inputs 2d.
 
I played with it now, one of them creates a long vector and the other a 2d matrix
 
yep that could make sense, but I prefer using stack(...,dim=x)+explicit reshaping as I can never remember which one is wich:)
 
of course... it seems completely arbitrary the way it works with those other stacking functions
 
@Dev-iL do you know the x[None,:] trick to reshape a shape (N,) to (1,N)? comes in very handy in this situation
@Adriaan what about helicopters?
 
12:30 PM
@flawr That's probably the department of Hochbau rather than Tiefbau ;p
 
you never know, I mean soon we will have self flying cars
 
@flawr If by "know" you mean "observed @AndrasDeak do it multiple times" then yes, if you mean whether it comes naturally to me - then no.
 
@flawr Is there an established name for the series \sum_{n=1}^\infty 1/n^r? For r>1 it converges (to Riemann's zeta). For r=1 it is the harmonic series (which diverges). Any name for the series with arbitrary r>1? "Generalized harmonic series"?
 
@LuisMendo I just know it as the zeta function (I mean for r<1 it also diverges)
 
posted on October 20, 2020 by Loren Shure

Recently we had a customer ask how to fill in NaN values in an image with a neighborhood local mean. My friend, colleague, and occasional blogger, Brett Shoelson, joins me today to show you several viable techniques.... read more >>

 
12:38 PM
or maybe you dirichlet series helps? (s(x) = \sum{n=1}^\infty f(n)/ n^x, where x is complex)
@Dev-iL I just like to think about the shapes: If x.shape = (u,v,w) and we want to insert a new dimension between v and w you just write x[:, :, None, :] (x[:, :, None, :].shape = (u,v,1,w)), it is quite visual
 
@flawr Sorry, my real question is something else. I have a result R that holds for a function f when f is O(1/n^r) for some r>1. I want to say "R holds whenever f decreases faster than <...>". But I cannot say "faster than 1/n", because 1/(n log n) decays faster than 1/n but it's not O(1/n^r) for any r>1. So, is there a name or a way to compactly say "decreases at least as fast as an inverse power of n with exponent greater than 1"?
 
@flawr no argument there, all I'm saying is that it's just not high enough on my list of go-to solutions at the moment
 
@Dev-iL ah ok:)
 
@flawr ah, right
 
@LuisMendo I'm not sure if there is a word for it, but you usually see expressions like O(1/n^{1+\varepsilon})
 
12:48 PM
@LuisMendo isn't that exactly the Riemann zeta?
 
@AndrasDeak the rieman zeta is the analytic extension of this function
(you can define it "nicely" outside of where this function converges such that it coincides with this function where it actually converges)
 
ah
so it is the Riemann zeta
 
aaaaahrg
 
😃 😃
 
@AndrasDeak zeta takes r as the input, and outputs the sum of the series. I want to say that a function decays faster than the series terms for some r
@flawr I think that's weirder than "at least as fast as 1/n^r for some r>1" :-D
There should be a name for this
 
12:54 PM
@LuisMendo so instead of the infinite series you want a finite upper limit, right?
I feel like one of the o-notations should convey that
 
@LuisMendo in number theory there are a lot of epsilons:)
 
@AndrasDeak No, I mean, the function f(n) = 1/n^2 +1/n^3 satisfies the criterion, because it is O(1/n^2). The fucntion f(n) = cos(n)/n (for example) doesn't
I'd like a shorter name for "decays at least as fast as 1/n^r for some r>1"
 
@LuisMendo Sure it does!
abs(cos(n)/n) <= 1/n, right?
 
Yes. So it is O(1/n). It is not O(1/n^r) with r>1
 
@LuisMendo oh, right, sorry, I focussed on the cos(n)
That makes it tricky. I was going to suggest "power-law" but the need to separate 1/n will probably mean that you can't phrase this concisely.
for what it's worth I don't think your "decays as fast as..." version sounds bad
 
12:57 PM
I feel that this criterion is so common that there should be a name. For example, instead of O(log(n)) you say "logarithmic increase". Instead of O(1/2^n) you say "exponential decay". Etc
 
inverse power law decay? :)
 
@AndrasDeak Yeah, I think I'll have to settle with that
@Dev-iL but that doesn't incorporate the r>1 requirement
 
the same for "subhyperbolic"
 
@LuisMendo it would be power-law, if it weren't for your r>1
 
@LuisMendo ... outside the unit circle
 
1:01 PM
@LuisMendo can you just say it is o(1/n)?
have to look up the definitions again
 
@flawr Good point. Maybe. The lowercase o() notation always confuses me
 
hey, that's my suggestion! :P
 
ah no that doesn't work
 
@AndrasDeak Ah, sorry. You are right. I didn't see that
 
well O(..) is "at most" and o(..) is "vanishes quicker than"
 
1:03 PM
But it doesn't work, like flawr says
 
typical bad flawr suggestion
 
1/ (n log n) would qualify as o(1/n), but not as {O(1/n^r) with r>1}
@AndrasDeak Hehe
 
in numerical analyis the o(1) expression is used for "converges to zero"
 
I think you have to be explicit
@flawr nooo. Bounded!
No?
 
f in o(1) literally means abs(f(n)/1) converges to zero
 
1:08 PM
Lowercase o() is tricky. I prefer O, Theta or Omega
But then you have to define them in the paper, as they can have different meanings
 
I was taught that a(n) = o(b(n)) if exists c, N such that a(n) <= c*b(n) if n > N (maybe |a(n)|)
with that definition o(1) means bounded
 
@AndrasDeak Isn't that the definition of O()? (Uppercase)
 
o̸̞̠̹ͅ (1) , o̷̦̘ (1)?
2
 
:-D
 
@LuisMendo that's for the lower bound if I recall correctly
OK, lunch time, be back later
 
1:12 PM
Enjoy your lunch!
 
1:25 PM
let it make your hunger o(1)
 
2:06 PM
@LuisMendo have you found anything? otherwise you could ask on math.se
 
Sorry, I'm still in a coma :) Won't attempt to untangle little and big O notation
 
 
2 hours later…
4:12 PM
posted on October 20, 2020 by Steve Eddins

Lately, I've been spending more time on MATLAB Central, and I'd like to encourage you to try out some of the resources there, if you haven't already.Have you heard of Cody? It is an addictive MATLAB... read more >>

 
@flawr Nice and informative vid. The last two minutes of advertisement suck though But I guess everyone should make a living somehow
 
all his videos are great
 
4:34 PM
@Adriaan I like those still better than many other ads:)
 
@flawr yea, they are easily skipped. But it's a bit silly to complain about having only 10 minutes, when after 8 mins you go blabbing about unrelated stuff for 2,5 more minutes
 
the semseter is starting to pick up
and therefore, most matlab questions are homework dumps of some sort
 
4:54 PM
@flawr I thought about that, but writing a good question there takes time. I think I'll settle with "decreasing as a power of $n$ with exponent greater than $1$"
Or maybe I should say "decreasing as the inverse of a power of $n$..."
 
 
1 hour later…
6:01 PM
@LuisMendo do you mind if I write a question?
 
 
5 hours later…
10:37 PM
@flawr Do I mind if you do my work? Of course not! :-D
 

« first day (1938 days earlier)      last day (1274 days later) »