« first day (2067 days earlier)      last day (1153 days later) » 

12:10 PM
I want to add a histogram contour to my scatterplot (where colors are already taken up), to show the e.g. 50th percentile of the data since there are a lot of points and some of them overlap (and it's not sightly to look at if they're plotted smaller)
would a good approach be using a 3d histogram, then use the bin heights in z and somehow re-plot it using mathworks.com/help/matlab/ref/contour.html ?
 
 
3 hours later…
2:51 PM
@user2305193 I think you are complicating your plot too much. It is “one plot = one story”, not “one plot = all my data”. A plot is about communicating your finding to your audience, not about demonstrating how much work you did to make that finding. What story do you want to convey with this plot? Plot just the stuff that you need to express that story. The simpler the plot, the easier it is for your audience to read the plot and understand your story.
 
3d plots sound great but they are very bad at conveying exact information about data
 
 
1 hour later…
4:17 PM
Yeah, no I understand, and intuitively I think I would respond the same if I re-read my question. It's more like I have two axis in a scatterplot and I want to visualize and quantify the spread of all the dots in x and y. It's not a substantial difference, so I want to gauge if it's worth it to even visualize (my gut feeling would be 'no'). Can be done in many easy (easier?) ways but I was wondering if I can add this for clarity to the readers
just because dots are also overlapping - it's more transparent if you show the distribution. To this end I considered the histogram in a scatterhist or scatterhistogram like fashion
(the difference is significant but small)
thanks for answers
!
I don't really care about 3d, it was just the easiest way for me to think about a median or 95% contour line in a scatter plot
 
I'd say that if you plot contours for various density levels, you should leave out the scatter plot. The two represent the same information. Pick the one form that shows the aspects of the data you want to show.
There's also this type of plot:
It's called a marginal histogram plot.
A 3D histogram is useless IMO. You can show a surface plot of an estimated density function if the shape of the function is more important than the values themselves, or the exact location of the maximum, etc. If those aspects are more important than the shape, use a contour plot. If the correlation between the two axes is most important, use a scatter plot.
And remember that if you want to show multiple aspects of a data set, you can always have multiple plots. You don't need to put everything on top of each other into the same plot.
 
5:19 PM
posted on February 26, 2021 by Steve Eddins

A MATLAB user recently contacted MathWorks tech support to ask why the output of fft did not meet their expectations, and tech support asked the MATLAB Math Team for assistance. Fellow Georgia Tech... read more >>

 
5:32 PM
@CrisLuengo that's what I was thinking too
Not sure if MATLAB has that
 
@AndrasDeak I always find it funny to see this one being used - it reminds me of all the video games where everything that is supposed to look futuristic is hexagonal:)
 
Well it does look nice :)
 
6:30 PM
@AndrasDeak With a Numpy array A, the function call np.sum(A) and the method invocation A.sum() seem to do the same. Is there some subtle difference? Which is preferred (faster, better style, ...)?
 
@LuisMendo exact same as far as I know. I prefer the method lookup. Might be marginally faster but doesn't matter. If you use JIT accelerators np.sum might work better.
Not sure about ndarray subclasses, I don't use those.
 
6:47 PM
Thanks. I've also noticed that A.sort(axis=0) changes the original array bound to the name A (if that's correct terminology), whereas A.sum(axis=0) doesn't. Why is that? @AndrasDeak
 
7:01 PM
@LuisMendo probably because list.sort is in-place
Besides, sum is a reduction but you can't change an array's size
 
7:23 PM
Ah, you can't change an array's size. So to append to an array there is no method, so you use a function, which gives a new array. Correct?
 
@LuisMendo yup, and that's a crime
 
What is a crime?
 
np.insert and friends should cease to exist
Equivalent of x = [x 1];
 
I don't follow. To extend [1,2,3] to [1,2,3,4] what's the right way?
np.append, right? tio.run/##K6gsycjPM/7/…
@AndrasDeak So, using functions on an array never changes the array, whereas using methods on the array sometimes changes it (A. sort) and sometimes doesn't (A.sum). Is that correct?
And if so, isn't that a mess? You have to memorize what methods change the array and what methods don't?
 
7:39 PM
@LuisMendo I can't name any other ones, so it's not that difficult. But yeah, I'd look up the docs to see how np.sort behaves
@LuisMendo if it's just one element, sure np.append. But it's never just one element. I'd even be tempted to do np.concatenate([arr, [4]])
@LuisMendo maaaybe
as I said I think arr.sort is the odd-one-out here
Cf. list.sort and sorted().
I wouldn't be surprised if the devs now thought it was a mistake. But numpy is very keen on backwards compat.
And it inherited some API from Numeric
 
7:57 PM
If you allow me to make a recommendation: I much prefer using pytorch over numpy - I guess they learned some lessons from numpy and avoided making the same mistakes:)
But that only makes sense if you don't have to rely on other libraries that only work with numpy.
(of course they do have a .numpy() function:)
 
Damn, hexbin is nice!
The hexagonal grid is a more efficient sampling than the square grid, you need fewer samples to get the same resolution. And it looks cool. But it's annoying to work with.
 
Yeah, I can imagine.
 
@LuisMendo This is typical with object-oriented code though. It happens everywhere. And actually it also happens with imperative languages that have pass-by-reference. You always need to look up whether the arguments are being changed or not. This is why I like MATLAB so much (before the handle class shit was introduced): no pass-by-reference means inputs are never changed, the function call tells you exactly what is going to happen.
 
Yeah, python's model of name binding is both powerful and slippery (especially for newbies)
 
@CrisLuengo I feel like you would love Haskell (if you don't already do:)
 
8:16 PM
@flawr Yeah, functional programming makes sense in that it has no state, and therefore pass-by-reference is meaningless. I should try Haskell some time, just haven't had a reason to (I'd need a project to implement in Haskell, rather than just reading a book about it).
Still, I didn't say I don't like pass-by-reference, it leads to efficient programming. A state is often necessary, and being able to modify that state without copying it first is really nice.
 
8:32 PM
You could just stick to immutable types in python, but that would be silly (and very difficult :P)
 
9:10 PM
Everything is a tuple!
 
9:47 PM
@AndrasDeak So the usual thing for a method is to not change the array, similarly to a function, which never changes it, right?
@flawr Thanks! I need to use Numppy, though; both for a project I'm working on and for a course I'm taking
 
@LuisMendo yup
and specifically for ufuncs I'm certain there's no difference between method and module-level function
Or, hmm, perhaps that's a silly statement. Come to think of it most ufuncs are probably not array methods.
there are 71 "public" ndarray attributes, which sounds like a lot, but visually not that many
[k for k in dir(np.arange(1)) if not k.startswith('_')]
and a handful of those are actually properties
Looking at that list there's no obvious contender other than arr.sort. I can see arr.put now which mutates the input, but that's pretty esoteric.
and arr.resize but I never go near that
 
@CrisLuengo I'm ok with having both pass-by-reference and pass-by-value as long as there is a rule (or some differentiated syntax) for telling when each of them applies. But if a method can either change the array or produce a new array with the same syntax that's confusing, I think
Yes, I probably have been spoiled by almost 25 years of Matlab programming :-)
 
In [330]: arr = np.arange(10)
     ...: also_arr = arr[:]
     ...: arr.resize(1, refcheck=False)
     ...: print(also_arr)
yeah, no
 
@AndrasDeak Thanks! So I'll assume that methods typically don't change the array
 
@LuisMendo well that's par for the python course
@LuisMendo yeah, that doesn't normally happen
there's also arr.setfield and arr.setflags apparently, but they have "set" in the name
 
10:01 PM
Yeah, that clue helps. And they look sufficiently advanced that I'm not going to use them :-)
 
>>> import numpy as np
>>> arr = np.arange(10)
>>> also_arr = arr[:]
>>> arr.resize(1, refcheck=False)
>>> also_arr += 1
munmap_chunk(): invalid pointer
the first time I got a more fun error message
corrupted double-linked list
@LuisMendo another pitfall you might encounter: I presume you know what views are and how they behave. But arr.real and arr.imag are also views.
OK, that might be less surprising. But np.real(arr) and np.imag(arr) keep this behaviour :)
 
I've heard about "views" of an array, but I never quite got what they are
 
@LuisMendo Arrays themselves are a very thin wrapper around a chunk of malloced memory. A view is a different wrapper over the same chunk of memory (or a subset of it)
 
Ah, I see. So a view shares the data with the array
 
arr.data is the memory an array's data sits on. arr.base tells you the original array into which the array is a view. For an array that owns its data (i.e. is not a view) arr.base is None.
While for python lists lst[:] is a (shallow) copy, for a numpy array arr[:] is a trivial view
 
10:08 PM
Good to know, thanks!
@AndrasDeak Now that's confusing. The more I see about Numpy, the more messy it seems
 
@LuisMendo nah, views are clear once you understand what they're about. But numpy arrays are very much not python lists. Think of arithmetic for instance, or bool(lst).
In [16]: arr = np.arange(10)
    ...: brr = arr[2::]

In [17]: arr.data
Out[17]: <memory at 0x7f10547b4ac0>

In [18]: brr.data
Out[18]: <memory at 0x7f107403b7c0>

In [19]: arr.base is None
Out[19]: True

In [20]: brr.base is arr
Out[20]: True
 
I see
 
And a bunch of things in numpy give you views. One key thing about basic indexing and fancy (aka. advanced) indexing is this: slices as indices are basic indexing and give you views. Array-likes as indices are fancy indexing and give you copies. And the distinction is clear if you consider that any slice into a memory can be described by strides, whereas a general array-valued index can't be represented by just strides.
And by strides we mean "the number of steps to jump on the underlying chunk of memory with each dimension of the array"
In [28]: arr = np.arange(10)
    ...: brr = arr[::2]  # every other element: double the strides

In [29]: arr.strides, brr.strides
Out[29]: ((8,), (16,))
(I'm done for now :P)
 
Yes, that makes sense. But... you have to remember / name your variables appropriately to know which are "linked"
So Numpy designers introduced quite a few things out of "normal" Python. This concept of view, and fancy indexing, for example
Thanks for your help!
 
Absolutely, numpy is a whole new world compare to native python.
@LuisMendo yeah, you have to keep track of that. You normally avoid using multiple views at the same time, because you don't want to mutate your arrays inadvertently. Just call .copy() to get a new array that owns its data.
And no worries :)
feel free to ping me with any issues
 
10:15 PM
While Python looks neat and clean, Numpy seems (to a newbie like me) messy and "patched"
 
These are important concepts, and they make a lot of sense even though they might seem overwhelming at first. The concept of the view is fundamental for efficient programming, and is the one thing that MATLAB is really missing.
 
Python has plenty of warts and quirks. But first and foremost numpy gets shit done.
 
I hope "shit" there is not literal :-P
 
I also tend to find numpy views and indexing concepts clean, but that might just be familiarity.
@LuisMendo depends on how much you abuse it :P
 
Yeah, I guess. To me, almost everything in Matlab seems clean :-P
@CrisLuengo Yes, I see how they can be more memory-efficient
 
10:20 PM
@LuisMendo For example, notice how in MATLAB subsref and subsasgn are different functions, that mostly implement the same functionality? The one copies data out of the array, the other copies it into the array. If you have views, these two would be the exact same function, creating an array into its input. a(1:3:end) just returns the view, whereas a(1:3:end) = 0 fills the view with zeros.
 
those are different functions in python too :P
__getitem__ and __setitem__, because you can't assign to function calls
 
Yeah, you can assign different functionality to them, but they don't need to have different code. In NumPy they're mostly the same.
 
Which is actually very important, that's why you can assign to fancy indices
 
Sure, with fancy indexing they're no longer the same in Python.
 
arr[:, [1, 3]] = val works, whereas first taking subarr = arr[:, [1, 3]]; subarr[...] = val won't touch your original
 
10:22 PM
In DIPlib they are the still the same, though. :)
 
11:11 PM
@CrisLuengo I never thought to view it that way :-D
Neat!
But... how would they be the same function? One still reads from the view while the other writes into the view
 
I think he means thin wrappers around one get_view function
at least that's how I read it
 
Yes, the part "select a part of an array" would be common, and that gets replaced by the concept of view
Then that part would be read from or written into
 
I think "select a part of an array" is the view
Funny that in python 1:5 is not a first-class object but arr[1:5] is, whereas in MATLAB it's the other ay around (well, you know what I mean).
 
@AndrasDeak Yeah, I would have liked to be able to use the colon in function calls as well. A('foo',4:7). Instead you need to do A('foo',slice(4,7)).
@LuisMendo The a(1:3:end) part (in my examples above) can be interpreted identically in both cases (OK, so Python doesn't do this, but DIPlib does). The = 0 is a second action, that takes the result of a(1:3:end) as one of its inputs.
 
11:29 PM
@CrisLuengo I don't see why it should be special-cased then. Just make 4:7 a synonym for slice(4, 7). Then again it would more often be useful to make it mean range(4, 7) but that's out of the question.
 
I'm not sure I follow. I expected 4:7 to be a synonym for slice(4,7) always, but this is only true within an indexing expression. func(4:7) is illegal, arr[4:7] calls __getitem__ with slice(4,7) as one of the arguments.
 
@CrisLuengo yes, that's why it can only mean slice. My point, however, is that It might be nice to have a standalone range object with the same syntax. Same as how 1:5 is a range in MATLAB.
 
Also, range() and slice() should be the same thing. They should both be utterable objects that generate the values in sequence.
What is 'utterable'? My computer corrected "iterable" to it. It tried to do the same thing just now. Weirdo.
 
@CrisLuengo I guess there's no reason why modern python couldn't do that, but in python 2 range just returned a list.
 
Right, that makes sense. A lot of these ideas about how stuff should work are developed over time by experience, but it's hard to change how the language works. This is, I guess, why there are so many different languages. :)
 
11:34 PM
@CrisLuengo that's why I don't let dumb machines touch my typing. I insist on my own typos.
@CrisLuengo and some mistakes are bad enough that they do change them. And then after more than a decade we still see people scrambling for python 2 support :'(
 
I would have 10x as many typos without autocorrect, so I guess I shouldn't mind the occasional weird typo.
And there's still people so angry about Python 3 that they still refuse to switch over from 2.
 
they are literally the worst
Especially the ones that insist on python 2 because they don't want to use parentheses in their prints. That's as dumb as it gets.
 
Indeed. There are bad reasons and there are stupid reasons. That is a stupid one.
 

« first day (2067 days earlier)      last day (1153 days later) »