« first day (1538 days earlier)      last day (1690 days later) » 

8:23 AM
Interesting find today when answering a question about a logical matrix:
array1 = 5*rand(496736,1);
array2 = 25*rand(9286,1);
% output = zeros(numel(array1),numel(array2))` % Requires 34GB RAM
% output = zeros(numel(array1),numel(array2),'logical'); % Just 4.3GB
tmp = abs(array1.' - array2) <= 2; % Skips the intermediate full matrix, directly 4,3GB
 
8:40 AM
huh
How the hell? Does it vectorize the <=2 part somehow?
 
Prob has to do with broadcasting optimization, no?
when broadcasting it probably just loops
 
but it has to compute 34 GB worth of doubles
 
and it loops the entire operation
 
yeah, I was thinking perhaps it loops over one of the dimensions
But then the size difference wouldn't be double vs bool, it would be n1 vs n2. I was just about to compare the two.
>>> 496736/9286
53.49300021537799
How large are bools? 1 byte (uint8)?
 
8:43 AM
@Adriaan just to be sure: you're saying there's no intermediate memory surge, right?
 
@AndrasDeak didn't check with top or something, but it performs ~20 times faster than my naive loop, and does not error out on memory, like with the nonlogical zeros() initialisation
0
Q: Finding array elements close to another array in space?

PaysdeGallesI basically want to use the function ismember, but for a range. For example, I want to know what data points in array1 are within n distance to array2, for each element in array2. I have the following: array1 = [1,2,3,4,5] array2 = [2,2,3,10,20,40,50] I want to know what values in array2 are <...

 
yeah, lack of out-of-memory is a good sign
@Adriaan what OS are you on these days?
 
@AndrasDeak Ubuntu 18.04
 
Gnome or unity? (I hope not too tough a question :D)
 
Former
 
8:51 AM
I have CPU use, memory and network in my top bar
 
@AnderBiguri so you suggest that what the broadcasting does is an implicit, faster version of my explicit loop?
 
very obvious when something starts hogging RAM
@Adriaan that's exactly what it is
 
@Adriaan I think thats the most likely explanation
 
On the CPU everything is a loop sooner or later. Broadcasting is only about memory pointers not being incremented while some array indices are changing
it's much more obvious with strides in numpy
 
MATLAB is quite good parallelization. When I do that kind of stuff in my 64 core machine, it uses all cores
while maybe explicits loop don't (havent tested)
 
8:56 AM
@AndrasDeak I installed it (through the terminal as suggested on their github page), how do I now run it?
 
@Adriaan have you installed the gnome extension plugin on ff?
all the gnome extensions are controlled via the firefox addon
you can find the addon here
it will appear as gnome-foot-icon
 
So, I did that, I tried installing chrome-gnome-shell which borked
 
it should tell you if you need the shell integration extension and which one...
the extension should Just Work^TM afterward
 
@Adriaan what borked?
 
sudo apt-get install chrome-gnome-shell
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages were automatically installed and are no longer required:
  brasero brasero-cdrkit ca-certificates-mono cli-common dvdauthor
  libdbus-glib1.0-cil libdbus-glib2.0-cil libdbus1.0-cil libdbus2.0-cil
  libgdata2.1-cil libgdiplus libgkeyfile1.0-cil libglade2-0 libglade2.0-cil
  libglib2.0-cil libgtk-sharp-beans-cil libgtk2.0-cil libgudev1.0-cil
  libmono-addins0.2-cil libmono-btls-interface4.0-cil libmono-cairo4.0-cil
 
9:02 AM
> If you are using GNOME 3.4 or newer and installation still doesn't work, check to make sure that the "GNOME Shell Integration" plugin is installed and enabled in your browser preferences.
@Adriaan who told you to install that? Chrome?
 
@AndrasDeak the add-on page :s
 
OK, just checking
that's a very weird error
 
If I rerun the install command I get chrome-gnome-shell is already the newest version (10.1-5)., so I guess it's there?
 
hmm...let's hope so (and not a half-installed monster instead)
455
A: sh: 0: getcwd() failed: No such file or directory on cited drive

HoomanThis error is usually caused by running a command from a directory that no longer exist. Try changing your directory and re-run the command.

 
apt-get remove first?
 
9:05 AM
did you do that?
if you did that, yeah, purge and install again from an existing dir
 
Ah right, that's indeed the case. It should be installed now
It all works now, thanks!
 
Cool, cool. It has a lot of settings (mainly graph and numeric display of various monitor subsystems)
 
Also a nice customisable clock
 
while we're at it I use a bunch of other extensions which I need, like a bottom panel ("frippery bottom panel") and static workspaces and whatnot
 
"Remove alt-tab delay"
there is also a nvidia-gpu stats tool
and "put windows"
 
9:08 AM
@flawr That's a thing? Huh.
 
@AndrasDeak saves about 1t of nerves per day
 
doesn't sound crazy at all
Hmm, solves only half the problem. The actual switch is slow too :( But I guess it's a bit better, thanks :)
 
@AndrasDeak "Impatience" is the other one I have:)
@AndrasDeak oh that's great!!
 
the bottom panel one needs a slight hack for me to be pretty though, in ~/.local/share/gnome-shell/extensions/Bottom_Panel@rmy.pobox.com/stylesheet.css I have to edit
.window-list-item-box {
    spacing: 3px;
    border-radius: 4px;
    border-width: 1px;
    /*border-color: #bbb;*/
    border-color: #666;
    color: #bbb;
    padding: 0px 2px;
}
the original colour is just too intrusive
 
thanks a lot:)
 
9:16 AM
no problem
 
I have to get used to the horizontal scrolling:)
 
How so?
 
I mean scrolling through workspaces
the default is vertical
 
I know, but why do you have to get used to that?
you can still use vertical I think
 
But I don't want to:) I actually never understood why they arranged the workspaces vertically by default. It just seems so counterintuitive (I mean now I got used to it, but still)
 
9:18 AM
Ah, yeah. I hate it vertically.
Unfortunately when I upgraded my debian a few days ago the default "vertical scroll" animation would stick even with horizontal layout, so I needed yet another extension to disable the animation :D Not a huge loss though, it's quite annoying.
 
haha
do you know "put windows"?
for arranging multiple windows on your screen
or are you using something else?
 
stackoverflow.com/a/57953509/5211833 Interesting; Luis' answer does not scale to 4.3GB directly
 
@flawr I prefer artisanal manual placement ;)
I've heard of it and friends, but not my cuppa
most of my windows are full-screen, and when rarely I need something else I use (alt+)super+arrow for half-window placement
but I'm always on laptop, perhaps it would be different with a monitor or two
 
9:41 AM
@AndrasDeak ok I see
well "put windows" works nicely when using a numpad
 
I don't have one ;)
 
that is why I was mentioning that:)
repeated presses scales the window also into 1/3, 2/3 in each dimension, so you can easily put them in corners or just side to side etc, and you can customize the heck out of it - so just in case oyu ever want to get a large screen:)
(and a numpad :P)
 
thanks, but I don't see myself wanting something like that
 
a numpad you mean
this VIMer
This weekend someone tried to convinceme to use VIM
He ded now
 
rip
@AnderBiguri both I guess :P
 
9:51 AM
@AndrasDeak you mean you work an a touchscreen ? :D
 
no, ew :P
 
there are some screens in my building that have a label " this is not a touchscreen, please don't touch"
 
@flawr hehe, sounds like the stickers "Please do not use permanent markers." on whiteboards, or "This is a Smart Board, please do not write on it"
 
Imagine how cool a touchscreen was when we were kids (well, maybe not you). Now everything is a touchscreen.
 
well not everything apparently :P
 
9:53 AM
 
@Adriaan hehe, right:)
 
@Adriaan Are you sure that the implicit expansion approach doesn't use the full ~37 GB?
It creates the matrix of differences, so I'd say it does
 
@AndrasDeak when I was a kid we still had b/w crts and floppy disk drives named A: and B:
I hope that reassures you:)
and my could probably still "sing" you the sound of my first internet connection:P
 
BZZZZZZZZ boing boing boing
and that pang of fear when you realized you shouldn't have picked up the phone
 
@LuisMendo as mentioned above in this chat: it runs within a few seconds and does not error on my PC. Since I have just 16GB RAM, I presume it actually does not create this intermediate double matrix (results are equal to my looped approach)
 
9:57 AM
@AndrasDeak In my house we needed to physically unplug the phone and connect the internet
 
@AndrasDeak when already half of the image had loaded and you have to start over again
 
@AnderBiguri that's safer actually
 
yup, bu tmy mum would get mad because people called and we could not pick, so she would try to minimize my use of internet
 
@Adriaan But how can abs(array1.' - array2) know that it is going to be reduced to a logical result? I don't think Matlab does that kind of optimization
 
@LuisMendo fair point, but just try to run the different options on your machine using the array sizes I suggested in my answer. Implicit expansion and my loop work and give a 4.3GB output, the other two options (bsxfun and pdist2) bork on a RAM requirement
I don't claim to understand why MATLAB does this, I just happen to observe it
 
10:02 AM
@Adriaan It works as you say. I find it hard to believe, but it does
 
So, another MATLAB question? :P Or is this something only TMW themselves can answer?
 
It would be an interesting question, yes. Care to proceed?
 
almost certainly TMW-only
although I'd try changing the array sizes so that there's a very obvious difference between n1/n2 and double/bool
then again there's a factor of 8 as it is now, I think
n1/n2 is around 50 and double/bool size is 8
let me know if my rambling doesn't seem to make any sense because I'm sort of making a point :D
 
The difference is large enough now. Both in @Adriaan's and in my computer, the logical result fits in the RAM, and the double one errors
 
OK :)
what if you use switched dimensions...?
Smaller first, larger last?
if there are internal loops for broadcasting those could be affected, unless it's smart enough to choose the better one from two cases
 
10:09 AM
Same thing, on my computer
 
thanks for checking
 
It seems like implicit expansion (but not bsxfun) looks at the surrounding code and exploits it to optimize
Weird
 
very weird
But it has to compute the doubles to compute abs() <= 2...
 
Yes. It probably computes them in small chunks, so no memory issue
 
perhaps it rewrites it to elementwise abs(a_i - b_j) <= 2, but still
0
Q: Does MATLAB's implicit broadcasting take care of types?

AdriaanA question asked today gave a surprising result. array1 = 5*rand(496736,1); array2 = 25*rand(9286,1); output = zeros(numel(array1), numel(array2)); % Requires 34GB RAM output = zeros(numel(array1), numel(array2),'logical'); % Requires 4.3GB RAM output = abs(bsxfun(@minus, array1.', array2)) <= 2...

someone has the same problem on main
 
10:12 AM
:-)
@Adriaan For clarity, I would add output = abs(array1.' - array2); % Requires 34GB RAM right after or before the bsxfun line. And emphasize that whatever optimization is applied, it happens with implicit expansion but not with bsxfun
I have to go. I'll catch up later
 
good point
see you
 
@LuisMendo done, thanks
 
10:49 AM
Maybe add the tag ?
I'm sure Yair Altman would have something to say about this matter @Dev-iL
 
@Adriaan Did you try adding the `abs` and comparison inside the `bsxfun`? Like so:

inner = @(x, y) abs(x - y) <= 2;
bsxfun(inner, array1.', array2)
(Sorry, haven't read all the messages... I was afk for the last 2 1/2 hours... If that had been already suggested)
 
@Adriaan A suggestion for the title: currently it might suggest "does implicit expansion care about input types" (of course it does, and it's expected). The key, I think, is: Does MATLAB's implicit broadcasting look at surrounding code? Or, Does MATLAB's implicit broadcasting optimize based on surrounding code?
@HansHirse That uses the expected 4 GB, but it is about 8 times slower than abs(array1.' - array2) <= 2; which is also expected, since bsxfun with custom functions is not very fast
 
@LuisMendo Ha, ok - thanks for testing. :)
 
11:52 AM
unclear (half English/half Farsi, no explanation, pure code dump) stackoverflow.com/q/57954927/5211833
@LuisMendo I used your suggestion, thanks. Feel free to edit if you want
@HansHirse works as expected; I'd edit your answer with it; then mine is completely irrelevant anyway.
wrt Obchardon's answer I'm not sure, as he gets a different type of output than what is requested
 
12:25 PM
stackoverflow.com/q/57902745/5211833 irrelevant dupe, one answer is wrong, accepted answer is performance-wise bad
 
 
4 hours later…
4:04 PM
posted on September 16, 2019 by Cleve Moler

I have learned a lot more about Kuramoto oscillators since I wrote my blog post three weeks ago. I am working with Indika Rajapakse at the University of Michigan and Stephen Smale at the University of California, Berkeley. They are interested in the Kuramoto model because they are studying the beating of human heart cells. At this point we have some interesting results and some unanswered q

 

« first day (1538 days earlier)      last day (1690 days later) »