« first day (2866 days earlier)      last day (2081 days later) » 

12:26 AM
@JerryCoffin Evil is when you put emotional feelings into such things. Innovations are doing mostly useless things nowadays.
Money is only useful in human societies, when you eliminate that factor, a lot of 'innovations' are fairly useless ...
 
How will I redeem my work for food and shelter without money?
 
You can grow/make them yourself
 
Boo, I'd rather program, goof off, or learn a new computery skill.
 
@AaronHall That would be called "barter". In normal work, you have two separate negotiations involved: one in which you determine the value of your work (in some abstract form), and another in which you determine the value of something else you want (also in some abstract form). In barter you combine the two, and directly express the value of your work in terms of what you want--a single concrete transaction.
 
What if the landlord and farmer don't need a programmer?
 
12:36 AM
@AaronHall That's where conversion to an abstract form that's of value to everybody involved becomes awfully useful...
 
What should we call that abstract "store of value" then?
 
@AaronHall Money.
 
🎉
 
But it also means whatever you can exchange is what humans can provide.
 
Or, in the US, we don't officially have money. That stuff people put in their wallets is strictly a substitute for money. The US constitution requires actual money to be backed by gold, silver, or other precious commodities. Since federal reserve notes aren't, they're not actual money, just something you can use as a substitute for money.
 
12:38 AM
I wonder what form money will take in a 1000 years...
 
In Time is a 2011 American dystopian science fiction action thriller film written, directed, and produced by Andrew Niccol. Justin Timberlake, Amanda Seyfried and Cillian Murphy star as people in a society where people stop aging at 25. Instead, a new economic system uses time as currency, and each person has a clock on their arm that counts down how long they have to live. The film was released on October 28, 2011. == Plot == In 2169, people are genetically engineered to stop aging on their 25th birthday. Everyone then develops a countdown on their forearm set for a year. When the clock reaches...
 
@AaronHall Zombie scalps.
 
Having multiple skills and good at them gives you more options, thus power ... and freedom.
 
Hey guys, back-of-the-paper calculation question: if you theoretically packed floating points into a single bit and ran xors instead of multiplications, on a 2k x 2k matrix would you experience a 100x increase for gemm?
 
byte?
 
12:46 AM
2kx2k matrix is 400k bits = 50k bytes < 1MB which means it can comfortably fit inside of a L2 or L3 cache, meanwhile 400k floats = 1600k bytes > 1MB which means it doesn't fit inside of L2 or L3 cache
 
you saw the hackernews article on discrete floating point right?
 
@Mikhail which one?
Actually, 50k bytes can fit inside of L1 cache
 
@Mikhail I see the edit, and I'm still not sure which one you're referring to
 
@TelKitty Being good at a number of things provides some value under a variety of circumstances--but being superb at one thing tends to have more value overall (e.g., a decent brain surgeon tends to make more than even a great a general practitioner).
 
12:52 AM
@JerryCoffin But if there ever is a surplus in brain surgeons ...
Never leave your vulnerability to others mercies
 
@OneRaynyDay A day or two ago some dude released an avx optimized discrete floating librayr for C++. Had some machine learning angle.
 
@Mikhail shit.
 
found in my upvote history
 
oh. This one. I didn't know it was so recent. I've looked at his code. His approach is good, but it's a different direction from mine
Mine is lower precision but should theoretically yield higher performance
 
12:56 AM
^ That's also a matrix-vector multiplication, not a gemm
 
@OneRaynyDay Depending on processor, of course. On the machine I'm using at the moment (with a Xeon processor), yeah, it'll fit really easily. On the other machine I use quite a bit (with an AMD Bulldozer processor), you'd need all four cores' L1 caches to hold 50 K bytes.
 
Notice that 32bit to 16 bit has little advantage, likewise 8 to 4 doesn't help too much
 
@Mikhail I suppose - but mine is 1, which lends itself to built-in avx instructions that can do bit-wise operations
he uses some clever 256 intrinsics masks to do something similar though
 
@Mikhail "Discrete floating-point library"? Isn't all native floating-point discrete?
 
The motivation for 1 bit isn't clear to me
 
12:59 AM
haha, I saw that
 
I'll send some papers after team dinner tonight
 
or you could produce a cogent statement
 
suppose A is a floating point matrix - you can approximate it with A ~= B * s, where B is {-1, +1}, and s is the average magnitude(l1) of each element in A
 
that's not binary
 
Like the discretization step of the current NN stuff?
 
1:02 AM
@Mikhail represent -1 as 0
and it is
@Mysticial yessir
 
Anyways, you can represent the actual nueral network as binary, operations but the input data and training are still discrete. This only helps with the evaluation, possibly at the expense of training/performance
 
read the xnornet paper for more information - a single cogent statement doesn't do enough justice
 
Oh great, so NN went from:
- SP to 16 x 16 -> 32-bit accumulation
- 16 x 16 -> 32 to 8 x 8 -> 32bit accumulation.
- And now, it's 1 bit?
What's next? 0.5 bits?
 
noobs should have just simplified their architecture
 
hm...
 
1:07 AM
So 80% of the stuff at Hot Chips this year is DL/NN stuff. So I know a bit more than I did 2 days ago.

These NN matrices are going to be sparse after pruning. It still makes sense to treat them as low precision dense structures?
Maybe not 80%. Probably 40% DL/NN, 40% security.
 
@Mikhail Not sure how it makes sense here, but it's pretty common for compressing some kinds of data. If you don't need to high a slew rate, a one-bit delta can represent data with fewer bits than something like PCM.
 
@JerryCoffin One of the methods that the presenters showed was to pick a small number of discrete weights. And store indices to them in the actual storage.
During computation, you pull them out via LUT.
That way you can choose non-equally spaced apart weights in the matrix.
 
@Mysticial MmmHmm. At my previous job, one of our math guys was working on some stuff like that.
 
The point being to reduce the memory footprint since apparently DL figured out how to make a matrix multiply memory-bound. (or maybe this is the case only after the matrix becomes sparse after pruning)
 
@Mysticial I find sparse matrix stuff frustrating--it seems like everybody who comes along feels obliged to define and use a new and different way of representing the data. A year or so ago I did some playing around, and it seemed like there were roughly three classes (so to speak) that worked well for different tasks, but there were a dozen (or more) gratuitously different representations in each class, none offering a significant advantage over the others (at least for most tasks).
Of course, I only tested on a few machines, so I suppose it's possible there could be more variation when you take more machines into account. But given the general level of similarity between machines, I doubt it makes a huge difference in most cases.
 
1:59 AM
@JerryCoffin Yes, but you can't compress beyond the intrisic information content. Now, the actual neural network, when optimized, should be approach the smallest representation. So, given a complex enough function, all internal representations should be the same size. I also suspect that it is much harder to minimize many discrete values, rather than a few floating point values.
 
 
1 hour later…
3:04 AM
I have found a perfect use for action camera in a waterproof case - to send it down a sewage pipe and see what's blocking it >_<
Instead of spending, say, $200 a day for a pipe camera :x
There is no limit to a cheapskate with imagination!
 
 
1 hour later…
4:13 AM
@TelKitty what pipe? (What isn't draining)
 
...
buy reading glasses ...
 
5:14 AM
They will help with my comprehension?
Do you have the blueprints for the house - is the sink/tub on the same drain? If they're draining OK, then it's simply the toilet?
 
It's the drainage to the kitchen that's blocked ... more specifically, the sewage pipe that kitchen drainage is connected that's blocked.
 
Why do you need to see what's in there? Why not simply force it through and charge the resident?
99% chance its rice/noodles.
 
Owner pays for it not tenant.
Thus to save money, we solve the problem ourselves.
 
Not of they misused/abused it.
Abuse/misuse/neglect = they can be charged for it. In America anyway..
So, why do you need to see it? Why not simply run a drain-dnake (auger?) through, and be done with it.
Does your father pay you by the hour? ^.^
 
It's unblocked now.
Thanks to my superior physics skills.
 
5:24 AM
You didn't..!
I hope you washed it first.
 
Also we own a garage full of tools, include a small drain snake.
 
: )
Do you have an Australian accent?
 
My A0 sized poster for rental advertisement has arrived. I am going to put it up tomorrow.
 
Mm..Steve Irwin impressions probably aren't as funny to you.
 
@Mikhail you should read the paper on how the discretization is done
all your questions will be answered
but it's okay if you're not too interested in it
 
5:32 AM
The discussion isn't about how, my initial question was as to the motivation.
 
@Mysticial you're talking about deep compression by han et. al., they remove 90% of the weights that are centered around 0
well, you were saying that it is dubious to optimize a discrete value problem, i.e. integer linear programming on {-1, 1}
in this case the answer is that you're not solving an ILP - you're solving the neural network with binarization activation function
 
No. I believe its certain possible to do. I suspect that the total size of the network should be the information content, independent of the discretization, etc.
Important to note that its not making a floating network discrete, its making an equivalent network using discrete functions. aka you're not simply converting the floating point network to discrete.
 
you are right on that - simple conversion never works
 
yeah but you aren't addressing what I wrote about the performance of the network being related the the intrinsic information density
 
if the point was "you can't make the input discrete" - you can
you pass the input through a signum function and operate on that instead
 
5:49 AM
@ABuckau I am not joking.
Took me A$58 to get this printed out and laminated.
I still have to find a frame for this.
 
oh by the way, this poster reminds me @Mysticial remember when I asked you about stress tests:
There is another mandelbrot set with proper colors on your floor, hung on the wall :) not sure if you've seen it
the rendering ended up being 40 GB's and crashed the printer 4 times before the above figure managed to print (with incomplete colors)
 
6:39 AM
wut
 
better change the locks
 
locks are not enough
 
I would assume Mysticial's life is lock-free
 
2 stalkers, 1 target ... who will win? Let's find in the next episode of The Lounge ...
 
On the subject of social degeneracy, time to render my scene in Blender on a 980 GTX = 14.0 seconds, on 36 cores on an e5-2686 its 13.6 seconds. Who wants to compare cost and energy consumption?
 
6:53 AM
Assuming equal power efficiency of the processing unit and equal implementation efficiency, shouldn't power consumption be pretty close/nearly equal?
 
Another reading of your statement is to assume the two components are physically the same, which they certainly aren't.
 
7:09 AM
@telk I didn't know how big A0 was. Looks good.
 
Is centimeters not used much? Always thought it was weird that a smaller unit was used..but I also think centi is weird because most others are 1000 not 100...sometimes messes me up when I convert.
..the house, I can not afford, but maybe if I fold that paper correctly.. :p
(Removed duplicate)
 
you can always share with another bear
I need to work on my presentation now ... need to put in extra effort because I am an inferior public presenter ....
 
7:43 AM
@TelKitty Inferior compared to whom?
 
whoever does it for a living
 
Everybody is inferior compared to Scott Meyers.
 
8:01 AM
> The state of the world is broken.
I like LWG bikeshed
 
8:14 AM
Does explicit actually do anything when placed on constructors that take several arguments (disregarding default parameters and variadic stuff)?
 
8:24 AM
@ABuckau engineers prefer mm likely to avoid a decimal point that can confuse things and will jump straight to m when needed.
 
8:42 AM
@Morwenn it has nothing to do with number of arguments in a ctor, or did I misread your question?
 
Makes sense. As an American, I default to cm instead of mm 1) my estimates aren't that accurate 2) I was taught 1in=2.54cm, not 25.4mm ..but I notice most people (YouTube videos) use mm.
 
nwp
The main reason why I hate explicit. It makes code so ugly and cumbersome and somehow people like that.
 
{} is a crime against static typing, if you used the constructor you could save the two characters to type out the curly brackets
 
nwp
Just like auto.
 
Well. I deal with a lot of pod like classes and I've been fucked over when a float is subbed for an int or most recently a int became a bool.
 
nwp
8:54 AM
That doesn't even compile because narrowing conversion.
 
Also typed this up on a mobile phone, while riding a bike at night, without a helmet.
 
nwp
You probably shouldn't do that.
Definitely a crime, but not against static typing.
If you have proper names this is probably the future.
 
or if you have a C compiler, but also that sucks
 
nwp
I don't know why bool b{42}; does not trigger a narrowing conversion error. That seems to be the root cause of the problem.
I guess it simply is not on the list.
 
9:14 AM
@nwp It almost seems weird ^^'
@login_not_failed explicit is mostly used on single-argument constructors to avoid implicit conversions
 
uniform initialization
More like pseudo-random initialization generator
4
 
Ven
Hi
 
Hey ^^
 
nwp
I wouldn't fault the feature for an implementation implementing it incorrectly. There is a 1 year old bug report for it.
 
9:29 AM
Fortunately C++20 comes with plenty of new initialization rules, both on the language side and on the library side
I count at least 4 proposals that change initialization rules in C++20 + one that changes std::variant initialization rules, and I might forget a few WGC or LWG issues
 
nwp
I'm looking forward to some of them. struct S { int i; double d; }; S s(42, 3.4); might end some of the make_unique issues.
And probably cause 10 new ones, but oh well.
 
That one is currently on CWG side IIRC, but it's likely that it will get accepted
The ones that makes aggregates with deleted constructors not aggregates is already in
NSDMI for bitfields shouldn't change many things
And deisgnated initializers look conservative enough to not break things but to not be useful as well
The paper to change the initialization rules in std::variant doesn't solve everything but looks saner than the status quo
 
Ven
@nwp C++ works like The Real Worldâ„¢: the development time doesn't include the bug-fixing time.
 
nwp
There is a comment "testing patch now" from 1 week ago on the bug report. I guess they are getting around to it now-ish.
 
nwp
10:32 AM
Also I'm bad at reading and the bug report is over 3 years old, not 1.
 
 
3 hours later…
1:08 PM
Ranges is getting in good shape for C++20
With some additions, some deletions and a lot of bikeshed compared to the original design
Looks like the stack trace library is aimed at C++20 too
 
 
3 hours later…
4:28 PM
Morwenn: Is there an updated pdf for ranges somewhere?
 
 
1 hour later…
5:43 PM
2 questions Why 😂😂 And what the actual fuck? I love it https://t.co/R0598OvhGY
^ Lol. A lot.
 
5:59 PM
Without comment
 
 
1 hour later…
7:16 PM
Hi, anyone here read this paper? cs.utexas.edu/users/pingali/CS378/2008sp/papers/gotoPaper.pdf I have some possibly dumb questions
 
7:32 PM
No, but I am always eager to give my unconsolidated, miss informed, and incomplete suggestions.
4
 
nwp
7:45 PM
That's Jerry's job.
 
8:14 PM
@nwp My opinions are unconsolidated and incomplete, but mostly mister informed.
 
 
1 hour later…
9:22 PM
I will accept any suggestions; give me your tired, your poor, your unconsolidated/misinformed/incomplete -
the wretched refuse of your shore
but the dumb question was resolved for now after staring at it hard enough
 
 
1 hour later…
10:39 PM
Question teasers.
 

« first day (2866 days earlier)      last day (2081 days later) »