« first day (2861 days earlier)      last day (2080 days later) » 

12:07 AM
Anyways, the discuss at the top about FP16 missed a key point. One can think of the neural network training problem as minimize F(x)=y. Anybody with experience knows that its not possible to find the global min. You won't even come close. You might minimize the energy by like 20%. That's why the precision doesn't matter, as you can't even approach the machine precision for the minimization problem
Also all Pascal GPUs have native 16 bit floating point support, not just the "tensor cores"
"tensor cores" are for mixed precision FMA: see stackoverflow.com/a/48073989/314290
Effectively getting the equivalent of 2 TB/s local memory bandwidth on a single card (1 TB/s in hardware, when using singles). Compare to this to CPUs :-)
 
I was revising matrix operations ... so dot product ... I stumped upon this simple question ...
Two vectors E: [a b] & F: [c d], dot product of two is a x c + b x d, but Geometric definitionis |E| x |F| x cos(θ). The two gives two numeric values for most cases ...
 
12:27 AM
dot product is always a single scalar result
 
...
that's not what I am asking
 
not much else I can parse from that though
 
take two vectors, [2 1] and [2 -1], the dot product is 3, but by geometric definition is 2.5
 
the angle between them isn't 60°
atan(0.5) ~ 26°
cos(2*atan(0.5)) = 0.6
 
12:48 AM
damn ... mixed tan & cos again ...
 
 
2 hours later…
3:23 AM
Does anybody have a link to the old "new programming terminoligy you coined" deleted historical question?
I would be quite able to open it
 
 
2 hours later…
5:18 AM
57
Q: Can we un-delete "New Programming Jargon You Coined"?

Ben LeeHere is the post in question: https://stackoverflow.com/questions/2349378/new-programming-jargon-you-coined According to Jeff Atwood's Stack Overflow: Where We Hate Fun, there are three things to consider when deciding what content should be allowed: Does this question match the criteria ...

 
 
3 hours later…
Ven
8:24 AM
Hello
 
Hi <3
 
 
3 hours later…
11:48 AM
hello :)
 
12:03 PM
9
A: Do C++ compilers eliminate assignment duplications?

Matthieu M.Yes, this is commonly known as Dead Store Elimination (read = load and write = store in compilers parlance). In general, any useless operation can be optimized away by the compiler providing it can prove that you (the user) cannot notice it (within the bounds set up by the language). For Dead S...

Closest thing to a confirmation of what I was thinking
Too bad there isn't some standardese to go with that :(
 
Ven
@Borgleader The legalese is the same as usually: cimpoplers are allowed to change anything that isn't visible to the user
(plus some stuff that's visible)
 
Well the argument I've been served up was "the compiler can't know that another thread isn't interacting with that variable so you have to write to it" to which my response was "it's not volatile and theres no sync so its a data race/UB the compiler shouldn't give a shit" but so far it hasn't been convincing enough so I was hoping for some standardese
 
Ven
12:19 PM
ah that prevent it from being done?
like I said it just feels like a ?SufficientlySmartCompiler issue to me.
 
if there was any non-inlined call out I could understand the conservativeness but when it's all fully inlined...
 
@Borgleader You are describing the as-if rule. (Also on cppref)
 
Ven
"Languages [...] implementable in compilers" As opposed to those which may not..?
 
@Borgleader But it seems that the argument really is about clean code? You want to avoid needing an extra variable for the hash calculation?
 
12:35 PM
@StackedCrooked No we were strictly arguing about whether the compiler was allowed to remove the dead stores on the member variable.
Their argument was "it cant do it for a member variable because another thread might be observing it", but my argument was that's UB if no volatile/atomics
and in absence of UB the as-if rule should allow us to remove the dead stores because only the last one would count.
 
Ven
that requires full program analysis though.
probably why the compiler doesn't do it
 
What would?
I probably just need to hunt down the standard's section on data races
 
Ven
ah nvm you mean to keep only the last store
 
Yeah
https://godbolt.org/g/kifAWr
This is the simplified version of the code (which sadly doesn't reproduce the optimization problem as the assembly is the same for both versions). But in my case going from A -> B saves a few mov instructions.
 
@Borgleader Yeah. Once there's UB there's nothing left to discuss really.
 
12:57 PM
> The execution of a program contains a data race if it contains two potentially concurrent conflicting actions,
at least one of which is not atomic, and neither happens before the other, except for the special case for
signal handlers described below. Any such data race results in undefined behavior. [ Note: It can be shown
that programs that correctly use mutexes and memory_order::seq_cst operations to prevent all data races
and use no other synchronization operations behave as if the operations executed by their constituent threads
From section 6.8.2.1 Data Races (from N4727)
 
1:11 PM
Hey
 
Ven
henlo
 
Could you please help me out and solve a DS/Algo problem.
Actually I need this
https://chat.stackoverflow.com/rooms/139/java
 
1:32 PM
@AnkitPandey Lol. No. It is your homework.
 
@wilx He said he needs the java room, so it's all good.
All we need is to help him stay in the Java room.
We are actually very helpful people as you can see ...
 
2:27 PM
ahhh
the Queen's personal physician has died
he was a homeopathy expert.. they tried treatment but it was ineffective
how glorious
@Borgleader It's UB if no atomics- volatile doesn't play a role here. The compiler can absolutely remove dead stores under as-if.
 
2:55 PM
I was pretty sure that's the case I just couldn't "prove it" so to speak.
 
broadly, you pretty much already did
the essence is that doing that stuff without atomics is a data race and thus UB, therefore under the as-if rule the compiler can optimise as it likes
 
Can a homeopath drown?
 
3:15 PM
More implicit moves for C++20? Answer another day :D
 
@Morwenn implicit move as in a=b;b=c; can be translated to a=std::move(b);b=c; and/or {Foo B{...}; a = b;} can be translated to {Foo B{...}; a = std::move(b);}?
 
More implicit moves when objects are returned from functions
Currently compilers aren't allowed to implicitly move several classes of objects in a return statement, like function parameters (even when they are rvalue-references already) or any object T if the return type is U with a converting constructor accepting a T
 
I thought we already had moved for returns
oh, T foo(T&& t) { return t; } would have copied t, but now it can just move it?
 
Well, it's even more subtle: if U has a move cosntructor accepting T&& then T can be implicitly moved thanks to a core DR, but if U has a constructor accepting T instead, then T can't be implicitly moved
@thecoshman Nope because t is a function parameter
 
in C++20 compared to 17?
 
Ven
3:26 PM
It's 3 more. Easy.
 
There's a proposal targeting C++20 to allow t to be implicitly moved in your example
 
Ven
obviously T is of type T&& so it will be moved from automatically... : ^^^^)
 
Today it's annoying when you try to use that with std::unique_ptr x)
 
@Morwenn I thought it was obvious my example was set in the context of having the 'more implicit moves for c++20' :P
 
Everything is obvious
 
3:34 PM
Obviously
 
 
2 hours later…
5:32 PM
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0595r0.html
Anyone know what the status of this is? Or where I can check that?
 
 
3 hours later…
8:41 PM
Have you guys thought about this problem? If you have a point cloud (x,y,z) points, and you want to go to a volume enclosed by those points? So, it seems making a convex hull is trivial, but what are the strategies if you also want to have concave points?
something to do with cubic spline interpolation
 
9:13 PM
Hi guys, I'm trying to use possibly unalligned memory access on AVX2 registers YMM0-16 (or XMM0-8 whatever), is there a way to allign an entire vector of unalligned memory, or is there a way to load it into the registers without the general protection fault from occuring?
 
The quick fix is to use unaligned loads. The not-so-quick-fix is to align the memory.
 
@Mysticial from your experience have you seen unaligned loads be much slower?
 
The latter usually being the more performant option.
@OneRaynyDay It depends. Think of an unaligned loads as being two aligned loads in parallel.
 
If I have a bitset of uint8_t's, is there a way I can initialize the bitset correctly by specifying its allignment
I see. That makes sense :o
 
On modern hardware, unaligned loads aren't necessarily longer in latency, but they do consume more of the resources. So if you have a lot of loads in parallel, the unaligned ones will count as two.
 
9:17 PM
I see. Well, I might be doing premature optimization, but a function that I am using for popcount in AVX2 requires the input be __m256i *
 
Why can't you align?
 
and from the intel guide that Jerry mentioned last time: software.intel.com/sites/default/files/m/d/4/1/d/8/… , I don't see an unaligned ptr
@Mikhail The bitset is uint8_t
Maybe there's a compiler directive to tell it to allign by 256 bits at a time, but I don't know of it
 
That's mostly a semantic issue. The rule I go by is that all vector types are aligned. And if you pass a pointer into a function takes a vector type pointer, it must be aligned otherwise UB.
 
the documentation for these things are scattered everywhere
 
The paradigmatic solution is to check the alignment of the pointer and have two code paths.
But also align everything
 
9:19 PM
I see. So I'll have to enforce allignment. Let me check online how to do that
 
@Mysticial What do you use to get a chunk of aligned memory? posix_memalign, std::align?
 
I prefer to just align everything and not care about misalignment.
Only when I'm crossing API boundaries do I do alignment checks and code-branching.
 
if (not_aligned(ptr)){throw std::runtime_error("LOL");}
 
@StackedCrooked That's a loaded question. Depends on the source of the memory.
 
Also boost::alignment::aligned_allocator
 
9:21 PM
@Mikhail Right. (I've used tbb::cache_aligned_allocator in the past.)
 
Hmm. @Mysticial excuse the bad code, but this is my simple sandbox, and popcount in here is the one with the issue: github.com/OneRaynyDay/beating-blas/blob/… , it tries to dereference an __m256i ptr and gives a general protection fault. In this case, would the sane solution be to enforce that this dynamic_bitset uses <__m256i> as template type to enforce allignment?
Or would it be to allign all incoming data? My guess is that it will take a very long time to allign a large vector, so inefficiency will come up
 
@OneRaynyDay Alignment can only be enforced manually.
By casting the pointer and checking the bottom bits.
You can't do it at compile-time.
 
Ah, I use reinterpret_cast<intptr_t>(ptr) % alignment
(But that's not really portable.)
 
@StackedCrooked Easy, make a generic check_alignment function that you customize per environment.
 
@Mysticial I'm not sure what you mean by this particular part
 
9:26 PM
@OneRaynyDay IOW, there's no way to know that an pointer is aligned or misaligned other than to manually cast it into an integer and looking at it. Or to dereference it and let it crash.
You can't stop the user from passing a misaligned pointer at run-time.
 
@Mysticial so if it's unalligned I just throw my hands up and say "sorry, no AVX2 for you"?
 
Yes, but with more curse words
 
oh. well damn
 
@OneRaynyDay If you unconditionally use unaligned load/stores, it will work for everything. And it will work without penalty if the pointer is aligned. (This penalty went away starting from Nehalem.)
 
I see
 
9:28 PM
If you want to also maximize the performance when the pointer is unaligned, then you can check the alignment, and branch to a special version that peels the loop to make it aligned.
 
Well then in my case, if the user wants, say, an std::bitset<uint8_t>, could I theoretically do something where I create an std::bitset<__m256i>, and then reinterpret cast the underlying data?
peals?
 
The other approach is to conceptually declare that the __m256i* pointer must always be aligned and passing in a misaligned pointer is UB and the fault of the user.
At least that's what I do in my code.
If the function doesn't care about alignment, I make it take a void* or a scalar type.
@OneRaynyDay typo
 
I see. Gotcha
 
In your case you can do that since the data is fundamentally bit-width.
It gets more complicated if you're dealing with an array of multi-byte types (such as float or int64_t) where even the native type is misaligned.
 
Just double checking, peeling the loop - you take each contiguous(not alligned) chunk of memory, load it as unsafe, and then save it into a __m256 register to do ops with?
rather than straight dereferencing the __m256i* ptr which is misalligned and will die
 
9:33 PM
Although I think in most cases, it's hard to have a misaligned native type without committing some sort of UB in the first place by violating strict-aliasing.
@OneRaynyDay No, you check the pointer to see how much it's misaligned by. Then you use a scalar loop to do those iterations.
Same thing on the tail end of the loop.
 
I see - so:

[ x x x | AVX 2 | x x x ] <-- perform scalar loops on x, AVX on the alligned chunk
 
correct
 
dangit
 
Ugly as fuck, but yeah.
 
this is turning out to be sooo much more annoying than I thought
 
9:36 PM
In AVX512, you can use masking instead of a scalar loop.
@OneRaynyDay Which is why Intel put so much effort into making misaligned access actually fast.
 
I see
 
It used to be > 4x slower than aligned.
 
Well - if unalligned access is ~ok-ish~, then how would you do that in this case?
there's no __m256{unsafe}
 
@OneRaynyDay Now just imagine if you're reading from one pointer and writing to the other. What if the pointers have different alignments?
Then it becomes impossible to align both at once.
@OneRaynyDay _mm_loadu_xxx and _mm_storeu_xxx.
 
Oh yeah you're right. You're gonna need some kind of intermediate, alligned storage, and then do 2 ops for the price of one
 
9:39 PM
@OneRaynyDay And the overhead of that is greater than just using unaligned access for one of them.
 
cruel world of allignments
 
Which I why I always align the data whenever possible. So that you don't have to deal with it.
 
I see
 
Side Story: At one point, I got "overly aggressive" at aligning data. To the point where my unit tests would always use aligned data. And then when relevant function was called with misaligned data, the function broke because there was no longer any test coverage for misaligned inputs.
 
@Mysticial well - then that's not your fault, right?
 
9:44 PM
@OneRaynyDay Technically no. I caught this while I was testing out the AVX512 in MSVC. It miscompiled the misalignment handling. So even though the binary passed all the unit tests, it broke in production.
But it did highlight to me that I also broke my test coverage at some point.
The fact that the production code was feeding it misaligned data is not a bug.
 
I see. Low latency sounds fun ;)
actually not being sarcastic; you get to work with all these intrinsics and deal w/ assembly
 
10:04 PM
@Mikhail one that may (no longer) interest you /cc @LucDanton :
 
10:32 PM
@sehe curiosity: ✓, though I have to admit I’m not shopping for high perf storage
 
Me neither
However it's good to see when trade off downsides get lessened
 
plus, there’s always the resilver lining
 
Might build another storage node in September if this project on personalized medicine with Mayo Clinic comes through. Recent drop in price, and increase in density make it pretty easy to build 200 TB+ using a 24 drive Norco case, which cuts off another few thousand that would otherwise be spent on an expensive backplane.
But ZFS is still pretty bad compared to hardware RAID
Curiously, PMC hasn't updated their RAID controller which is now a few years old.
Which is sad, because, as mentioned, ZFS has low performance (if you know better). If you don't know better, then it seems great.
Also the Intel splitters I use are pushing 5 years without an replacement
If the splitter and RAID card get discontinued, we'd need to go through OEM vendors for hardware RAID, which would double the price of our builds.
 
JBOD FTW
 
You still gotta split that out
For example, an Intel splitter is around $300, but with SuperMicro they are part of the backplane, so its around $2k to hook up 24/32 drives
 
10:53 PM
@Mikhail I can do 32 drives with about $400 of card on a suitable HEDT motherboard.
Full bandwidth.
 
So, hardware RAID is $600 for the RAID card and $300 for the splitter, which is less than 10% of the cost
Take a look at the cost of these systems: 45drives.com/products/windows/storinator-q30-configurations.php
 
RAID is for nubs. JBOD FTW.
 
IDK, I can overclock my RAID card
 
I can too.
Actually no I can't. It's a JBOD card, not a RAID card. RAID is for nubs. JBOD FTW.
 
But for real, often you need a single FS as we have to run software written by other people
 
10:57 PM
Well then you're fucked.
 
Also do you have parity?
 
ECC parity?
Or rather, parity doesn't help you with ECC. But rather parity error checking?
 
like RAID5 or RAID6
 
It's a JBOD card, not a RAID card.
That's why I can get 3 of them for under $400.
 
2 mins ago, by Mysticial
Well then you're fucked.
 
11:00 PM
@Mikhail No I'm not, because I don't need RAID.
 
Your hard drive might die
 
You're fucked because you need it. :)
@Mikhail Don't care. Just resume from last checkpoint backup.
 
But the checkpoint is distributed across drives right? So, if one catches on fire, you're fucked?
 
@Mikhail It's actually on the same drive. But the backups are done manually to separate physical media.
 
Drives don't randomly catch fire. If one catches fire then it's probably because there's a fire in your apartment. Which means your RAID setup is fucked.
 
11:04 PM
most of the times it means they shipped it wrong
Recently had a small fire at work when the DC/DC convert on this guy blew: avr-optics.com/xy-spatial-light-modulators
 
I know a guy that had a fire in his apartment. The DVD from the movie Titanic laying on the edge his desk had become a stalactite.
 
@StackedCrooked Every drive has a halt and catch fire instruction. Mandated by international law.
 
11:29 PM
Forbidden hole
 
11:45 PM
@Mikhail A forbidden hole happens when you try to pass current through a semiconductor that exceeds the charge carrier density of the material (e.g., as computed with the Fermi integral).
 
Fuck, you might know more semiconductor physics than me
 
@Mikhail Fermi was indeed quite the manly man.
 
I got a D in that class
 
@Mikhail I suppose it's possible, but that sentence was roughly what you might get from a Markov chain after feeding it a few textbooks on semiconductor physics.
Well, maybe a tiny bit better than that, but it's basically nonsense anyway.
 
concept of exceeding charge carrier density is wacky
 
11:57 PM
@Mikhail Like I said--it's basically nonsense.
 
@JerryCoffin Let's try feeding astrophysics and maternal health studies into the Markov Chain.
 
"astrology"
 
@Mikhail Ass-troll-ogy.
 

« first day (2861 days earlier)      last day (2080 days later) »