« first day (3791 days earlier)      last day (1142 days later) » 

12:03 AM
yo
 
@Mgetz Well, the big thing wasn't just "they want those capabilities", but "they'd like those capabilities with lower latency communication between the CPU and the GPU whatever provides them".
 
@nwp "worst of both worlds?"
Reminds me of how screwed up multiprocessing/threading is in python
First time hitting some NUMA related issues, reminds me that the Linux scheduler is also terrible
 
No, stand back and listen to Jerry for a short while.
again...
 
12:37 AM
@Mikhail It's not so much that the scheduler is terrible, as that the scheduler is given an impossible task, so they do the best they can under the circumstances, but with the surrounding architecture, nothing anybody could do would really make it work well.
 
Meh, zero of these problems on Windows
Migrating threads for amusement across numa boundaries is probably not a good idea
 
@Mikhail Yeah--Linux fans hate to hear it, but having your graphics server in a process of its own, so all graphics calls execute with its priority instead of the priority level of the caller makes things a mess.
 
I wonder if the real problem is that Linux doesn't use a data driven approach for its resource management. For example, things like memory overcommit or thread migration require heuristics but in Linux they seem to come from some dude in 2001 and with Windows they come from actual measured profiles?
 
@Mikhail That doesn't help either, but in the case of X, I don't think there's any amount of measurement that could fix the problem.
 
Yeah but we're talking about something different ;-)
 
12:54 AM
@Mikhail I'd go so far as to say that Linux's memory overcommit fits into the same category: something broken on such a fundamental level that no amount of measurement can fix it.
 
That is an interesting thought, what is broken with it with the exception of the actual scaling numbers used?
 
 
2 hours later…
3:15 AM
I wonder if std::equal requires values to be of the same type?
 
 
5 hours later…
8:12 AM
@Mikhail The signature of the predicate is supposed to be bool pred(const Type1 &a, const Type2 &b); so it clearly caters to mixed-types comparisons
 
 
5 hours later…
1:22 PM
@Mikhail different approaches. On linux if you want to spawn 128 threads for 128 cores you just do it. On windows you need to deliberately create 64 of the threads in the other NUMA domain. Windows takes a more explicit approach, forcing the programmer to declare they really want to handle the issues of NUMA.
NT in effect forces opt in of NUMA support, whereas in many ways Linux requires opt out or opt in of better support using libnuma
 
1:35 PM
@JerryCoffin and yet IIRC the latency to the EUs wasn't great?
 
 
3 hours later…
4:38 PM
@Mgetz The question here would be "compared to what?" Latency was higher than doing CPU stuff, but pretty low compared to doing things via PCIe.
 
4:53 PM
@JerryCoffin The issue as explained to me by someone in the HPC world at the time was thus: the latency far outweighed the vector advantage for most kernels which would exceed the local memory for the EU. If you exceeded the local memory for the EU you'd constrain the entire processor as it only had one memory controller. To add insult to injury if you were actually writing a game you'd still be constrained by PCI-e anyway as you'd need to get from EU to PCI-e.
Basically it wasn't enough of a GPU to do GPU things well... and it wasn't enough of a CPU to do CPU things well in comparison to the more conventional xbox 360. It suited a few specific workloads very well but the vast majority were better served by the up and coming (at the time) GPGPU.
In fact at SC08... MIT was supposed to show up and kick everybody's butt using a dual AMD GPU system, but because undergrads are more grads there... all the students pulled out because they couldn't get a grant. But their professor showed up and was awesome
 
@Mgetz I guess that doesn't surprise me. The design for the BlueGene was a pain to use, but at least worked out. To get the price down for the PS, they had to reduce the capabilities to the point that it probably didn't make sense any more (or only made sense under very narrow conditions).
 
@JerryCoffin Bluegene IIRC didn't even use the full fat Cell but rather a cost reduced 4EU version that didn't actually have the CPU core. That was handled by a full fat POWER core on the server board. I think those cards also had more dedicated RAM per EU as well.
 
@Mgetz Yeah, the RAM per EU was what made the big difference. You could load it up, let it run, and do other stuff in parallel for a while. On the PS, you had so little RAM you spent more time setting it up than it saved by running on its own.
 
@JerryCoffin not just that... but it was interconnect bound hard, everything had to go through the main CPU to get to memory. That slowed the main CPU down and restricted available bandwidth on top of the already super constrained (and shared with the GPU) RAM
 
5:12 PM
@Mgetz I'd almost forgotten about the degree to which memory access was a bottleneck. Now you've brought it all back, and I won't be able to sleep for hours. I guess It's a good thing it's still morning here...
 
@JerryCoffin let me put you back to sleep, AFAIK it's a completely dead architecture for the most part as a result of all of that. Most people just use GPUs for anything that was remotely good at now
Honestly? I'm still kinda surprised @Mystical hasn't done a GPU version of YCruncher yet
 
We should make fun of him for being scared of new things :-)
 
5:57 PM
@Mikhail As I recall, he did look into it a bit at one point, but was pretty sure that was no way to make it work out. For large computations, I'm pretty sure it's already usually storage bound. The big gain would be an SSD with really high write endurance, or (far better still) a machine with a few hundred terabytes of RAM, so you could do a huge computation all in RAM instead of having to store most of it on hard drives all the time.
 
Okay guys, this has been bugging me for some time now: what's the sz in szPath? It's littered all over the docs, and it seems like some inside joke.
 
size?
@JerryCoffin Fun part here is that you can do that today. I remember when a Google engineer fired up a extremely expensive cloud and kept it running with his software. It was supposed to be somewhat of a whitepaper promoting Google cloud, but in my mind it backfired. They could have purchased a whitebox for their cloud computing bill.
 
@I.Am.A.Guy s = string, z = zero-terminated.
When Win API was new (and before, when HN was new) Pascal-style counted strings were still pretty common, so they felt obliged to qualify what sort of string was involved.
2
@Mikhail Yeah, I think the advantage (if there is one) would be in reliability, predictability, and such. Compare Google (crank it up, a few months later, it's done) to some of the previous (guy spending six months babysitting a pretty insane machine, at one point a month or so delay while he ordered a more hard drives, etc.).
 
6:17 PM
@JerryCoffin But I also think there are no real high IO nodes on the cloud.
 
6:32 PM
@Mikhail I've never paid close enough attention to how they configure their machines to have a good idea about that. Just based on what it seems like they're mostly used for, I'd expect them to optimize for doing database stuff.
 
So, for hard drives used for large storage requirements they basically tank at 400 MB/s. Is is like 3 drives on RAID0. When I was building JBODS, with the right controllers I could do ~1500 MB/s on RAID5 with 24 drives. RAID 0 would have probably saturated the link.
Their local NVME storage is also limited to a single chip, but any kid can just put two in their RGB gaming rig.
The difference in IO is one of the differences between cloud and super computers. Generally, super computers need more IO throughput for constant checkpointing, etc.
 
6:47 PM
It's hard to quickly distinguish between @JerryCoffin and @Mikhail avatars. So, can you guys fix that? :)
 
@StackedCrooked Hmm...I have lots other pictures (and I have been using this avatar for a long time--those bits may be wearing out).
 
Hello how can I create an infinite self recursive union which contains one element of itself?
Btw ranges are very cool except that msvc IntelliSense chokes when I use them.
 
@JerryCoffin Nah, don't change it.
We've become too attached to the wasp image.
 
@AnArrayOfFunctions At least offhand, I'm not at all sure you can. Until the end of the union declaration, it's an incomplete type. I'd have to check to be sure, but I'd expect the type of each member has to be a complete type.
 
7:33 PM
@JerryCoffin Thanks I didn't think so either.
 
Are you looking for a recursive variant? boost::make_recursive_variant
 
 
3 hours later…
10:14 PM
Ugg, I got stuck on some std::move stuff. I got something like auto *object = reinterpret_cast<T *>(other.data) and new (memory) T(std::move(*object)). I'm relying on default move constructor in T to move all members, but somehow they aren't moved.
 
10:34 PM
Whats even weirder is that the stack trace has these copy constructors that I didn't write and operate on pointers. Looks like T::T(T* const this), what the heck is that?
 

« first day (3791 days earlier)      last day (1142 days later) »