« first day (2030 days earlier)      last day (2909 days later) » 

12:17 AM
@sehe You have some experience with boost.log correct?
 
Very little. I don't like it
 
Ah nvm then
 
;)
 
12:33 AM
Oh wtf... The memory timings on this thing are 31-31-31-63.
Maybe that's why the machine seems so slow.
 
WTF locks almost look too good to be true. Definitely looking into those. https://twitter.com/filpizlo/status/728600371770347522
@Mysticial huh
 
@sehe Not done reading, I'll do that later but I feel like there must be catch
 
@sehe Memory timings for DDR4 are supposed to something like: 14-16-16-31.
That server has them twice of that.
Which means twice the memory latency.
 
@Borgleader I figure there might not be. Pthreads is old. Kernel locking is a known no-good for perf and modern CPUs have atomics in their DNA.
 
It could also be that CPUz isn't reading it right.
 
12:42 AM
@Mysticial That's going to be noticeable
 
@sehe that’s quite the novel
 
@Mysticial Where there's smoke...
 
Theres a hippie with a lot of weed
 
@LucDanton That's a GoodThing IYAM (better than the thing about floating point innovation recently. I hope?)
 
Floating point innovation?
 
12:44 AM
yeah but the tweet had me puzzled
although in retrospect that has much to do with the questionable WTF initialism
 
@elidourado @sarahjeong 1 phone lady "reading scripts verbatim" says account+routing suffice, and the journalist cries "wolf"!? #unimpressed
Journalism doesn't really seem to exist
@LucDanton Yup me too. That's the reason I didn't do straight re-tweet, but paraphrased/commented on it
Mind blowing! Ditch IEEE float representation, gain precise 1-clock binary ops with no NaNs. http://www.johngustafson.net/presentations/Multicore2016-JLG.pdf https://t.co/cPFoxSrztO
 
12:59 AM
oh, it's this
2.0?
let's see what they changed
tbh this is pretty cool
I find the fact that people work on such low-level details... heartwarming
 
Hah. @VisualStudio must have heard me complaining
Our bad! - Sorry for the recent volume of tweets as we uploaded new content to our YouTube channel. Enjoy your weekend.
@milleniumbug It is. But there are caveats. Big ones
@milleniumbug That's a nice choice of words. Yes.
 
1:16 AM
anyone here is a mac expert?
 
We have the kitty for that
 
@sehe mind blowinger: ⁻¹
 
:D
 
err ...
something strange happened to my mac ~facepalm~ ~embarrassed~
now I am running this command:
ls -1A {,/}Library/{Ad,Compon,Ex,Fram,In,La,Mail/Bu,PP,Priv,Qu,Scripti,Sta} 2> /dev/null
I am checking what should not be there ...
I AM SO USELESS!!! cries
 
@sehe thx
@Mysticial hey you gonna try the overwatch beta?
 
1:37 AM
> Since gcc 3.4.5 is used in the whole company, I have to use rtree of boost 1.57 because rtree of boost 1.59 has compile problem
No way. It's 2016
 
@Borgleader overwatch?
 
@Mysticial "Blizzard Team Fortress"
its in open beta right now, figured you might like it
 
oooh
Clearly I've been too immersed in computers lately.
Both at work and at home.
 
2:11 AM
@sehe lol
@Zoidberg hahahahahaha
> Circumcision required for ISIS recruits?
> if two boys do anal sex (zina) and discharged , and then do toba with real heart...then they are permissible for nikkah with sisters of each other(nikkah of 1st boy with sister of 2nd boy and nikkah of 2nd boy with sister of 1st boy)?...Nikkah is allowed or nikkah is haram (forbidden) for them with sisters of each other?
lmao
IOW if I fuck a guy is it haram to fuck his sister next
This site is a complete catastrophe
 
I am always amazed by the amount of time you spend on reading islamic sites
 
hi
can you help me with VB.NET
8
 
why would we be able to
does knowing c++ secretly give you wizard powers where you automatically know languages and frameworks without investing time in them? has my whole life been a lie?
it gives you wizard powers doesn't it.
why did no one tell me‽
so much opportunity, wasted.
...I suppose The Council knew I would do evil with this knowledge, and forbade telling me.
thank you for letting me in on the secret, friend. :)
now I have ...work to do.
 
@DmitriBudnikov Deleted or flagged?
 
@Telkitty that's pretty cute
 
:D
 
@Borgleader deleted
 
Fuck its 10:00pm and I'm working on a Friday.
 
2:53 AM
@Mikhail I'm sorry about that.
 
I'm also sorry, we have so much in common.
Does anybody have MSVC 2015 handy? I'm curious if the following answer compiles:
http://stackoverflow.com/a/32287802/314290
 
No, but I have 88 graphs in Task Manager. :P
 
Yet, still no penguins
You could have 88 penguins!
 
Gimme a few min.
 
3:08 AM
long time no see @Mysticial :p
 
hey
 
anyway - someone's got a time out - and I'm going back to sleep - see you around mate
 
Woah, a month? What context did I miss?
 
3:36 AM
@Mikhail At 44 cores, I can actually see a bit of Amdahl's Law show up in the unparallelized large number adds and subs.
But just barely.
 
Anybody tried Qt3D? Last time I touched Qt's OpenG in 2012, I found it really bad for multitasking as I wasn't able to load assets on a second thread. Now I got a faster GPU and this isn't a problem. Anyways, did Qt get better?
 
^^ Those dips in CPU usage = Unparallelized big number additions and subtractions.
They don't show up on your average box.
 
3:53 AM
What the fuck.
 
when ur cpu usage goes down, it means ur not using it as much as u could
it indicates poor optimization of usage
OoU, for short
 
4:32 AM
-3
Q: Why does C++ break down into nibbles?

SilcoreWhy is information stored in sequences of four bits (nibbles)? Is there any particular reason that four bits were selected, over perhaps three bits, or five bits? I've just been wondering about this question, and I haven't found a definitive answer (if there is one) as to why we group bits in thi...

nibble. now there's a tag
those pesky groups of bits causing problems
rip 44k guy
 
I really do love the moon logic questions
panta ray?
nah he knows what he's doing
 
@Mysticial looks good though, overall
 
@doug65536 It's worse towards the end.
 
that solid area probably represents an awful lot of work getting done
 
The inverse square root spent almost half its time doing unparallelized linear operations like add/sub/invert bits...
I never parallelized O(N) operations like those because they're memory bound.
But yeah. Having written this thing on quad and 8-core systems. It's nice to see first hand what it looks like on 44-core.
It takes 44 fucking cores to see Amdahl's Law inefficiencies.
But to be fair, the memory/NUMA bottleneck is a much bigger problem.
 
4:41 AM
yeah but you probably started with good work... you could easily have serialized it to hell if you were bad
 
It was designed from the very beginning to be parallelized.
 
I would like to see sudo perf top -m cache-misses of a linux build if I were you
 
So I knew what to avoid in the first place.
 
LLC-misses? I always forget the exact event names
oh there are node counters too.. for migrations and stuff node-load-misses
 
But linear stuff like add/subtract is basically unoptimizable unless you merge it into something else. But that makes things much more complicated. So I opted out of that. I guess it only took 7 years before I can finally see it show up.
 
4:45 AM
yeah, sounds right. you cant focus on everything
 
We're at the point where the memory bandwidth per node is high enough such that a single thread can no longer saturate it. So it might actually be worth doing some parallelism for linear operations.
 
DDR4 at high clock speeds in quad channel with a gigantic L3 cache sounds good to me
 
But that's later. The much bigger fish to fry is the cache efficiency and NUMA.
 
@Mysticial you've tried perf top in linux, right?
it is really good. real time "top" style event based sampling, and you can hit enter on a line and it shows annotated asm of the code with counts per line. updated in realtime
 
@doug65536 I don't have a NUMA system.
But I kinda already know where all the problematic areas are.
And it'll take a pretty significant redesign to fix things.
 
4:52 AM
44 and not numa? windows wont allocate memory locally if it thinks it is one node. right?
every cache miss on the other nodes probably pulls it from the main cpu, no?
 
It's not my box.
 
if it did, then it would be bottlenecked on one bank of dimms
all the loads and stores would be to the main cpu's sticks
 
The program's memory allocation doesn't play well with NUMA.
It allocates everything at the beginning (on one thread). So the OS will bias it onto one node.
So you're using only one bank and utterly destroying the interconnect since the other node wants to use the same memory.
 
exactly
 
You can solve that by turning on node interleaving.
So some extent that is.
 
4:56 AM
every miss/writeback on the other nodes will bottleneck against cpu0's sticks of memory
 
It spreads out the traffic so at least you get the full bandwidth. But it destroys locality.
I wrote a post about it on HWBOT and the owner decided to put it on their news page.
 
yeah interleaving makes the worst case better
at least 1/Nth of the time it is local, and it spreads out pressure on non-local dimms
 
The problem first appeared back in like 2010-ish when the program became popular enough that people tried running it on quad-Opterons.
Back then the program wasn't that well optimized for memory usage.
 
normal people use 44 cpu machines to host a pile of separate vms. in that case, locality is a non-issue
 
So in 2012-ish, I started working on a completely new class of algorithms/implementations that was cache aware.
And that actually turned out really well. It scaled very nicely on the really large Intel systems.
The problem is that it did it by simply being very cache friendly. I hardly touched memory at all. So regardless of how slow the memory was, it would run fine.
Now...
Fast forward to 2014. Those "new" algorithms have gotten 3x faster thanks to ISA improvements. So even those are becoming memory bound.
So I'm back to square one. And at this stage, becoming NUMA aware is basically unavoidable. The problem is that will require a fundamental change in the way data is stored. So it's gonna be affecting the program from bottom up.
 
@doug65536 I do wonder why they do that. It's cheaper to just get multiple smaller boxes.
 
saves electricity
a disk and CPU and network intensive workload can share the machine without interfering with eachother
 
It's gotta be more than just that. The price premium to pay for the bigger box is much higher than the power bill.
 
or so they hope
 
That 44-core box is "theoretically" about 3 - 4x faster than my 5960X. But looking at the specs, it probably costs around 10x more.
 
5:12 AM
less space, less power supplies generating heat, less separate redundancies equals less air conditioning
google is a big air conditioner with ad popup generator and search engine tacked onto it
 
I guess the price comparison isn't completely fair.
My current box can be built for around $1800 USD now. And it only has 64 GB of ram.
 
For my applications, you need to include $1000 dollars of hard disk controllers.
 
That 44-core box has 768 GB of ram. The raw price of that ram seems to be about 6-7k USD.
 
@Mysticial the cooling kills you when you scale up
 
@Mikhail That box has 16 x 6TB drives on it.
Completely empty.
That machine runs consumer Windows 10.
 
5:15 AM
Noobs, I have 24 6TB drives, and they are also completely empty.
 
It's not a corporate machine.
 
You can actually get Windows Datacenter for free from MS academic or BizSpark
 
It seems like he personally owns it. It's got nothing installed other than benchmarking programs.
And he gave me full root and BIOS access to it for an indeterminate amount of time.
 
He better, with the money he laid down.
You should install incredibuild, for fast build times.
 
Given that I haven't paid a penny for this. I think installing stuff is probably crossing the line unless he explicitly tells me it's okay.
@Mikhail Can you beat this?
I had to disable the HT on it. It wasn't helping.
 
5:20 AM
No, I only have 2 socket systems, with the same chip as him
But I have more empty hard disk space
There should be a HWBOT benchmark that checks the reported empty space on your computer.
 
1. The program is too memory bound to get anything from HT.
2. Turning HT pushes the vcore count to 88 which is over 64. Combine that with a buggy Windows and it leads to unbalanced processor groups which fucks up the program.
There's a lot of interesting BIOS options in there that I'm not familiar with. Since it's been a few years since I've been able to play with a dual-socket box.
Lots of cache coherency policies.
A whole bunch of memory policies (mostly involving ECC) which I didn't care much about.
Nearly full control of the C-states and power throttling.
 
You can get a 2% performance boost if you disable ECC
 
He already set all the obvious things for maximum performance.
So I didn't have to mess with the memory policies.
They have mirroring policy which cuts your memory in half and dupes it.
The cache ones are interesting.
They have 3 different snooping policies and a cluster-on-die option which splits each socket itself into multiple NUMA nodes.
While these server systems lack overclocking options, they have a shit-ton and a half of stuff like this which you don't find in an overclocker board.
 
I think you can overclock them, but you need to do it manually...
Just change the voltage, etc
 
One ting that really bothered me is this:
I can't tell if CPUz is bugged, or if those numbers are actually real.
But if they are real, that could be another reason why the memory access sucks.
 
5:31 AM
Mirroring?
 
I just noticed that it says 1.5 TB of memory. But my program shows 24 DIMMs of 32GB each for 768 GB.
It looks like CPUz is bugged.
The BIOS confirms that as well.
 
you'd probably benefit fully from faster ddr4 memory
tell 'em the machine isn't good enough, lol
31 cycles cas at 1066? I dont believe it. can't be right
 
I wouldn't be surprised if it was correct. I believe the chips are engineering or pre-release samples.
 
those numbers look like all binary 1's
 
And they'll have quirks like that.
I think CPUz is wrong.
It gets the total memory wrong.
 
5:46 AM
using 1066 on a 44-core in 2016 is tragic. can't be right
 
My unreleased version of the program can read the motherboard and memory specs.
Motherboard:
    Manufacturer:      Dell Inc.
    Model:             0H21J3
    Version:           A09
    Serial Number:     Suppressed - Personally identifiable information is opt-in only.

Memory:
    Usable Memory:     824,532,762,624 ( 768 GiB)
    Total Memory:      824,633,720,832 ( 768 GiB)
    DIMM:              32.0 GiB - 00CE00B300CE - M386A4G40DM0-CPB
    DIMM:              32.0 GiB - 00CE00B300CE - M386A4G40DM0-CPB
    DIMM:              32.0 GiB - 00CE00B300CE - M386A4G40DM0-CPB
    DIMM:              32.0 GiB - 00CE00B300CE - M386A4G40DM0-CPB
That memory really is 2133. So the clock speed is right.
 
yeah, the machine is good enough
around 68.2GB/sec per node
2133000000*8*4
 
My 5960X looks like this:
My uncore is only 1.2 GHz since the chip is mostly idling atm.
But it'll go up to 3.6 GHz.
So my 8 cores has more bandwidth than 22 cores on that box.
 
5:58 AM
what is the ram speed, when throttled up
 
It's always 2x the DRAM frequency.
2400 MHz.
 
wait though, your one node has more memory bandwidth than that one node
 
Correct.
 
but the 44 cores multiple quad channels will smoke your one node
but ya, that's cool that your local machine is comparable to a node of a big server
 
Right. But that's 44 cores. I only have 8 cores.
Granted my cores are at 4 GHz. And his seem to be at 2.5 GHz under load.
But still. lol
 
6:00 AM
Do they also have more cache?
 
55 MB/socket.
Mine has 20 MB.
 
lol, that cpu is a giant memory subsystem with a little square in the middle that executes things
 
It looks like they kept the same overall design as Haswell. 1.25 MB L3 cache per core.
 
Dual channel?
 
6:04 AM
no way
each channel can only drive 2 dimms. do the math
 
Is that a desktop Skylake with 4 x 16GB sticks?
Those are dual channel.
 
driving the parasitic capacitance and inductance of 2 dimms interfaces at 2144 is pushing it
 
2 channels is fine for desktop chips.
 
how many sockets is it
 
That 44-core box?
 
6:06 AM
yeah
 
Processor(s):
    Name:              Intel(R) Xeon(R) CPU E5-2696 v4 @ 2.20GHz
    Logical Cores:     88
    Physical Cores:    44
    Sockets:           2
    NUMA Nodes:        2
    Base Frequency:    2,199,997,664 Hz
 
24 / 2 = 12. so they are probably 6-channel
with that many cores, 6 channel sounds barely adequate. works because ddr4
 
I think it's 4 channels per socket. 3 DIMMs/channel.
I can't read Chinese, but that CPUz screenie does show the Chinese number for 4 in the channel box.
 
ok that sounds right. 1st time I have ever heard of 3 dimms on a channel
 
Back in the Nehalem days it was even weirder.
18 slots on a board.
Triple channel with 3 DIMMs per channel.
Two sockets makes that 18 sticks.
 
6:12 AM
yeah, because 6 = 1.5 * 4, and with 4 channel quad core, you obviously need 6 channel 6 core
and 6 cores have 50% more cache too
the whole memory bandwidth was upped 50% when core count upped 50%. thanks intel
 
What are you referring to?
 
4 core nehalem vs 6 core nehalem
they scaled the 6 up 50% in both L3 and channels
 
Oh
The first 4-core Nehalems had 4 cores for 3 channels.
 
Does somebody in here program in Java?
 
My 920 was like that.
 
6:14 AM
I had a 990x. you should have seen the cashier when I put a 990x and 2 gtx 580s and 24GB of ram on the counter in 2008
 
I'm desesperated for a solution to a problem.
Sorry to bother you.
 
I remember that day. I walked out of Frys after spending $1400 for a 920 with 12GB of ram and a mid-end video card + rest of the system.
That was my first gaming rig.
I had to build it because my dual X5482 server went down and I needed a 64-bit machine to continue coding while I waited for the RMA turn-around.
 
lol mine was from Fry's Electronics too! I love that name
 
I have a GT740M. Please kill me.
 
That 920 was my last flawless build.
Every single build I've done since then has had early hiccups.
My Sandy Bridge rig got caught up in the chipset bug.
My FX-8350 had bad memory which I had to exchange.
My 4770K was so heat-limited that I couldn't get a satisfying overclock out of it.
My 5960X also had bad memory which I had to exchange. lol
 
6:19 AM
Okay, I'll go. Bye.
 
@JavaNoob probably can't help, I never use java for anything
 
is there an application where I can find a formula of best fit through a seemining exponential line
 
and this is the wrong room anyway
@Telkitty excel
chart thing can find the approximate equation of the curve
 
I know; but there are no people in the Java rooms, so I thought that I could find a Java programmer that programs in another language too.
Bye
 
@doug65536 what about google doc spreadsheet?
 
6:21 AM
yeah probably has it. just guessing though
 
You should try Matlab. It only costs an arm and half a leg.
 
I used matlab for years, but I am not going to use it for once off thing since I don't have it installed at home
 
cool thanks
 
6:28 AM
So Skylake-EP is supposed to have 6 channel DDR4 per socket.
So maybe we'll once again see weird DRAM counts again.
And possibly 12 DIMMs on the HEDT boards. 192 GB of memory in a desktop.
 
@Mysticial yeah that makes a lot more sense than 3 banks on 1 channel. with crazy core count like that
 
polynomial != exponential
 
We might even see boards with 36 DIMMs if they do 3 sticks per channel.
It'll be a tight fit though. Even for the super-extended ATX boards.
 
@Telkitty you can make it solve multiple unknown equations too
@Telkitty lol
it's the star trek computer man! just say what you want
 
got it, thanks
fit exponential {{2000, 6127},{2005, 6520},{2008, 6764},{2010, 6930},{2012, 7097.5}}
I am trying to draw a line of best fit through world population
 
Too bad ram is maxed at 2^48
 
256PB is all you will ever need
 
Someone can recommend me a good boost asio programmer?
 
much cheaper to use std::future
 
@Mysticial makes the promises of thousands of cores sound completely hypothetical, if we're bottlenecked like this with < 100
 
6:44 AM
Think of the cache sizes
 
@doug65536 It's doable. It'll just take an extra generation.
The hop from shared memory to NUMA will be similar to single-core to multi-core.
It'll fuck up all the programmers of the previous generation.
 
But only for cache optomization
 
have you made all the nodes fight over the same cache line across nodes? I've never gotten around to trying that
 
@doug65536 I've actually been doing that at work for the past few weeks.
 
i.e. cmpxchg the hell out of one dword from every core
@Mysticial lol
 
6:47 AM
So, if I have a millions of 10 megabyte images, should I multithread row by row, or image by image?
 
@Mikhail you will be I/O bound
 
They had a bunch of low-level stuff that needed to be done. And when they interviewed me, they realized that I was probably the right person to do it.
So I got the job.
 
@Mikhail image by image will be faster, less interference
but your threads will be bored waiting for I/O
 
assuming not io bound
 
definitely image by image. concurrency is faster than parallelism, automatically, because no tripping over eachother
 
6:49 AM
You definitely want to parallelize at as high a level as possible.
With 88 threads, the time needed to sync them is visible in Task Manager. (barely)
 
concurrency = each thread processing its own image, parallelism = all the cores fighting over cache lines in the same image and all having to start and finish at the same time
getting all the threads to start and finish at once is what kills you with parallelism
you end up with bubbles where you dont have all of the threads going on it yet
 
@doug65536 Unfortunately, that's quite common in fork-join parallelism models.
 
my advice for that: have a lock that you acquire to read disk, so only one thread reads at a time, you will serialize against the I/O at first, but eventually you end up with all the cpus busy. it helps a lot when you have only one spindle of disk. no thrashing around
 
Fortunately, the latency of that is (theoretically) O(log(p)). Though I think in practice, it's more like O(p).
 
another approach is to have one reader thread feeding workers with a task per image. you read sequentially because the disk is sequential, but you hand off to parallel workers
if I/O is fast enough it gets all cpus going
if not, gets as parallel as it needs to keep up
 
6:55 AM
If you can pick how the images are laid out in your 24 hard drives, then you can do what I do in my pi program. 1 thread per disk for massive disk parallelism.
 
yeah, and that approach hides a lot of latency by getting started reading before a cpu is free to process it
 
I guarantee you that 1 thread will not be able to keep up with 24 hard drives unless you're doing something stupid like a memcpy.
 
1 thread per spindle
I was assuming he had a desktop :)
 
His box has 24 drives. 12 cores and 256GB of ram.
Though the 12 cores aren't very high clocked. So it's actually slower than my 8-core.
But the ram is impressive.
 
24 7200rpm disks? or ssds?
 

« first day (2030 days earlier)      last day (2909 days later) »