« first day (2802 days earlier)      last day (2151 days later) » 

12:05 AM
@sbi To be fair that is heavily amplified.
@StackedCrooked I also felt challenged by that. There is of course the dude that defines precision to be a musical note; youtube.com/watch?v=3j8mr-gcgoI. Then there's always the champ of champs: youtube.com/watch?v=oEfFbuT3I6A
 
12:34 AM
Have you guys ever ran into this problem: stackoverflow.com/q/34332117/314290 ?
At the heart of the problem is that the Fourier transform is periodic causing edge artifacts, either because the FT sees a mirror image (if you mirror) or zeros (if you pad) . But I'm not sure what the solution is.
 
what do you think of the suggestion of filling with mean(?) values
 
I suspect that if you just go the frequency space you can apply your filter directly, but sometimes your function is singular in the frequency domain, but you still gotta convolve it. For example, 1/k <-> sign(x), where sign(x) is perfectly fine in coordinate space, but 1/k isn't in momentum space.
Yeah, there are indeed some tricks to reduce the problem, but I don't see a clear way if you need to get an exact solution, given infinite compute time. Maybe you can expand the simulation domain and cut out the boundary?
 
what is the exact solution?
 
As if the functions weren't periodic
Basically, the way to reduce artifacts is to use a much larger region and cut the edges. You can, for example, pad your signal (and download more RAM) with flipped over copies and only keep the middle one.
But in my case one of the signals is generated analytically, so maybe I can resolve the edges by the analytic signal larger than the experimental one (in coordinate space)
The problem with what I just wrote is that it assumes there is no signal outside of my measured signal, which in this case is true, but generally, not...
 
 
4 hours later…
4:36 AM
@Ven thoughts on using different brackets for a hash context instead, e.g. %[] or %{}? do these come with pitfalls of their own?
 
 
4 hours later…
8:11 AM
@CaptainGiraffe also using a coil pickup if I'm not mistaken. But the acoustic nature of it will lend a much better sustain/resonance
 
 
1 hour later…
9:14 AM
Picture taken by my current drone, at the local park within 100 meters away from where I live >_<.
 
 
3 hours later…
Ven
11:50 AM
@LucDanton %{} is already something "else".
 
12:10 PM
@TelKitty Don't worry. You live about a continent away from anyone in this channel. :)
 
Yeah at this moment, but any part of the world would just be within 48 hours away and not awfully expensive if you fly cattle class ...
Actually you can see the house where I live in that picture. Although I am not going to tell the internet exact where it is.
 
@TelKitty The one with the trampoline, obviously.
 
...
 
nwp
12:25 PM
Now I want a drone too. One that can drone-strike and fly to australia.
Actually that's kinda mean.
I'll settle for one that keeps blocking the camera.
 
@nwp Heh. Drone wars!
 
12:48 PM
@Ven namely what?
 
Ven
@LucDanton %{Int => 500} vs {Int => 500}
 
I don’t follow
 
Ven
@LucDanton try .perl on both
Wait, it was supposed to be different wrt autoquoting? guess it's not anymore. So use %{}
 
no, I was looking for an unambiguous hash syntax that works for empty hashes, too
 
Ven
I think %{} is fine
 
12:53 PM
cool
 
1:03 PM
I need to replace std::get_temporary_buffer before C++20 :/
 
 
2 hours later…
2:46 PM
@Mgetz They should have created a new instruction instead, named "short rest".
 
I wonder what the motivation for that change was
a 14x change is pretty significant for a instruction that is essentially used as a timed nop
 
@ratchetfreak As far as I can tell it's based on bus propagation speeds, the prior pause duration was unrealistic for some packages leading to significant slowdowns and bus locks that were unnecessary.
it just so happens that .net was trying to be overclever instead of going "whelp we've spun for 10k cycles YIELD
they could potentially spin for millions of cycles
 
3:22 PM
@Mgetz Yeah, I read about that last year. Didn't notice a large difference in our use-cases since they're always paired with a cache miss which is much worse.
 
 
1 hour later…
Ell
4:30 PM
@StackedCrooked IIRC you implemented IP stack before. I have to implement IPv6 over Bluetooth for my master's thesis - do you have any recommended resources? Books? Thanks
 
Mostly reading RFCs.
Btw, IPv6 is not implemented directly on top of Bluetooth. It's implemented on top of a link layer (most likely Ethernet). So the fact that you're using bluetooth or something else shouldn't affect the IPv6 code at all.
Btw, IPv6 neighbor discovery is quite a pain to implement. Good luck with that :)
 
4:46 PM
 
 
3 hours later…
7:29 PM
@Morwenn What are you even using it for
 
was there ever an implementation that didn't just call malloc or ::operator new?
 
Maybe small buffers can be allocated using a stack allocator?
 
7:50 PM
@Puppy std::inplace_merge equivalent
@milleniumbug It's basically that, then repeatedly try to allocate a buffer half the size until you get a valid buffer or nullptr
Except the libc++ implementation seems to handle over-alignment or something
I would comment on that but tbh I don't understand much about alignment
 
@Morwenn wat? how are these two related?
 
@Puppy It tries to allocate memory, actually merges shit out-of-place in O(n) time if enough memory is available and gradually fall down to an algorithm that doesn't use memory at all and runs in O(n log n)
 
ah I see
 
At every step it checks whether the allocated buffer is large enough to perform an out-of-place, so it's actually fine if you get a small buffer
 
so it's bsaically just malloc but there's a bunch of extra runtime costs to handle situations that basically never happen?
 
7:57 PM
std::stable_partition does the same kind of thing IIRC
@Puppy basically
You need really specific conditions to be able to allocate heap memory and have it exhausted without causing a crash of some kind
In memory-constrained environments you'd probably just use the stable in-place merge algorithm by Kim & Kutzner that still runs in O(n), or allocate a small stack buffer to have it be a bit faster
 
8:22 PM
or not need to merge in the first place
 
9:18 PM
@Mysticial So, my code arena allocates ~200+ GB of RAM, on Windows it takes fucking forever (1+ minute) to start. Is there some trick to rapidly allocating memory on Windows? When I tried multi-threading the allocation and blasting 36 threads, the system became somewhat unresponsive.
 
@Mikhail No there isn't. Large pages may speed it up a bit, but it can also backfire if the OS tries to defrag the memory to make large pages.
So the best you can do it allocate once and pool manually.
 
Actually that's a good idea, so, I gotta go through VirtualAlloc for large pages?
 
And yes, if you multi-thread page-commit to heavily, you''ll put the system into some sort of catastrophic lock contention which takes like exponentially longer.
@Mikhail You also need the perms set for it.
And on older OS's (Windows 10 Creator's Update and earlier?) you also need admin.
 
Isn't the solution to perform page commits in a way that avoids lock contention?
I'm using Server 2012
 
@Mikhail The locking is in the kernel. The way to prevent it so limit the number of threads doing it simultaneously.
 
9:23 PM
Didn't you benchmark the multihreaded vs non-mulithreaded allocation on Windows? What was the outcome there?
 
In my pi program, IIRC, I use a limit of 2 threads per NUMA node.
 
But you see a speedup from using 2 threads instead of 1?
 
a little bit
Either way, the page-commit overhead in Windows is hilariously bad. A few years ago, I made library that exposes my pi program's bignum internals. And I also made a small project that uses it in a naive way to compute Pi and such. Basically every new object is a new allocation. And temp memory is allocated/freed at will since it's easy on the programmer.
On my 4-core Haswell, computing Pi to 100 million digits leads to about 50% if the CPU time spent in the kernel handling page-faults from the allocations.
On my 10-core Skylake, it's closer to 70 - 80%.
It's so bad that all the ISA optimizations (i.e. AVX) don't matter shit since the program literally spends all its time in the kernel spin-locking.
And now in post-meltdown world, I see page-table invalidation instructions added to the mix when I pull it up in VTune.
This is probably what they mean by, "Windows sucks for HPC".
On the flip side, I have a unit test runner that tries to budget memory and carefully chooses what tests to run at the same time to avoid running the system out of memory. This works very well in Windows. But in Linux, the actual memory overshoots by as much as 50%. Seems that either Linux or glibc is being much more aggressive with pooling that it - well - runs the system out of memory.
 
9:52 PM
@Mysticial it's kinda the difference between HeapAlloc and VirtualAlloc, glibc is in effect built on the latter. Whereas MSVC leans heavily on the former to do all its allocation (to the point they can't supply a c11 alligned_alloc)
@Mysticial Windows sucks for HPC for a lot of reasons, IO not the least
 
10:39 PM
@Mgetz When I worked in HPC, single node benchmarks (Lapack and OpenFOAM) were always a few percentage points higher on Windows boxes :-)
 
@Mikhail wut
 
@Mgetz better believe it, but nobody wanted to buy them :-)
 
@Mikhail ironically I do, I suspect the same things that went into the WSL supported that
 
Also multinode benchmarks on the target system, aka the CX1
 
FWIW you can do a lot at the driver level which helps answer a lot of that
you can get away with doing a lot of zerocopy stuff
 
10:45 PM
@Mgetz My disk-swapping/compute performance was historically typically better in Windows than in Linux. But that gap has narrowed to almost zero recently if all Linux is tweaked correctly.
 
@Mysticial not surprised
In theory the windows IO stack is more scalable IIRC, but it introduces latency
 
General compute was also better in Windows than in Linux. This was because Windows had a real thread pool whereas Linux didn't. This performance gap has narrowed since I implemented my own pool. But Windows remains ahead by a few %.
 
I wonder if hpc clusters will actually deploy spectre fixes. Most cluster nodes are partially managed by an external chip, and I'm not aware of real hpc where different users share a node.
 
@Mikhail I doubt it, quite a few disable stack canaries and ALSR too
 
@Mysticial What was the difference between atd::async thread pooling on Linux vs Windows? I think on Windows it sometimes pooled but on Linux, I never saw any pooling. Was that a compiler, libc issue or something internal?
 
10:51 PM
Linux/GCC still doesn't pool std::async.
 
@Mikhail windows std::async is no longer pooled, that wasn't actually compliant
 
@Mgetz wait wut? When did this start happening?
 
Well you can pool threads at a few places, including at the is level.
 
@Mysticial I could have sworn they released that change in VS2015 if not it's vNext
the standard actually requires a new thread if you launch async
 
When I tested it a couple years ago, std::async performance was indistinguishable from manually using Windows Thread Pool.
 
10:52 PM
it's pretty dumb
they may have fixed that
 
oh
I've been using my own thread pool since 2016. And I've only checked the older stuff (std::async, thread-spawning) for correctness and not for performance.
 
would not be surprised if someone raised a DR on that
 
My understanding was that if you spam enough, the OS will actually start pooling...
 
@Mikhail If you wanna test it, the latest version of my pi program was built with VS2017+ICC for all the AVX binaries.
Set the parallel framework to std::async launch, run the program through VTune and see if the thread life-time count is reasonable or through the roof.
@Mgetz Does it even matter if the thread is reused or not? Does reusing a thread violate the as-if rule?
 
@Mysticial only if thread_local is an issue, but I suspect that std::async doesn't make any guarantees there anyway
aaand I'm wrong
> If the async flag is set (i.e. (policy & std::launch::async) != 0), then async executes the callable object f on a new thread of execution (with all thread-locals initialized) as if spawned by std::thread(std::forward<F>(f), std::forward<Args>(args)...), except that if the function f returns a value or throws an exception, it is stored in the shared state accessible through the std::future that async returns to the caller.
it's all about thread_local
I suspect you could still implement that on a thread pool
 
11:06 PM
Oh. That's shit.
But you can still reuse the thread by destroying and re-initializing all the thread locals.
 
yeah let me peek at the CRT and I'll let you know
Honestly looks like it's irrelevant if you pass the deferred flag
> If the deferred flag is set (i.e. (policy & std::launch::deferred) != 0), then async converts f and args... the same way as by std::thread constructor, but does not spawn a new thread of execution. Instead, lazy evaluation is performed[...]
 
@Mgetz But if you pass the deferred flag, GCC/Linux will shit on you.
 
@Mysticial they are legally allowed to do that yes...
because GCC sucks in various random soul deadening ways
 
@Mgetz It's one thing to be legally allowed to shit on someone's face. It's another to actually do it.
 
45 secs ago, by Mgetz
because GCC sucks in various random soul deadening ways
Honestly I think it was laziness. They knew they could get away with it. There is no OS facility for a threadpool and Linus AFAIK is firmly opposed to adding one, so they said 'screw it'
 
11:17 PM
That makes sense. It isn't the first time Linus has shit on people/things.
 
He makes it a habit... and he's often wrong.
I doubt he cares or wants to care about a facility primarily for multi-threading when the Linux philosophy is to just make a new process.
Linux has fundamentally broken threads anyway
 
11:36 PM
@Mgetz kinda makes sense, if you hire a significant amount of compute then the only thing running on there is your own stuff so why would you need to hack around in that
 
@ratchetfreak If the AVX Spectre thing turns out to be real, I'm gonna laugh and cry at the same time if the fix is to disable AVX at the OS level by turning off XSAVE.
Monday is almost over and still no reply from Intel. If nothing happens by the end of the week, my blog is going up.
 

« first day (2802 days earlier)      last day (2151 days later) »