last day (16 days later) » 

10:00 PM
7
A: What is the penalty of mixing EVEX and VEX encoded scheme?

Peter CordesThere is no penalty for mixing any of VEX 128 / 256 or EVEX 128 / 256 / 512 on any current CPUs, and no reason to expect any penalty on future CPUs. All VEX and EVEX coded instructions are defined to zero the high bytes of the destination vector register, out to whatever the maximum vector width...

 
@harold As far as state transistions go, there's no difference between a 512-bit operation and any other 256-bit operation. But executing any 512-bit instruction (having any 512-bit instruction in the reorder buffer) causes the core to enter a "512-bit mode" that shuts off port1 to all vector instructions and gives the resources to port0 for dual-issue 512-bit (port0+port5) instead of the usual 3-issue 256-bit (port0+1+5). So interspersing small amounts of 512-bit code into a lot of 256-bit (or smaller) code will hurt performance of < 512-bit code.
Likewise, executing any 512-bit instruction causes the core to throttle down do its AVX speed. And executing any 512-bit instruction that uses the FMA unit causes it to throttle all the way down to its AVX512 speed. Though I have yet to determine if either FMA (port0+1 or port5) will do this or if it's only the port5 FMA. I have some 512-bit code that sparsely uses the FMA unit and only throttles the core to the AVX speed. I can see how that happens if the FMA usage is sparse enough that they all get dispatched to port0+1 and none go into port5.
 
@harold - there is a new, pretty bad effect with AVX-512 instructions on surrounding code: once a 512-bit instruction is executed (except perhaps for instructions that don't write to a zmm register) the core enters an "upper 256 dirty state". In this state, any later scalar FP/SSE/AVX instruction (anything using xmm or ymm regs) will internally be extended to 512 bits. This means the processor will be locked to no higher than the AVX turbo (the so-called "L1 license") until vzeroupper or vzeroall are issued.
Unlike the earlier "dirty upper 128" issue with AVX and legacy non-VEX SSE (which still exists on Skylake Xeon), this will slow down all code due to the lower frequency, but there are no "merging uops" or false dependencies or anything like that: it's just that the smaller operations are effectively treated as 512-bit wide in order to implement the zero-extending behavior.
@PeterCordes - about "IDK if simply writing the low halves ..." - no, it is a global state, and only vzero* gets you out of it. It occurs even if you dirty a zmm register but use different ones for ymm and xmm. It occurs even if the only dirtying instruction is a zeroing idiom like vpxord zmm0, zmm0, zmm0. It doesn't occur for writes to zmm16-31 though.
@PeterCordes - I'm not sure if it's the same state as the p1 shutdown that Mysticial describes, but it seems unlikely? I'll test. My explanation was simply that if the upper bits are dirty, every operation is effectively 512-bits wide since it needs to zero the upper bits, and the only mechanism for eliding that work is the "all zero state", so if that's not set you need the update the full 512-bit register and generally use the full 512-bit data paths, just like any zmm-using instruction.
@PeterCordes - I checked and vpaddd ymm... instructions can still execute at 3/cycle when the upper 256 is dirty, but you're stuck in the L1 license. So it seems different than the "p1 shutdown" state that Mysticial describes above (perhaps that state is more transient). This seems to kind of contradict the theory that it's based on limiting wide PRF access, and also my theory that it "extends" the operation to 512-bits since then it would only have access to 2 EUs. Maybe it just means the upper half of the PRF is powered up (and the PRF is not halved in size for AVX-512)?
 
@BeeOnRope I just realized that this provides a way to test the size of the 512-bit register file. I've always theorized that Intel's current AVX512 implementation doesn't actually double-up the size of the register file (that's a lot of area!). Instead it could be combining pairs of 256-bit registers. There's now a way to probe this a bit.
Write a loop with a very long dependency chain of 256-bit instructions such that each iteration is independent. The only way to achieve ILP is for the CPU to reorder across iterations. By adjusting the length of the dependency chain and measuring the performance, you can determine how large the reorder window is and perhaps the actual size of the register file.
The experiment is to see whether the size of that window changes depending on whether you have a dirty upper 256-bit state. Likewise repeat the experiment with 512-bit versions of the same instructions and see if the window shrinks. But watch out for the longer port5 FMA latency. So I suggest using the long latency shuffles.
 
@Mysticial - is there a place to read about the p5 extra latency? Does it apply to everything or only things that use the FMA unit? I'm doing most of my testing on a W-2104 that doesn't have the second FMA.
 
@BeeOnRope Only the FMA unit. And only if the data needs to cross back into the normal on-chip units.
The longest single uop non-memory vector instruction that doesn't touch the FMAs seems to be only 3 cycles for the shuffles.
Unless I'm overlooking something.
But if you're on a single FMA chip, it shouldn't matter. And you can hit it with 4 cycle FMAs.
 
10:04 PM
ISTM it would probably work even with latency 1 instructions
 
At minimum, it would have to be multi-ported.
 
but it would be clearer with longer latency since the difference between overlapping or not would be greater
there is also this:
basically the same approach but uses memory misses overlapping or not to determine RoB and PRF size
 
Ah, using NOPs to separate RoB and PRF. Smart.
 
10:41 PM
ya
 
I just tested it with a chain of vpermi2q ymm0, ymm0, ymm0 and zmm0 equivalent. No difference other than the clock speed throttle.
Both began showing signs of degradation from 32 -> 64.
But I'm also only using one register.
 
I guess you are hitting the RS limit
 
ROB most likely.
 
i.e., the scheduler fills before the PRF is exhausted
 
It's a 3 cycle latency instruction. So the CPU needs to combine 3 loop iterations to fully saturate the shuffle unit.
64 instructions / iteration (not counting loop counter + compare) puts it just over 192 which I believe is slightly larger than the ROB for Skylake.
 
10:51 PM
Skylake ROB is 220ish
but then why does it start degrading at 32?
32 * 3 << 220
 
32 was fine.
I tested 32 and 64.
64 showed maybe 10% degradation from 32.
 
ah ok
i see
not enough of a knee
 
128 was another 20% for a total of 60%-ish from 32 where no degradation was observed.
 
another approach you try is like the stuffedcow approach, but just use some long latency instructions instead of cache misses, like a chunk of 10 FP sqrt or whatever else is slow back-to-back
 
Testing the register file will be trickier. Since I think Skylake seems to recognize when a value is destroyed and thus can wipe it from the register file.
So I'll need to use an "expand/compress" idiom where I take one register, "expand" it out to a many register intermediate, then "compress" it back down before writing to memory.
 
10:54 PM
like if there are no more references to the PRF entry and the associated architectural reg has been overwritten it can reclaim it?
 
@BeeOnRope yeah
 
11:05 PM
BTW, I ran Henry's tests now that I look at the comments, lol
that was Skylake client. I didn't observe any early reclaim behavior though: there were 16 more physical regs vs Haswell based on my test, but that was probably because they changed the SSE <-> AVX transition stuff as I speculated there
 
11:18 PM
@Mysticial - I'm updating that tool and will run it on Skylake-W to see if it corroboraes any measurements you make
corroborates, ugh
 
I don't think I'm doing this test properly.
Maybe you really do need to be in speculation to saturate the PRF.
 
11:33 PM
I'm not able to saturate the PRF with either 256-bit or 512-bit code.
I have a 32 instruction sequence that expands out to 32 values - of which 16 are live at the widest point.
Compared to my linear single-register sequence, no performance difference. They both degrade at the same point - presumably from saturating the ROB.
The problem is that I'm unable to get an instruction/live-value ratio less than 2.0. Which means I'd need an ROB of 320-ish to saturate a presumed PRF of size 168.
Actually hold on, I can collapse more than 1 live-value at a time with the 3-operand permutes.
I can get it down to 1.5 instructions / live-value. So my loop is 24 instructions with 16 live values. It'll take about 10 iterations to saturate the ROB. But that's only 150 live registers.
No degradation on the 512-bit version either. Which seems to be evidence that Intel did double up the size of the register file.
Otherwise, it wouldn't be able to keep 150 512-bit values live at once.
Ah nevermind. That doesn't work. Only 3 iterations are needed to saturate the shuffle unit.
 
we looked at some die shots on RWT and it seemed plausible that the PRF was doubled
ie., there was a "PRF looking" thingy that was accompanied by an equally sized in Skylake-SP
but now that I'm thinking of it, it should be 4x since the reg count doubled too
 
11:49 PM
reg count doesn't matter
Going to a 48-instruction/144-cycle sequence with 32 live values still doesn't show a difference between 256-bit vs. 512-bit beyond the clock speed throttle.
The 96-instruction/288-cycle sequence maxes out the ROB.
goddammit.
But it does say that the 512-bit PRF is at least ~96 large.
 

  last day (16 days later) »