« first day (3047 days earlier)      last day (1916 days later) » 
00:00 - 18:0018:00 - 00:00

6:00 PM
Oh yeah, right.
Any that would have thrown get put into the errored array?
Maintaining key -> future?
 
yep, and timedout in timedout array, maintaining idx, yeah
 
Cool
@JoeWatkins If you get time, try that cluster example against an http benchmarking tool. It's quite impressive.
 
I will do that, but at the moment I broke ma php
 
@Trowski It's a pity that PHP is shared nothing, it means you need to keep state in every worker, so X times.
 
6:04 PM
Using wrk -c100 -t8 http://localhost:8080/ against the simple-http-server.php example, I'm seeing 36,000 rps.
@kelunik It is. Some of our options don't translate well to a threaded server, for example, number of allowed connections.
 
what you get with processes ?
 
@Trowski You should get the same as with processes, no?
 
For some reason I'm only seeing about 33,000 rps with processes after doing many runs. Threads are consistently a little faster.
 
Maybe context switches between threads are cheaper than between processes?
 
No idea, but sounds reasonable.
 
6:11 PM
93
Q: Thread context switch Vs. process context switch

LeonCould any one tell me what is exactly done in both situations? What is the main cost each of them?

 
Stopping the cluster is way more consistent with threads too. Processes sometimes like to hang and I haven't figured out why.
 
how many workers you using and how many cores you got ? (real cores, not ht) ?
 
@kelunik why is that even a pity? sure, it'll eat a few MB of ram, but not too much, and that's about it
 
okay, working with this, FFI is no where near complete enough for production
github.com/php/php-src/blob/PHP-7.4/ext/ffi/ffi.c#L2231 <-- what did I pass? what's expected? which arg?
 
@bwoebi Depends how much state you keep. If it's several GB, it matters.
 
6:14 PM
@ircmaxell what are the main missing points?
@ircmaxell doesn't stacktrace tell you what you passed?
 
error handling is completely non-sensical
@bwoebi I pass 4 args. It tells me one of them is bad
 
ugh, that's stupid
 
object(FFI\Exception)#1845 (7) {
  ["message":protected]=>
  string(28) "Passing incompatible pointer"
  ["string":"Error":private]=>
  string(0) ""
  ["code":protected]=>
  int(0)
  ["file":protected]=>
  string(58) "/home/ircmaxell/Workspace/PHP-Compiler/FFIMe/lib/FFIMe.php"
  ["line":protected]=>
  int(37)
  ["trace":"Error":private]=>
  array(3) {
    [0]=>
    array(6) {
      ["file"]=>
      string(58) "/home/ircmaxell/Workspace/PHP-Compiler/FFIMe/lib/FFIMe.php"
      ["line"]=>
      int(37)
 
the gap should widen the more workers you can run on real cores
 
$ffi->gcc_jit_context_new_param(
    $ctxt->getData(),
    $loc->getData(),
    $type->getData(),
    //$ffi->cast('char*', $ffi->stringToCData($name))
    $name
)
 
6:17 PM
dmitry is clearly practising for the days when he's going to get 6000000000 lines of code and a report with the description "my code is broken by jit and I can't debug it" ...
 
which one of them is the error? I have GDB open, and found out it's #3
but I can't see what's expected...
nor what it is, because it appears that info is lost after parsing
 
@JoeWatkins 4 cores, 8 threads.
 
@ircmaxell You should just fix that in php-src directly or tell Dmitry - looks like a severe oversight to me
 
@bwoebi this isn't something fixable from what I'm digging in
Well, it is "fixable", but wouldn't be a minor patch
 
yeah (at least telling what exactly is expected)
 
6:26 PM
Fatal error: Uncaught Error: Class 'Amp\Parallel\Context\Parallel' not found in /opt/src/cluster/src/Watcher.php:149
what am I doing wrong ?
nvm
 
all the info needed to "fix" it is thrown away inside the parser.
from what I can see at least
and I am not familiar with that parser structure
 
what parser is that even - why no bison? :-/
 
[2019-02-18 18:28:38] cluster-29130.error: Worker 3 failed: Amp\Parallel\Sync\PanicError: Uncaught Error in execution context with message "Class 'Amp\Http\Server\Server' not found" and code "0" in /opt/src/cluster/vendor/amphp/parallel/lib/Sync/ExitFailure.php:46
@Trowski ?
 
for all types, it changes the name to a specific pointer
and then it compares the pointers.
 
6:30 PM
@ircmaxell yeah, I mean, what program is even able to generate the C code for it?
 
@bwoebi maybe libffi?
 
@JoeWatkins http-server isn't installed by default. composer require --dev amphp/http-server
 
@ircmaxell well, the generated code is remembering me of lxr.room11.org/xref/php-src%407.4/ext/ffi/ffi_parser.c bison generated c (also the yy etc.) but the syntax in parser file confuses me
 
I have no more time to dive into this now, going out for dinner
 
krakjoe@fiji:/opt/src/php-src$ wrk -c100 -t16 localhost:8080
Running 10s test @ localhost:8080
  16 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   724.08us  271.89us  16.16ms   82.23%
    Req/Sec     8.34k     1.19k   11.73k    68.42%
  1340459 requests in 10.10s, 227.55MB read
Requests/sec: 132719.91
Transfer/sec:     22.53MB
 
6:34 PM
Good lord… lol
 
@ircmaxell LLgen apparently
 
that seems to be ceiling ... more connections and more threads doesn't make any difference ...
 
@Trowski this looks similar to what I achieved on an 4+4 ht cores i7 like 3 years ago
on the old aerys
 
@bwoebi Yeah, my laptop is an i5 with only 4 cores.
 
@bwoebi either way, the lack of debug info makes this pretty useless for anything of any non-trivial size
 
6:37 PM
fully agree
 
krakjoe@fiji:/opt/src/php-src$ wrk -c200 -t16 localhost:8080
Running 10s test @ localhost:8080
  16 threads and 200 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.44ms    2.29ms  76.66ms   97.85%
    Req/Sec     9.81k     0.87k   13.99k    75.98%
  1576669 requests in 10.10s, 267.65MB read
Requests/sec: 156112.53
Transfer/sec:     26.50MB
 
:-P
 
that's all 16 ... but I don't have another machine to run wrk on ...
 
I'm working on libgccjit, through the examples:
gcc_jit_type *void_type = gcc_jit_context_get_type (ctxt, GCC_JIT_TYPE_VOID);
gcc_jit_type *const_char_ptr_type = gcc_jit_context_get_type (ctxt, GCC_JIT_TYPE_CONST_CHAR_PTR);
gcc_jit_param *param_name =
  gcc_jit_context_new_param (ctxt, NULL, const_char_ptr_type, "name");
becomes in PHP:
  $ctxt = gcc_jit_context_acquire();
  $void_type = gcc_jit_context_get_type($ctxt, GCC_JIT_TYPE_VOID);
  $const_char_ptr_type = gcc_jit_context_get_type($ctxt, GCC_JIT_TYPE_CONST_CHAR_PTR);

  $param_name = gcc_jit_context_new_param($ctxt, null, $const_char_ptr_type, "name");
 
@JoeWatkins Any better with 400 connections?
 
6:38 PM
which are all fully typed
function gcc_jit_context_get_type(gcc_jit_context_ptr $ctxt, int $type): gcc_jit_type_ptr {
    return new gcc_jit_type_ptr(
        $ctxt->getData(),
        $type
    );
}
 
krakjoe@fiji:/opt/src/php-src$ wrk -c1000 -t8 localhost:8080
Running 10s test @ localhost:8080
  8 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     5.75ms    3.99ms  67.19ms   88.89%
    Req/Sec    21.34k     3.71k   36.65k    65.20%
  1697319 requests in 10.08s, 288.13MB read
Requests/sec: 168396.47
Transfer/sec:     28.59MB
 
and there I found my problem showing you what I found
GDI pHP ACCEPTING MORE ARGS THEN YOU NEED
 
I think for FFI this shouldn't be permitted
 
that code should have been:
function gcc_jit_context_get_type(gcc_jit_context_ptr $ctxt, int $type): gcc_jit_type_ptr {
    return new gcc_jit_type_ptr(
        getFFI()->gcc_jit_context_get_type(
            $ctxt->getData(),
            $type
        )
    );
}
 
oh
 
6:41 PM
@bwoebi for this?
 
I can't make more than that ... I think I could if I had another machine to run wrk on ... it's not really right to run on same machine ...
 
@ircmaxell passing more args than expected to a FFI function
 
@bwoebi no, I mean fora regular function (constructor)
 
ah hm
 
few cores are not fully utilized by amp because wrk is using them too ...
 
6:44 PM
yeah sure
 
parallel is remarkably stable, very happy about that ...
 
@JoeWatkins Sure, I bet you'd see 200k+ if you had another machine.
Though network I/O would probably slow things down.
Clearly Amp's http-server will not be the applications bottleneck.
 
I saw 173k at maximum and with a 30s test, when I do that, amp is only getting 60-70% utilization of each core once it settles ... wrk is using the rest ...
 
yeah, that's been my experience as well, about 25-30% overhead for wrk
 
Was parallel always going to be more capable than amp?
 
6:53 PM
they're working together ...
it's quite sexy
 
Ah interesting, I should read closer..
Will need to run some ideas by you someday re: parallel and thread-safe ds
 
you just did, don't do it
 
Roger
 
that was simple :)
parallel is not going to support objects all the while zend doesn't support that, it's just not worth it ...
if zend ever supports that, easy, but the only other way is absolutely horrible ....
 
Fair enough, good things take time or was never meant to be.
eg. generics
 
7:06 PM
@JoeWatkins Better, I think: github.com/amphp/parallel/commit/…
 
yes, much
 
Does select() modify the $resolving array?
 
yes, removes errored and resolved
 
Which would be fine actually, so I probably don't need to dup it.
 
generics is going to be so hard, it seems quite simple on it's face, but when you get down to it, it is not simple at all ...
no you don't
I think three times now I've modified the parser to work, and then started on the implementation, but I can't model the whole thing in my head well enough to finish ... it's going to take a lot of us, or nikita, to do it ...
 
7:14 PM
Heh we should start a new language..
 
I've been down that road too, just before we started work on phpdbg, one of the internals guys name felipe made it, was called clever-lang, I can't remember anything about it except it was C++, and I really hate that ...
I spend longer reading docs than coding when using C++ ...
and people are so opinionated about it, half your time reading docs, the other half arguing about the correct way to do things and how to cast and junk, it's so unimportant ...
 
Using C++ is tricky
You need to make sure that 99% of your code only uses 5% of the language
If you stick to "simple C++" for most of the code it's possibly sometimes somewhat okayish
 
yeah, it's very much more complicated than C, I feel like there should be more than two plus ... at least 5
> possibly sometimes somewhat okayish
I lol'd
 
7:42 PM
I think the leave/enter thing is a dead end ... it doesn't achieve anything we can't do today, it doesn't allow us to remove executor hook ...
I just can't think of a nice way to remove it and still retain the ability it gives you ... I don't think there is a way that we could actually use, we're stuck with recursive executor and loops in extensions, I'm pretty sure ...
but maybe, someone will see something, provoking people to think is good at least, but I'm not confident ...
/cc @beberlei
pcov is stuck using the interpreter, but that's not the end of the world, it's fast enough and nobody needs their test suite to be jit'd, not just for a coverage run, not really ... and in the case of other tracing tools that don't need instr resolution data, we need to come up with a way that stays far away from the executor, and touching opcodes directly ... I don't think there's another way ...
I may start to work on that instrumentation thing and try to make it as generally usable for everyone as possible, and maybe we can have that in core at some point so everyone writing exts can rely on it's api, rather than reinvent the wheel ... I can't think of another way that can actually work ...
 
@JoeWatkins but isnt adding it about tomorrow when JIT hits? you can't call zend_execute_ex in JIT context, but you can call enter/leave
 
7:58 PM
if your executor doesn't call opcode handlers directly, and forwards to the ze impl, then it will enter jit'd code ... the normal executor is stackless - it's not called recursively, when you set an executor function it removes that advantage, that's why we don't want that hook when the jit comes, but just setting it should not stop jit'd code being entered
hmm
that's wrong
let me read a minute
no that's right
sorry I haven't read the hybrid thing before ...
but it just invokes the handler directly, which is jit'd code, it will be entered
347  #if (ZEND_VM_KIND == ZEND_VM_KIND_HYBRID)
348  #define HYBRID_NEXT()     goto *(void**)(OPLINE->handler)
349  #define HYBRID_SWITCH()   HYBRID_NEXT();
350  #define HYBRID_CASE(op)   op ## _LABEL
351  #define HYBRID_BREAK()    HYBRID_NEXT()
352  #define HYBRID_DEFAULT    ZEND_NULL_LABEL
HYBRID_SWITCH is the entry point for the hybrid vm
@NikiC what's all this VM_TRACE stuff ?
if we could make it so an extension can define those, we can remove the executor hook ?
 
8:14 PM
how should I proceed if I have my full website developed in PHP but I want to integrate one chat application in that as well?But I prefer django channels, So is it possible?If yes how should I proceed?
 
8:26 PM
@JoeWatkins why do you hate c++ (actually, as long as people don't hack around with the language to squash things into it which are not really meant to work that way)?
 
I'm going to say C++ culture ...
We mostly code based on the Google C++ Style Guide, which seems sane enough. The LLVM one is also quite nice.

Use the .cc extension for C++ source files and .h for C++ headers, hopefully your compiler is smart enough to identify C++ source inside a .h (don't worry, GCC and Clang are!). Oh, I almost forgot, please use tabs instead of spaces, UTF-8 file encoding and UNIX file endings (Line feed a.k.a. n).
 
@JoeWatkins so no problems with the language itself?
 
this is about one third of the readme for that clever thing ... there is an obsession there with doing things a particular way, any many ways are correct .... so you spend more time discussing correctness than getting stuff actually done ...
not really bits of it are really nice, I've used it before for non oss stuff, but collaboration is so hard ... although it is a whole bunch more complex than C, and I'm not sure that the 5% of the features you can/should use (according to nikita) are worth all that complexity ... and all that complexity is what created the culture ...
it will never be my goto, that will be C, until C is outlawed ...
 
But I agree with Nikita, only use the features you feel they're really needed
 
@bwoebi My main problem with C++ is that they are trying to fix it, and they're failing horribly
Peak C++ was around C++11, now they're just trying to make it worse.
 
8:44 PM
It's not like you have to use the new features... but yea, people will use them
Also most annoying thing in c++: const
so many compile errors due to some const specifier missing or too much
 
can you think of any way to rewrite the executor loop of phpdbg ?
 
const methods on objects feels seriously like a bit wtfy ... and some std:: structures do not really like const values inside them...
 
so that we don't need to override the executor at all
also, dmitry is suggesting pcov should use zend_ser_user_opcode_handler rather than the loop, thoughts on that ?
(I've done it, it seems to make no measurable difference)
 
@JoeWatkins you already did? can I see the code please? :-D
 
oh you meant phpdbg ... no, I've got no ideas ...
 
8:50 PM
no I meant pcov :-D
exactly what you gave me
@JoeWatkins well, using user opcodes probably would work as well.
 
well I don't want to change it unless there is actually some advantage, I done a few tests and there was no difference ... but I guess it's nicer not to set executor ... but I don't know if that's really better ... and being able to do this doesn't mean we can remove the executor hook from php-src, so it doesn't seem like a win
@NikiC you got any thoughts on this ?
 
I always thought user opcodes were slow
But they aren't?
 
not slower than actually calling zend_vm_call_opcode_handler I guess
 
I can't seem to measure a difference on the suites I was running when I developed it
 
they are slow if you compare to the native executor
 
9:01 PM
the call handler thing is complex, but this means more frames because dispatch ...
it looks about even, but I'll try some bigger suites tomorrow ... I was hoping for some sort of win, like maybe the jit is more compatible with or something, or some obvious perf gain ... but it doesn't seem to be there ... and on really big suites it probably will degrade ...
krakjoe@fiji:/opt/src/PHP-Parser$ diff -uNr develop.log uop.log
--- develop.log	2019-02-18 22:05:13.498438655 +0100
+++ uop.log	2019-02-18 22:04:27.930274089 +0100
@@ -22,13 +22,13 @@
 ............................................................. 1220 / 1261 ( 96%)
 .........................................                     1261 / 1261 (100%)

-Time: 3.7 seconds, Memory: 266.00MB
+Time: 3.81 seconds, Memory: 264.00MB

 OK (1261 tests, 2199 assertions)


 Code Coverage Report:
-  2019-02-18 21:05:09
so it's a bit slower on that one, that's the biggest suite I got locally
I think it's not really worth it, this isn't really a huge suite and it's consistently slower ... on normal suites for normal size packages that normally run in half a second or something you can't see a difference, but I think if I test on something really big, it's going to be exaggerated ... and none of this allows us to remove the executor hook
 
9:20 PM
Have to read YAML in Jenkins without a library. Shakes Fist at Sky
 
@JoeWatkins I'm going to shamelessly borrow from your parallel ci/coverage setup, is so nice. Anything I should look out for?
 
nothing that's not obvious I don't think ... you'll need Makefile.frag, changes to config.m4, and mostly just copy .travis.yml ... I forget what project it started on, but I copy/paste that into new stuff now, it's quite easy to setup ... make sure you activated coveralls before first build
 
Thanks I'll take a look. Finishing touches on ext-decimal 2.0 while ext-ds 2.0 is in early stages. It's fun to have some things to work on - very excited about ext-ds ahead.
I'll try apply to ext-decimal, hopefully mpdecimal influence won't be an issue.
 
9:49 PM
is that installable with apt ?
> We don't normally bundle things with no user base at all, that are totally unproven (except phpdbg :grin:).
hehehehe
 
@JoeWatkins vimeo/psalm is huge… like 15000 tests I think, if you wanted to take a crack at that.
 
yeah please do
 
With xdebug the runtime is like an hour… I don't know if I even want to, lol
Let's at least see what it is with pcov.
Dunno where I got 15000 from, it's actually only 2944 tests.
It's still a slow test suite.
 
10:06 PM
sometimes number fall from the air, other times, we pull them from the air :D
 
Or other places.
With pcov: Time: 9.24 minutes, Memory: 2450.00MB
 
IIRC symfony/symfony was around 22k
 
psalm is interesting because the number of instructions per test is pretty crazy.
 
@Allenph why without a library?
 
can you patch it with gist.github.com/krakjoe/3bd2ee8fc19c104ca0799d503d0dbd23 and see if there's a difference ?
 
10:17 PM
I can, I'm running xdebug right now… so it'll be a while.
 
that will take one year ...
 
16% done… wow…
 
@JoeWatkins yeah I think libmpdec-dev
 
10:35 PM
oh yeah, it's right there in .travis.yml :D
@Trowski are you loosing the will to live ?
also, did it install phpunit 8 ? composer.json says 6 or 7 ?
 
I'm good, though my computer seems sad, the fans are going full tilt.
I had to use your clobber lib.
 
oh right
 
It does inspire me to never use xdebug for coverage on a library of any size – which means updating to PHPUnit 8 soon.
 
or clobbering ... which has the advantage that you get to use the word clobber, and talk about clobbering ...
no but you should upgrade ... sebastian done a good article on it with actual solutions, which may work for some people ... somehow my colleagues have found reasons not to actually do it, but I'm pestering them daily ...
I've even heard "I don't like automated tools" ... in the context of phpunit ...
 
Unfortunately for Amp we can't upgrade to 8 (or even 7) because we're supporting 7 for right now.
 
10:44 PM
7.0 ?
 
Yeah, 7.0
 
that's a very hugely bad terrible idea
whose idea was that ? they should be slapped ... and encouraged ... with slapping ...
 
Didn't Ubuntu 16 LTS ship with 7?
 
that doesn't matter
@_francislavoie @Ocramius @jrf_nl @asgrim @phpunit It's strictly true, the debian package maintainers are package maintainers, not internals developers, to my knowledge, none of them are involved in internals at all, they do not have the expertise, nor access to security reports (not public) in order to provide actual security.
 
I'm fine dropping 7 support, though I'm not going to make that decision myself. /cc @kelunik @bwoebi
Any new libs have been moved to 7.1 already.
 
10:48 PM
@FélixGagnon-Grenier ping
 
time is running out there too, I hear the rm forgot to do a release last month ... he's really crap ...
this month, it was this month ... he doesn't know what day it is a lot of the time ...
 
it's all the same fucking day, man
 
@JoeWatkins I have that problem
 
In that case maybe we should move to 7.2 :-P
 
sometimes Monday, Tuesday and Wednesday feel like one big blob of a day, separated by periods of sleep
 
10:53 PM
just to drive home the point, I'm an rm and can't always know what is going on with security patches ... I should, but php is a shit show behind the scenes, things don't always work ...
I wasn't sure if it was necessary to do a release, so I didn't do one, and nobody complained, then after it should have happened I was told a patch was waiting ... and now there's two, but one is not security ... but I can't do a release until the schedule says I can ...
shit show
 
Just a little more organization is needed I guess. I assumed releases were always done, even if nothing significant changed.
 
no 7.1 is in security only now, so there's only releases if there are security fixes ...
and it doesn't make sense to be depending on it anyway, security has a very specific definition, and it doesn't include "may bring your servers crashing down" ...
 
meh.. entropy was gonna do that eventually anyway
 
A cosmic ray changed the jump address, and caused some random code to be executed. When will this be patched?
Right, forgot 7.1 was already in security-only.
64% done…
 
so unreasonable
 
11:09 PM
@Tiffany pong
 
11:24 PM
opens a PR for travis changes, makes it a draft PR because it's actually not finished, travis doesn't trigger on drafts :|
or maybe that wasn't it...
 
@JoeWatkins With xdebug: Time: 1.24 hours, Memory: 1364.00MB
 
so ... one year ...
 
I was right about it being over an hour… what a joke.
I've always disabled coverage when running that for obvious reasons.
 
it's busy getting it "right" ...
also, it gets it wrong ...
 
Code Coverage Report:
  2019-02-18 22:14:55

 Summary:
  Classes: 23.50% (47/200)
  Methods: 58.29% (724/1242)
  Lines:   87.00% (24491/28152)
That's xdebug ^
Code Coverage Report:
  2019-02-18 22:02:12

 Summary:
  Classes: 23.50% (47/200)
  Methods: 58.37% (725/1242)
  Lines:   86.43% (24089/27871)
pcov ^
 
11:36 PM
it really does make mistakes, sebastian got a 30mb diff in size of xml on some project because x was just missing a bunch of data ...
see what the patch does ?
 
Running right now.
I should have saved the clover files… oh well.
Actually it's not too late, I still have the xdebug (which is what matters)
The html coverage folder is 1.8 GB O_O
 
@Trowski the diff would probably be interesting ... but don't waste another hour of your life on it ... the heart beats a limited number of times ...
 
I renamed the xdebug one quick, so I do have it if you want.
@JoeWatkins With patch: Time: 11.64 minutes, Memory: 1642.00MB
 
okay chuck it in the bin, burn the bin ... thanks for all that time ...
 
I was doing other things of course, but being that's not even close apparently it didn't help.
 
11:47 PM
it's all dmitry's fault ...
 
Memory was significantly lower.
 
probably/must be missing data ...
I dunno how, but let's not waste any more time on it, definitely can't accept that decrease in perf ... it would have to start laying golden eggs to consider it ...
 
It says only 43% of lines are covered now… wtf…
 
not super surprised, I wrote it the way I wrote it because it's the only way I know of to be certain that every instruction is potentially recorded ... the patch is how xdebug does it because dmitry thinks it's better ...
a better method ... but only because he wants to remove some engine hook to make his life easier ... not happening ...
I gotta sleep, thanks for all the effort ...
nn all
 
Night o/
 
11:57 PM
\o
 
00:00 - 18:0018:00 - 00:00

« first day (3047 days earlier)      last day (1916 days later) »