« first day (3052 days earlier)      last day (1896 days later) » 

12:22 AM
we’re all fucked
 
1:43 AM
@ircmaxell how much of that is off the top of your head vs. following a reference?
 
2:07 AM
interesting one to test: github.com/php/php-src/pull/3874
 
2:28 AM
^^ arrays are weird
That was a clever attempt though, quite impressed.
 
I can't break this PR, no matter what.
 
 
1 hour later…
4:07 AM
@GabrielCaruso what do you mean you can't break?
 
 
4 hours later…
7:58 AM
@rtheunissen honestly, about 95% is from my head. Some references here and there, and reference manuals...
 
That's crazy, I salute you.
 
chockablock brought close together
 
8:10 AM
@beberlei because credit where due, he went from saying he's not going to work on it anymore, to putting some effort in to solving the problems that have been pointed out ... doesn't change the fact 7.4 is still too early, but he needs to be encouraged to work on the thing ...
 
8:23 AM
@beberlei I sorta think begin and end are not useful, they cause quite some bloat in code, and in the vm, add a considerable overhead even when the extension handlers aren't doing anything, you can measure about a 10% slowdown even in simple code when the handlers are null ... the most efficient way to do it would be with enter/leave, and the easiest, but I can't convince dmitry it's the right thing to do, I can't convince him of anything at all ...
 
@JoeWatkins I've just realized that ext/inspector is not compatible with master
 
oh, what are you using that for ? it must be scary ?
 
@JoeWatkins I'm not currently, I just realized I had it in my config.nice still from last time I was debugging something in it for you :-D
and currently building master again in that directory
 
@JoeWatkins the extension handlers iterate over all extensions and call the hook, even if the extensions doesn't have them they get iterated over again and again. is that the 10%? there is an optimization possibility where fcall begin/end call a global callback, that is chained by extensions.
or is it even just evaluation of the additional opcodeS?
 
8:36 AM
it's both yeah
 
@beberlei that's a good point
 
i have looked at the current execute_ex, man its a monster, do i remember the php 5 ones wrong as just a few lines? :-) no wonder dmitry doesn't want you to extend it for pcov
 
does it iterate over modules as well or only zend exts?
 
oh, i havent checked that, i assumed modules
i just saw the apply code something
just zend_extensions, you are right
i have two lenses on this, because i need two kinds of instrumentations:

1. xhprof like - where I don't care about specific functions and just want to count the total time of evrey function (the open source part of tideways)

2. specific instrumentation, for example turning Symfonys HttpKernel::logException() into an error, instrumenting nrk/predis Composer (userland) library for Redis calls or Elastica. Detecting the creation of a controller to "name" the transaction after it.
For the first one, I think i wouldn't mind if the JIT removed functions in production traces, you can explain that to users in the UI. but for the second approach, I find myself looking for a 100% guaranteed way to get enter/leave for a userland function call.

I think thats the major difference between profiling extensions (and many approach are open source) and APM extensions (where I only now the datadog extension to be open source atm).
 
8:51 AM
@beberlei it's just the hybrid VM mode which inlines the hottest functions within execute_ex
 
@bwoebi yes, saw hybrid vm - that a special mode that works with GCC only yes?
requiring some global register feature or something as far as i understood the code
 
just zend_extensions
they are the only things that can install handlers ...
 
@beberlei global registers aren't required for it
 
@bwoebi ah ok, the macro checks for it to enable thats why i assumed :-) defined(HAVE_GCC_GLOBAL_REGS) i wouldn't even know what that means, the VM code is several levels above my comfort level of Zend-flavored C
 
@NikiC 2 is what would be best achieved with hotswap, that's what's making hotswap relevant to apm ...
you can achieve it other ways, they already do ... but the nicest thing to do is hotswap for whatever kind of instrument you need, and have that instrument call the original function ...
some of them must be updating function tables, I guess (they're closed, so guessing) ... which becomes impractical/broken/stupid with preloading ...
so you want to swap at the vm level ... just in time for the call ... not by editing function tables and possibly causing strange side effects ...
some extensions have their instrumentation written in C, others have or may want to have it written in PHP, that's where the need to be able to swap user for internal and vice versa comes from ... I don't really think it's a crazy thing to support, at all ... and it's not really a hack, it's a pretty reasonable thing to want to support ... at least, if you're going to support instrumentation at all ...
but it looks like we're not, so will have to find other ways ... which will inevitably be much nastier than having hotswapping support, note that it's only missing from crapcache ... the vm supports it fine ...
these tools target production, there's no getting away from that, you can't just say disable opcache ... people have opcache enabled in production, and need their tools in production too ...
 
9:09 AM
for internal functions at least the tideways extnesion already overwrites the handler function. we have a code generator for it to generate a version of what phar/func_interceptors.c does
@JoeWatkins @NikiC what about a specialized versions of the ZEND_DO_FCALL_BY_NAME, ZEND_DO_ICALL and ZEND_DO_FCALL opcodes along the lines of ZEND_DO_ICALL_INSTRUMENTED, which then call enter leave callbacks, and two ways to enable it, first with a global compiler flag, second by extensions that use the AST hook to modify them only for what they need
 
there's no such thing as a global compiler flag, opcache is free to ignore them (it already does), and I can't convince dmitry that is wrong
 
uh and i assumed CG(compiler_options) is handled correctly by opcache
 
nope, and he doesn't care to respect it either ...
he equates the need to instrument zend to hacking, he doesn't see any difference, he doesn't want to change opcache to respect compiler flags because he says it targets production, and tooling doesn't ... that's his position, and I can't change it ...
that's why I said I gave up on the thing ... I can't make him see reason here, he has a very narrow scope of correctness, he doesn't see that opcache should be backward compatible with what the vm supports, and if the vm supports something "by accident" he doesn't see that we should put effort into keeping that support, through opcache, through jit ... I don't know what to say to him any more ... it's all very stressful for me and I'd rather not interact with him further ...
 
@JoeWatkins sorry :-/ been pushing you through this the last weeks. i get the frustration
 
Polish language missing? – #77658
 
9:24 AM
@beberlei it's not your fault, it's not even all dmitrys fault ... I'm not good at words, or humans, I know that, I'm too emotional a lot of the time, I can't help it ...
 
@JoeWatkins <3
 
I'm going to do going outside and leave the computer alone for the day ... catch you all later on :)
 
@JoeWatkins have a nice weekend :)
 
 
2 hours later…
11:08 AM
hi,
I would like to keep one or more checkboxes checked after a submit.
I already places them in an array an they showed up.

But if I save it like <?php if (!(empty($_POST['Color'])) && (in_array($_POST['Color'], $Colors))) "checked='checked'"; ?>

No checkboxes are checked, but they are in the Colors array.
echo "checked='checked'"; ofcourse :P
 
12:00 PM
i have written down the current state including requirements, the challenges, alternatives and potential new approaches to instrumenting code: public.3.basecamp.com/p/XDL2B2NEu3tbHMahvTaMypNP - i would love to turn this into an RFC but probably need a core sponsor to agree with the way forward :-)
the document probably misses some angles as well to be a complete overview
 
12:12 PM
http://i.stack.imgur.com/P6bNr.jpg

I came across this image this morning. I keep asking myself—if an exception is thrown because a lock cannot be acquired, then this means that a lock definitely does not have to be released. So you can remove `finally` and `release()` at the end of the `try` block.

I assume `optional()` creates and returns a wrapper object with `__get()` or `__call()`, which intercepts method calls and returns the value or `null`.

Seems quite contrived. Or am I missing something?
 
@DarraghEnright yes, the example is rather weird
you put the try { } finally { release } around the code using the lock, not the code creating it
 
Good morning, Room.
 
@NikiC Thanks for clarifying @NikiC — Exactly! I saw it on Twitter, where it was getting a lot of praise, but I was looking at it thinking... wait, what?
 
@JoeWatkins Good for you- good to get out. Is it warm over there yet?
 
12:28 PM
@DarraghEnright The better way to handle this is via either a locked(function() {}) closure or a RAII destructor release anyway
You don't want to leave room for missing unlocks
 
1:18 PM
hahaha
 
2:07 PM
Morning
@Donovan use $_SESSION
...or just get machform...
 
Wes
2:50 PM
@DaveRandom @Danack watched the latest orville yet?
 
3:40 PM
@Wes haven't seen any yet....
 
Wes
4:37 PM
@Danack you totally want to watch it... it's your kind of stuff :B in fact i assumed you were watching it already :D
 
4:55 PM
@rtheunissen Like, I couldn't test that patch to return false. Bob did github.com/php/php-src/pull/3874#issuecomment-466630369
 
5:12 PM
@beberlei Dmitry is planning to integrate Optimizer into core for PHP 8. So I would mandate him to have the core respect its own APIs (namely CG(compiler_options) here).
\cc @JoeWatkins ^
 
@bwoebi good to hear :-)
what do you think about a patch of this kind to allow extensions to control nojit? gist.github.com/beberlei/0ea5ea0ea484b103ca6b673363c66db4
 
Balls. I changed the mobile chat site to full so I could search for a room, and now I can't change it back to mobile.
Oh there it is, bottom right
 
@bwoebi I tried to make an exported compiler api function (zend_get_call_op) respect it's own internal api, namely compiler_options, and he says it's wrong ... what seems reasonable and logical to you and me is the opposite to dmitry, apparently ...
@beberlei imo it's not generally applicable to instrumentation, if a profiler uses that, it changes the performance characteristics of the code it is profiling, thereby making the profile it generates pretty useless ...
 
@JoeWatkins that is always the problem with function level profiling though, especially in tight loops the overhead of measuring piles up, say a 5000 times strlen
 
@JoeWatkins I call bullshit on that (to Dmitry)
@beberlei sampling profilers <3
I've been looking at perf+(e)bpf
Just quickly though
but looks like it should be stable enough
On a high level we could store php stacktraces and C stacktraces at the same time
 
5:28 PM
@bwoebi nikita agreed with him ... opcache is relying on the broken API, so it's wrong to fix the API and obviously opcache, and right to revert the patch ...
 
@bwoebi i experimented with that :-) github.com/beberlei/interrupt-sampler-poc
 
@beberlei You know about github.com/nikic/sample_prof?
@JoeWatkins well, I think opcache and the API shall be both fixed, in PHP 7.4+ only
 
@bwoebi yes, but it fetches the backtraces in the pthread and regularly crashes
 
@beberlei umm, really?
 
@bwoebi I did only patch 7.4+, he still wants to revert the fix so he doesn't have to fix opcache ...
 
5:32 PM
@JoeWatkins Well, then push against
 
@bwoebi yeah, because no locks are done, the execute_data could be changed while snapshotting it afaik, talked to niki about it and he confirmed thats possibliilty
that was some time ago though
 
that code isn't safe, at all
 
my attempt just changes the EG(vm_interrupt) variable in the thread, i have no clue about threads, thats probably also not safe :-)
i think when niki build his the zend_interrupt_function didn't exist yet
 
@bwoebi when nikita and dmitry both agree that something is incorrect, nothing I say makes any difference ...
@beberlei eg is thread local, it's a safe-ish approach ... the coolest sampling profiler I know of is phpspy, it's complex, but very very cool, and not an extension, not even part of php ... it would be a lot of work to make it work everywhere and I'm not sure it's even possible on windows, but it's definitely the best approach I've seen to a sampling profiler ...
 
5:43 PM
yeah its really cool.
 
o/ persistent b-trees, saturday morning reading..
 
@JoeWatkins one negative maybe about phpspy its requirement to run as root
 
@beberlei It needs the ptrace cap which you can offer to non-root users technically
 
ha, and the author appears as if by magic ... \o
 
Haha yes coincidental timing. I read scrollback in here every couple days. Still wish there were an irc bridge for this channel.
 
5:59 PM
yeah, phpspy is really the proper way to do it...
 
Never thought I'd see phpspy and proper in the same sentence
 
@adsr in that case, I apologise for all my frequent and long rants ...
 
@adsr ohi, nice to meet you :)
 
have you tested phpspy with the jit branch yet ?
 
Hello!
No I haven't, haven't looked at the jit branch at all yet
We are still on 7.1 at work
 
6:06 PM
/me sticks fingers in ears and sings really loudly
I'll pretend I didn't hear that ...
not sure what the side effects are going to be, if there are any ...
the approach is definitely the most superior, not having the restrictions imposed by executing inside of php, being a separate process, interference at the level below zend, all of that makes total sense ... I'll bet it wasn't easy to figure out ... I like it a lot ...
 
@JoeWatkins you should read julias blog, she has a large section of posts on how she worked on rbspy jvns.ca - not to steal @adsr thunder here but mentions its modelled after rbspy :-)
 
Absolutely Julia deserves credit for the idea, I merely ported it
 
I reckon if you put a (compatible with most distros) license in the repo and made noise at the right people, it might get a bit more attention ... it would be cool to be able to install it without having to decide how to build it, most php programmers don't have a clue about that ...
I did start to work on a cmake build for it, but it's gone now, it was pre-last-disk-disaster ... but it wasn't that hard ... that would also help ...
this is a bit out there ... but maybe, even, develop your own extension api ... it would be super cool to be able to use the data from C (or whatever) ...
(I have no good ideas how that would actually work)
 
@JoeWatkins did you see my summary on the zend_execute thing? i tried to gather as much detail about the current different ways of using it for tracing, tried to generalize it, offer potential existing alternatives and their downsides and potential new ones that I see 3.basecamp.com/4049078/buckets/8221808/documents/1618222514
i imagined having a central document about what it is used for now would help dmitry see my POV and hopefully a good oslution :-D
 
why is it asking me to login ?
 
6:21 PM
uuh sorry, not the public link :D
 
6:33 PM
Hey guys. Since from 7.2 this : if (count($this->implements) > 0) will yeld a warning, do I really have to do this: if (count($this->implements) > 0 && !is_null($this->implements)) or is there a better option?
 
I TBH think the way phpspy went with ptrace(2) is the way to go for profiling and debugging. The only way to make it really robust. Sure the jit will need to expose some information, but in general I feel like debugging PHP shall be external to the php process itself.
 
yeah it's definitely a source of inspiration ... I just don't really have the energy to write another debugger ...
 
@bwoebi i have to disagree though, it is for expert users and when you really dig into something, what tideways (and other apm extensions) provide is a 24/7 coverage. not in the detail of phpspy, but of the full application including when you are not looking
this space is very large, with many different approaches, i wouldn't go so far as to declare on approach the only way to go forward
 
hm
 
@Varin initialise it to an empty array if it's meant to be an array...or do an early return
 
6:44 PM
Guys, what does this "It's pretty spammy and probably not worth messing with in the last 4-5 years anyways." sentence mean? Which one?
- this is not worthy to do in these 4-5 recent years. (already, long time ago it was worthy)
- this is not worthy to do that for 4-5 years. (doing that for 1 year is ok)
 
@Shafizadeh the first one
 
thx
 
@Danack. Thanks, that is bit silly of me not thinking to do this in the first place, since it will always be an array :)
 
7:02 PM
I just patched the jit to work with that overload method I was working on ... so that it works in jit mode and non-jit mode ...
but it's pretty pointless to suggest anything to dmitry at this point ...
I don't get why he's being so difficult, or making out that what I want is impossible anyway, I just saw it work and it took me five minutes to make it work, it would have taken him three minutes ...
 
7:15 PM
I know this is incomplete/not the best/doesn't produce the best jit'd code, but I don't care about that, we need instruments to work, and I found a way that's nice, I don't see what the problem is ...
something about sscp and constant propogation was mentioned, I don't want to change what I don't fully understand ... but he could do it in five fucking minutes, and he definitely should ...
 
Wes
7:27 PM
is it normal that glob(__DIR__ . "/*") does not include a .dotfile ?
hi everybody
 
yes
php is missing some flags that are not part of posix, GLOB_PERIOD
 
Wes
thanks, so i am not going crazy
jordan peele's doing a new twilight zone
 
@beberlei if phpspy itself had an extension api and could be run as a daemon ...
in general, it's how profilers work ... being a part of the process you are profiling doesn't make good sense, it means your own software is included in the profile, it's like xdebug having a profiler, it's really just profiling the impact that xdebug has on an application, but doesn't tell you a great deal about how the application behaves when xdebug is not loaded ...
I'm going to read through that thing you wrote tomorrow, and make suggestions about it then ... but this all relies on dmitry being flexible, and he's proven he's not, he's proven he's not willing to change how opcache works to accommodate instrumentation ... so it might be a dead end ...
 
7:53 PM
@JoeWatkins thanks, again if you don't want to continue sinking time because you don't think its worth it feel free to tell me. i am willing to invest a lot of time to move this forward, because ultimately its my business that is on the line here :-)
 
8:20 PM
@beberlei I'm quite happy to work on code ... but there's no point in working on code that is broken by the jit/opcache, if I'm not allowed to change opcache ... it's not like I want to change opcache to be compatible with the code, I want to change it to be compatible with zend itself, and I'm not allowed, that feels like a brick wall I'm pushing up against ... and you're heading toward the same wall with me as far as I can tell ...
you haven't got anything to say to dmitry that I haven't already said, he doesn't care ... php isn't really a democracy, especially when it comes to opcache, I could propose a patch tomorrow and it be okayed by someone reasonable (nikita), and then dmitry insist it's reverted ... this happened last week ... I don't know how to move forward in that case ...
 
8:37 PM
if we force optimizer to disable the passes that don't behave, all we need is something like
diff --git a/ext/opcache/jit/zend_jit_x86.dasc b/ext/opcache/jit/zend_jit_x86.dasc
index b72bb4ff2f..9f96b34a2f 100644
--- a/ext/opcache/jit/zend_jit_x86.dasc
+++ b/ext/opcache/jit/zend_jit_x86.dasc
@@ -7098,6 +7098,10 @@ static int zend_jit_do_fcall(dasm_State **Dst, const zend_op *opline, zend_op_ar
                }
        }
        if (!func) {
+               if (CG(compiler_options) & (ZEND_COMPILE_IGNORE_INTERNAL_FUNCTIONS|ZEND_COMPILE_IGNORE_USER_FUNCTIONS)) {
+                       /* assume extensions require the fallback */
or some other way to tell the jit parts how to behave like we have for optimizer ... but I feel strongly that the inconsistencies introduced by optimizer itself should be fixed, and jit should not go down the road of introducing the same inconsistencies ...
 
8:52 PM
oO I just managed to configure the jit to be compatible actually ...
NOISE @beberlei
 
@JoeWatkins WAH! :p
 
(gdb) r
Starting program: /opt/bin/php test.php
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
y
Breakpoint 2, execute_ex (ex=0x0) at /opt/src/php-src/Zend/zend_vm_execute.h:55232
55232	{
(gdb)
(gdb) c
Continuing.

Breakpoint 2, execute_ex (ex=0x7ffff2e16020) at /opt/src/php-src/Zend/zend_vm_execute.h:55232
55232	{
(gdb) c
Continuing.

Breakpoint 1, zend_overload_begin (execute_data=0x7ffff2e16020) at /opt/src/zend-overload/php_overload.c:398
there's a break on execute_ex, it's never entered after first entry, both the target and overload function are jit'd (the handlers are jit code addresses) ...
it ... works ...
no patches ...
I'll put the thing on github tomorrow ... it is not bug free yet, but it's worth chasing now for sure ... I still think opcache should be fixed, but whatever, we don't have to wait for that to happen ...
 
i have some genious as well, need a few minutes to show :)
 
go ...
 
hm how do i specify void* as return value of a function?
ah, just do void* :p
 
9:05 PM
yeah
 
meh --disable-all still leads to php-src/ext/standard/image.c:246: undefined reference to `uncompress'
 
yeah standard can't be disabled
some others can't too ...
 
@JoeWatkins its not working yet, i haven't fully understood all the required changes to zend_vm_gen.php but you get the gist looing at zend_vm_def.h and the zend_vm_gen.php changes: gist.github.com/beberlei/09aac5d965b721b96f1ac9f2a5e6622e
the diff sucks with the generated code in there :-(
updated with just my changes
i need to teach vim to noexpandtab in php-src
 
9:21 PM
see here
aren't you missing an end ?
and you'd want to begin later, I don't think you want to trace undefined function errors
VM_ENTER doesn't return ...
 
aaah ok i was thinking that could be a problem
back to the drawing board :o
 
What are you guys trying to do?
 
yeah, it's like enter/leave callbacks, but that's the problem, keeping the vm stackless (so not using execute_ex), and finding where to end the trace ... the patch I proposed also lost stackless vm by forcing execute_ex to be entered ...
@rtheunissen looking at ways to instrument the jit ...
 
@rtheunissen integrating some kind of function enter+leave hook system that dmitry approves of and would work with and without jit
@JoeWatkins stupid question, what is stackless vm?
 
So that something like pcov works? gathering context
 
9:27 PM
@rtheunissen no that already works, but isn't compatible with jit (it can't be) ... it's more for profilers/apm/uopz kind of things ...
 
Sorry I meant pcov+jit
Ah k, gotcha
 
@JoeWatkins ah it makes sense now, DO_FCALL is at the call site, and jumps to the op_array of the called function, VM_ENTER only instructs it to process at that new point
i looked at some vld output to understand
 
@beberlei when you call a function (c level) recursively, it must create a new call frame for each call, very deep recursion - which is the nature of execute_ex - then may lead to a stackoverflow ... stackless in this context means that VM_ENTER doesn't create a new call frame, instead it jumps around in the switch (vm loop) to the correct address, having loaded call frame (zend level), and opline and whatever ...
 
oooh, so overwriting zend_execute_ex completly prevents that
 
exactly
 
9:35 PM
that makes a lot of sense
 
@JoeWatkins hm so TRACE_END() could happen in ZEND_VM_HOT_HELPER(zend_leave_helper, ANY, ANY)
 
yes, it should, but when looking at enter/leave thing, I notice that generators are hard to trace in this way ...
they don't return, or more precisely yield doesn't ... I'm not sure of the best way around that ...
 
generators are weird with execute_ex hooks as well, each yield rentry is "measured" as its own call
 
while dmitry wasn't involved in the conversation on github about enter/leave, he did email me, he doesn't want to do it ...
that's why I closed the pr ...
 
yeah i though its easy to add this as an extra spec handler to the opcode, this way it would have zero impact on the hot path
without instrumentation/tracing
 
9:59 PM
yeah I mean he doesn't want to introduce support in the jit for it ... I could have redone the patch for the vm, but it's pointless if dmitry won't support in jit ...
I think it might be an idea to circumvent asking for his permission, maybe an RFC is the thing to do ...
when you lay out the case for why it's needed, most people are going to be sympathetic, and want their tools to work ... if enough people get on board and it's voted in, then he'll have no real choice in the matter ...
 
yeah, but we need a bullet proof implementation for this
 
I tried to work with him, it's impossible, so work around him with the processes we already have may be the only way ...
get as far as you can, then holla at me and I'll finish it ... I'm not interested in writing an rfc, arguing on internals, or arguing with dmitry, but I'll write code all day long if you want me too ...
you know having the enter callback before entering is problematic, it means you don't get the args
and having end after execute_ex you won't get return value
 
i was placing them as best as i could for now
 
I'm not sure of the solutions to those problems either ... the only way I see at the moment would be to introduce an opcode after prologue and having begin cb invoked there, rather than on enter, then having end in leave helper ...
I'm unsure
 
i_init_func_execute_data() does the arguments it seems
 
10:11 PM
oh yeah sorry, SEND is before the call, the prolog is recv_*
I still think you might want enter after prologue ... but I'm unsure ...
 
the vm enter insight was alredy super healpful, i want to re-evaluate now how that might work, but tomorrow :)
 
yeah, mistakes are coming out, so I'm also quitting for today :)
nn
 

« first day (3052 days earlier)      last day (1896 days later) »