« first day (3830 days earlier)      last day (1102 days later) » 

4:11 AM
BTW @FélixGagnon-Grenier discord.gg/CJvZEds4
Sometimes we are on there when doing stuff
 
 
2 hours later…
6:30 AM
moin
 
6:58 AM
posted on April 11, 2021

 
 
1 hour later…
8:04 AM
@NikiC ping
partial(?, ?, "c", f(3 as $a, $b, $c)) == f(2 as $a, $b)
achieving this, which I very much want to, lest we give another reason for people to make fun of php ...
it looks like it requires editing/nop'ing code
correcting the arginfo causes incorrect prologue, we would also have to correct that ...
I'm unsure about doing that ... I mean I see how to do it, but would that really be okay ?
maybe I'm looking at it wrong ... but I don't see a way to produce the correct function without editing the prologue, being that we definitely do not want to add any code to normal prologue opcodes (recv*) ...
and editing them is clearly not simple (because it will involve copying them first) ...
maybe it's not sensible to care ... another reason to make fun doesn't make a lot of difference, just throw it in the pile of existing reasons ?
maybe my thought that even an unexpected branch in recv is wrong ?
 
8:27 AM
@JoeWatkins I don't get what that is supposed to do
What is 3 as $a?
 
number of argument as $
 
I still don't get what that code is doing
 
oh it's not code, it was a (very terrible) expression of just what partial application is supposed to do
sorry, I'm sure you don't need to see the math ... we want the result of partial application to create a function with the correct number of arguments
oh I think I see another way, we have to produce an internal function with the correct number of arguments that forwards to the original function, rather than editing the original function ...
 
@JoeWatkins ah, okay
yes, I think you need to use a trampoline function
 
yeah I got there eventually
 
9:19 AM
@IMSoP keeping the os up to date can be solved with unattended updates
 
10:07 AM
morns
 
With PHP's AST parsing/compiling... I'm looking at some work that DanAck has started in relation to the [is_literal RFC](https://wiki.php.net/rfc/is_literal), the bit I'm looking at are summarised here:

https://github.com/craigfrancis/php-src/commit/06e54b14a05185e2eab4bdab8c3324dd22028c7d

When I run this with `is_literal('hi')`, I get:

is_literal1 = hi (0) (0x107e7f068)
is_literal1 = hi (1) (0x107e7f068)
is_literal2 = hi (0) (0x107e190d0)

The first 2 lines show that the flag gets set in `zend_compile_expr_inner`, but the flag has been lost when it gets to the `is_literal` function.
 
 
2 hours later…
12:10 PM
(Equally out my depth) but wouldn't it be a flag for the zend string, rather than the zval?
 
Ahh, ok, I’ll have a look at that in a bit (food time), thanks Mark.
 
 
1 hour later…
1:38 PM
@CraigFrancis
diff --git a/Zend/zend_compile.h b/Zend/zend_compile.h
index d4ee0bf27c..fb6ed186fe 100644
--- a/Zend/zend_compile.h
+++ b/Zend/zend_compile.h
@@ -697,6 +697,9 @@ struct _zend_execute_data {
 /* convert constant from compile-time to run-time */
 # define ZEND_PASS_TWO_UPDATE_CONSTANT(op_array, opline, node) do { \
 		(node).zv = CT_CONSTANT_EX(op_array, (node).constant); \
+		if (Z_TYPE_P((node).zv) == IS_STRING) { \
+		    GC_SET_IS_LITERAL(Z_STR_P(zv)); \
+		} \
 	} while (0)

 #else
@@ -711,9 +714,14 @@ struct _zend_execute_data {
 
Thanks again Mark, while they are still both _zval_struct, checking the flags has led me to zend_add_interned_string(), which seems to be replacing the value (with new flags).
Thanks @JoeWatkins, erm, you might need to give me a bit to wrap my head around that.
 
you know how to apply it ?
 
yep, I know how to apply a diff... it's just my understanding of C is, well, very limited :-)
ok, while that's compiling, you've updated ZEND_PASS_TWO_UPDATE_CONSTANT (x2) to pass along the is_literal flag?... is that by any chance related to the bit where I was seeing the flag disappear when going though zend_add_interned_string()?
 
many things happen to the op array while being compiled, such as caching ... some of those things may result in loss, making the update in pass two makes sure the flag is set even when opcache is done caching
diff --git a/Zend/zend_compile.h b/Zend/zend_compile.h
index d4ee0bf27c..fb6ed186fe 100644
--- a/Zend/zend_compile.h
+++ b/Zend/zend_compile.h
@@ -697,6 +697,9 @@ struct _zend_execute_data {
 /* convert constant from compile-time to run-time */
 # define ZEND_PASS_TWO_UPDATE_CONSTANT(op_array, opline, node) do { \
 		(node).zv = CT_CONSTANT_EX(op_array, (node).constant); \
+		if (Z_TYPE_P((node).zv) == IS_STRING) { \
+		    GC_SET_IS_LITERAL(Z_STR_P(zv)); \
+		} \
 	} while (0)

 #else
@@ -711,9 +714,14 @@ struct _zend_execute_data {
sorry, incomplete patch
should apply against master, I've made a mess locally, was working on something else ...
 
(I know that feeling... with many fprintf's now littering everything)
 
1:50 PM
Too slow, your flag has already been eaten
 
oh no, what did it get used for ?
 
gonna be even slower now, I think I'll reset to master and re-compile again (yay, another segmentation fault)
 
@JoeWatkins Interned strings can now cache the class entry for the class name
 
I'm looking at php-8.0 on heap.space :(
IS_STR_CLASS_NAME_MAP_PTR
 
The refcount doubles as the map ptr offset in that case
 
1:54 PM
and I saw that happening, the fast class cache pr ... but didn't connect the dots
#define IS_STR_VALID_UTF8 (1<<9) /* valid UTF-8 according to PCRE */
so after that, right ?
we're always running out of space for things ...
 
Sorry, not following... we can't use define GC_IS_LITERAL_FLAG (1<<5) any more?
 
@NikiC anywhere/anything we can (re)use ?
@CraigFrancis right, some other feature took that flag for another meaning already
 
@JoeWatkins as strings don't use gc, maybe it's possible to overflow the flag space with some adjustments elsewhere...
 
2:12 PM
One could say you've got a "bit" of a problem.
 
yep, just a bit... and a segmentation fault, so not going well :-P
 
that won't work against current master now, you're seeing interference from the thing using the flag
or most likely ...
 
@NikiC How does main.php.net run on PHP 8 while using the old mysql_* functions?
 
ahh, ok
 
2:20 PM
diff --git a/Zend/zend_types.h b/Zend/zend_types.h
index 3aa16ffe13..1765fce50c 100644
--- a/Zend/zend_types.h
+++ b/Zend/zend_types.h
@@ -675,10 +675,6 @@ static zend_always_inline uint32_t zval_gc_info(uint32_t gc_type_info) {
 #define GC_PERSISTENT               (1<<7) /* allocated using malloc */
 #define GC_PERSISTENT_LOCAL         (1<<8) /* persistent, but thread-local */

-/* Literal string re-uses the GC_PROTECTED bit. This is safe */
-/* to do as GC_PROTECTED is only used for arrays and objects */
does that look safe to you ?
(it appears to work)
I don't see what adjustments I might have to make, other than that ... but elsewhere I'm not sure ...
and tests pass ...
 
Maybe you're in luck then :)
 
services.db.environment.MYSQL_RANDOM_ROOT_PASSWORD contains true, which is an invalid type, it should be a string, number, or a null
YML <3
@NikiC Thanks!
 
but ... I still feel like you know something I don't ... what did you expect I would have to change ?
 
@JoeWatkins some gc code that looks at the top bits to check whether a root is registered
But possibly it just never gets hit because it's NOT_COLLECTIBLE
 
that has to be true ...
 
2:36 PM
ok, compiling seems to have worked... and the basic principal seems to be working... I suspect the seg fault with NULL and numbers is because the is_literal function definition needs to check that it's IS_STRING?
@JoeWatkins thanks for the patches, they are very helpful.
 
@CraigFrancis yes
 
RETURN_BOOL(Z_TYPE_P(arg) == IS_STRING && Z_IS_LITERAL(*arg));
I think that kinda works
 
@kelunik auto-parsing of yml values to a type is stupid
 
2:52 PM
Nice, thank you so much for this... I think DanAck wanted to make some additions, like a literal_implode() function that can implode an array of literals (as we don't think an RFC would pass that allowed string concatenation of literals, due to performance concerns)... but do you know what would happen next... do I use your work to "open a pull request"?
 
Danack (and separately Dan's beard) are seasoned contributors ...
I'd just make noise at him ...
 
You should consider moving the flag up to the opcode level, so I can mark some of my old work as literal rubbish.
 
Thanks again
 
at this point I bow out, unless you ask for help again ...
btw I don't really like the macro names ... I think they ought to reflect that this is specifically for strings, maybe be moved to ZSTR_ api ... but dan can do all that on his own ... and he can do the functions most likely too ...
 
Yeah, I think I'm done for today, but I really do appreciate this, it took me a few days to get where I did, and still had no idea where to go next.
 
2:55 PM
@Danack noise ^
 
Yep, I think Dan was mostly there, it was just that step of getting the flag set during the AST phase (that's where I broke out the fprintf and hoped for the best).
I know I'm really trying my luck here, but if string concatenation was to pass on the IS_STR_LITERAL flag (if both strings had it set), do you know where I could look... or do you think Dan's right, and there is no chance it would be voted for due to performance issues?
 
he's right, almost certainly
 
fair enough, thanks again.
 
well
yeah it's not really worth it, that's the sort of thing you do if you fail to get the simple thing through
you'd be looking at ZEND_CONCAT, ZEND_FAST_CONCAT, and the ROPE handlers ... it would not be pretty ... but it's doable ... but it would be a small win, extensions can concat however they like, and do in a variety of ways...
there's no way to make them all behave the same ... actually nevermind perf, that's the best reason not to do it ...
the handlers and perf thing isn't really a problem, in theory you could output specialised opcodes and not interfere with the original ones, but the extension problem would still be in the way ...
 
tbh, I think the version you have made already would be helpful to ORM's already, the literal_implode() would be useful for the more complex cases (like dynamically building up an SQL string), and that would be good enough for (hopefully) PHP 8.1, and we could re-look at string concat if enough people ask for it.
 
3:06 PM
do what you can well, and this is about it ... perhaps with some utility functions (although not really convinced of that)
 
when you say extendions?
*extensions... I assumed that a simple $table = 'user'; $sql = 'SELECT * FROM ' . $table; wouldn't affect any extensions?
(says he with pretty much no experience of internals).
 
where extensions exchange strings with userland, they're not obliged to use the zend_string api - the input could be all literals but the output won't be ...
 
interesting, do you know of any examples? just wondering how that could affect things.
 
they have to return a zend_string, but they're not obliged to use zend_string* functions for formatting the input (they can access the raw char array, reallocate a larger space and so on), so there wouldn't be a sensible way of carrying the information ... you would have to find each extension and make it comply if you wanted extensions and userland to work the same way with regard to concat/formatting strings
someone opened a pull request to change an extension from using char arrays to zend_strings just the other day, I forget which ... the point is, this is common place ...
tl;dr;du forget concat at the code level, provide at most an implode/format function ...
you might be best to leave that to userland ...
 
yep, I think you're right... I'm just wondering if a baby concat thing might be possible for a later version (as in, if it works, great, but no worries if not)... do you know of an extension that would break this? maybe xdebug?
 
3:21 PM
you'd have to test/look ... but I've read the source code, at least once, of probably every extension in general use ... and it's a loosing battle is my opinion ...
 
Yeah, sorry, not trying to convince you, just trying to understand what the extensions might be doing (and could that cause issues in other ways, like removing the flag just because they just copy the value, for some reason).
 
krakjoe@Fiji:/opt/src/php-src$ cat is_literal.php
<?php
$concat = "Hello " . " World " . 42;

var_dump(is_literal($concat));
?>
krakjoe@Fiji:/opt/src/php-src$ sapi/cli/php is_literal.php
bool(true)
I just looked at the handlers ...
 
ohhh, ok, how did you do that?
 
I didn't, it's optimized away
in that case
function name: (null)
L1-6 {main}() /opt/src/php-src/is_literal.php - 0x7f52bc081000 + 8 ops
 L2    #0     ASSIGN                  $concat              "Hello  World 42"
 
ahh, ok, I think this is what Dan might have been suggesting in the RFC... so that only works when concatenating literals in the script... but would break as soon as a variable was introduced?
 
3:35 PM
@NikiC people.php.net seems to be broken: people.php.net/kelunik
 
krakjoe@Fiji:/opt/src/php-src$ cat is_literal.php
<?php

$b = " The ";

$concat = "Hello " . $b . " World " . 42;

var_dump(is_literal($concat));
?>
krakjoe@Fiji:/opt/src/php-src$ sapi/cli/php is_literal.php
bool(false)
but not in that case
that makes it a bit fragile
it makes it harder to define exactly what "is literal" means ...
 
yeah, tbh, it's more of a nice to have
 
no I didn't do anything to make that happen, that's how it behaves now
 
yeah, sorry, I meant the use of variables.
 
@kelunik Hm...
 
3:38 PM
But taking a typical ORM that takes small strings, like Doctrine doing this:

$users = $queryBuilder
->select('u')
->from('User', 'u')
->where('u.id = ' . $_GET['id'])
->getQuery()
->getResult();
Those $queryBuilder methods could work with that, as they don't need concatenation.
 
@kelunik Not really obvious to me what broke
 
On another queryBuilder related note. How can you query a relation where it doesn't exist?
 
It looks like people.php.net caches the result for one day -- regardless of whether it's an error response or not
 
@CraigFrancis this is not good, pass it as a parameter - doctrine-project.org/projects/doctrine-orm/en/2.8/reference/…
 
@KerrialBeckettNewham Yep, that's what I'm trying to identify :-) ... sorry, long tangent, but the first 3 strings are safe literals, the 4th one, concatenating the $_GET value, is not safe.
 
3:45 PM
$id = $_GET['id'];
...
->where('u.id = :id')
->setParameter('id', $id);
 
Yeah, sorry, I should always post these examples along with a "DO NOT DO THIS!!!"... the RFC I'm putting together is to provide a way for PHP to identify these mistakes, and (phase 1) get the ORMs to protect themselves from them.
I've given up trying to teach every programmer about these issues, so thought I'd find a solution... ~6 years later... wiki.php.net/rfc/is_literal
 
@CraigFrancis nice :)
@CraigFrancis honestly, I didn't read the whole thread and assumed it was a genuine question :')
 
Yeah, sorry, been a long few days - me, trying to write C, not a good idea :-)
 
I think I have more changes ...
 
3:55 PM
krakjoe@Fiji:/opt/src/php-src$ cat is_literal.php
<?php

$b = " The ";

$concat = "Hello " . $b . " World " . 42;

var_dump(is_literal($concat));
?>
krakjoe@Fiji:/opt/src/php-src$ sapi/cli/php is_literal.php
bool(true)
 
nice, how did you get that to work?
 
diff --git a/Zend/zend_vm_def.h b/Zend/zend_vm_def.h
index c30fbe85b9..058dba0648 100644
--- a/Zend/zend_vm_def.h
+++ b/Zend/zend_vm_def.h
@@ -418,6 +418,10 @@ ZEND_VM_HANDLER(8, ZEND_CONCAT, CONST|TMPVAR|CV, CONST|TMPVAR|CV, SPEC(NO_CONST_
 			memcpy(ZSTR_VAL(str), ZSTR_VAL(op1_str), ZSTR_LEN(op1_str));
 			memcpy(ZSTR_VAL(str) + ZSTR_LEN(op1_str), ZSTR_VAL(op2_str), ZSTR_LEN(op2_str)+1);
 			ZVAL_NEW_STR(EX_VAR(opline->result.var), str);
+			if (Z_IS_LITERAL_P(op1) && Z_IS_LITERAL_P(op2)) {
+			    Z_SET_IS_LITERAL_P(EX_VAR(opline->result.var));
 
@NikiC So maybe still the token rotation?
 
but there's no reasonable way to have the most complex cases, ie.
$c = count([1,2,3]);
$a = 'WHERE id IN (' . implode(',', array_fill(0, $c, '?')) . ')';

var_dump(is_literal($a)); // true, the one exception that involves functions.
pass ...
 
oh, don't worry about that one, that's really complicated (would need a special function)
 
4:01 PM
well no, you could change implode, but you'd have to change an enumerable number of functions, some of them in extensions - this is what I was trying to explain ...
 
but, I have absolutely no idea what you've done there... is that likely to be accepted in a vote?
 
it's not that expensive ... but I don't know that it helps you to precisely define what is considered a literal, or if it makes it harder ...
 
I'm basically trying to say a "programmer defined string" (that's what the JavaScript implementation uses)... I don't want the flag to be preserved though any functions, as they are likely to alter it in a way that's could be dangerous (e.g. thinking that escaping will work).
 
With doctrine queryBuilder, how can you query a relation where it does not exist? for example contests and applications you want to display only the contests the user has not applied for.
 
then I'd probably include the vm changes, if it's too slow dmitry or nikita will tell you ... you could make take some measurements yourself and determine what it really costs ...
(I think it's fine, but I'm always wrong about everything)
 
4:07 PM
Could you commit that to your is_literal branch, and I'll see what I can do... does seem a little too good to be true, so I'm not going to celebrate just yet, but you've really got me much further than I hoped.
 
4:17 PM
@kelunik This uses a different endpoint with a different token
And I believe a token mismatch would result in a different error
 
@CraigFrancis done, with a test
missing rope stuff
--TEST--
test ropes + is_literal function
--FILE--
<?php
$hello = "hello";
$world = "world";
$sprintfed = sprintf("hello %s", "world");

var_dump(
    is_literal("{$hello} {$world}"),
    is_literal("{$sprintfed} {$world}"));
?>
--EXPECT--
bool(true)
bool(false)
I think that's it ...
 
4:42 PM
@kelunik By the way, wiki.php.net is making a lot of cvsauth requests
Could it be doing something dumb like querying authentication every time a logged in user visits a page?
Yes, I'm pretty sure that's exactly what it's doing :/
 
@NikiC github.com/php/web-wiki/blob/… looks like it, no?
 
that would go some way to explaining the slowness also
wow
when was that ever a good idea, that's not because the code is old, or written for php4.1 ... it's just crap
 
5:32 PM
@NikiC You can skip the bcrypt check if the password is empty I guess (for now).
 
 
1 hour later…
6:46 PM
@JoeWatkins Thanks Joe, I’ll follow up with all the other bits during the week (sorry, had to leave quickly, didn’t realise the time)... really appreciate all the magic you’ve done today.
 
7:45 PM
@kelunik You have push access to web-master, right?
 
@NikiC Yes, I do. But I don't have existing contributions.
@NikiC I don't have server access, so don't know what's available there nor can I do DB migrations.
 
@kelunik Just checking whether I need to merge your changes ^^
 
@NikiC Please do for now, yes. :P
 
people.php.net stubbornly remains broken. The endpoint it uses does work for me locally :/
 
The search still works. :o
 
8:10 PM
I've got a bit of time I can use to look into it if I can get a hold of the token
 
8:28 PM
@Derick When you have time, could you check what /tmp/ca8d0c7620c6d0f94853386f6e6f5e4b92a947c2 looks like on svn2? I believe that's the file where people.php.net is caching the user data...
And maybe what mtime it has...
 
@NikiC Given the recent events, is it also reasonable to start using libraries for php.net web projects?
 
@kelunik Depends
 
@NikiC :P Who's to decide?
Care if I change the namespace to PHP\Web\Main?
 
8:44 PM
\PHP is reserved in the language... for god only knows what reason at this point, but you might want to use PHPNet or something instead
 
@kelunik Don't particularly care, but it's the standard namespace
Used by web-bugs, web-pecl etc. which is why I went with it ^^
 
So that's why you didn't want to use vendor prefixes? ...:P
 
@MarkR Those weren't my idea :P
 
eugh, every single function in this class has its parameter stubbed as "$codepoint", except one which uses "$character" github.com/php/php-src/blob/master/ext/intl/uchar/…
technically, it would now be a breaking change to correct it :(
 
9:04 PM
@IMSoP i might remind you of wiki.php.net/rfc/named_parameter_alias_attribute ;)
 
yesssssss! I wantssssssssss it!
I see internal use is currently under "open questions"; I think this is a good example of why that's definitely also needed
 
i need to ship doctrine 2.9 with attributes first, then i have time for php-src again :D
 
@IMSoP We're still somewhat liberal about that
 
@NikiC I did wonder - and it is a function with only one param, so actual named use is pretty unlikely
if I wanted to slip a change in, should I target 8.0 or master?
 
@NikiC Adding PDO broke the docker setup :P
 
cmb
9:28 PM
@IMSoP target 8.0 and ping RMs?
 
I'll have a look tomorrow
need to pop some of the other things I've found off my stack/queue first 😆
I think it started with wanting to add a note to the mb_detect_encoding docs...
current focus is fleshing out the stubs for mb_ord and mb_chr
 
10:13 PM
@JoeWatkins that would be me :p but it was also to use some clearer APIs which use zend_string (mainly zend_string_equals_literal(_ci)) because I found the code very confusing (and it's "easy" cleanup)
 
 
1 hour later…
11:33 PM
@PeeHaa @Ekin where'd @Jeeves go? @PeeHaa's fixed so @Jeeves is chopped liver? @Jeeves also why does @Jeeves show two portraits... (browser refresh still shows two)
 
@Tiffany There's @PeeHaa Jeeves and Jeeves Jeeves, thus two Jeeves images, isn't that obvious?
 
lol
 

« first day (3830 days earlier)      last day (1102 days later) »