last day (41 days later) » 

10:55 AM
1
A: How to combine skipping and non-skipping (lexeme) rules?

seheThe answer I linked to earlier describes all the combinations (if I remember correctly): Boost spirit skipper issues In short: any rule that declares a skipper (so rule<It, Skipper[, Attr()]> or rule<It, Attr(), Skipper>) MUST be invoked with a compatible skipper (an expression that can be ass...

 
Answered a side-question as well
 
llm
thanks again - the flexibility of Spirit is so hight that i just don't see all the equaly attachable strategies
 
Yup. I like spirit because it allows you to be very productive when you know the sweet spot of convenience features. Until then, it's a swamp that will slow you down at least as often as it would save you time (like on testing the building blocks)
 
llm
i started to write my expression parser manualy - a friend forced me (without decent spirit expirience) to use Spirit - so now im knee deep in the swamp, btw: your example doesn't work for me - see Update
 
@ilm if you can share the code (even if it doesn't currently compile) it would help me spot the problem more quickly - right now I'd have to make many assumptions (I do note that you have char_(",") again, on purpose?)
 
llm
10:55 AM
sorry, updated the question with coliru-link: coliru.stacked-crooked.com/a/adcf665742b055dd , i used the qi::char_ for getting the chars also in the result - maybe i should replace that with qi::lit
even removing all the char_ and lit_ downto simple chars will give the same not castable to Skipper error
 
Okay, the problem is that you mixed qi::blank and ascii::blank_type which do not correspond.
 
llm
Hello
 
:)
Besides, I'd say write a >> *("," >> a) as a % "," (equivalent) and to expose the "source string" use raw[a % ","] again. However, I still fail to see the value of parsing anything when all you do is return the input as text again. That's not going to much except a very rough syntax check
 
llm
i thought it would be easier to just get the parsing aka "rough syntax check" done first, due to the dynamic of the interpreter behind i need to do the semantic check after the full parsing
 
It's going to hurt you down the road because you will have to rewrite all the rules to match the attribute types.
 
llm
11:04 AM
i've got a dynamic object/member(function) tree behind the parser that is referenceable with my identifier_chains like "a.b.c[23].e(2)"
 
You can read my many answers, and 99% of the time I begin by defining an AST type to receive the parse results, and then write 1:1 rules to be compatible with them
 
llm
"It's going to hurt"... ah that make a difference - i though it would be just easier
 
It is easier just to get the hang of parser expressions, sure. But to get a working parser that actually parses, not so much.
 
llm
its my first try - it was easiest to get into Spirit, and its still not 100% clear what the resulting syntax will be - im more or less try what i can reach with Spirit in the first place to parse
 
@llm Here's a fixed coliru that also fixes the placement of qi::skip to include the closing ']' (because otherwise `"a[b,c ]" would not be accepted): coliru.stacked-crooked.com/a/8125da8e1404a747
Oh, by the way I added BOOST_SPIRIT_DEBUG_NODES so if you uncomment the #define BOOST_SPIRIT_DEBUG line you get debug output: coliru.stacked-crooked.com/a/ca64d51b5d2ff9d4
 
llm
11:11 AM
thank you - ... testing...
this is my current Syntax-Idea:
more or less Pascal like expressions

Immediate/Value Types: Bool, String, Float, Int

Some const Values like: True/False, Pi, ...

Object/Namespace-Hierachy with Subscriptions and Function/Method Calls

'+' '-' '*' '/'

'=' '<>' '>' '<' '>=' '<=' ':='

bool-stuff:
'and' 'or' 'not' 'xor'

'(' ')'
the debug output is nice
i don't understand why the '[' needs to be part of
doesn't that mean that there can be a blank before/behind the [ and ]?
 
If you move it back out, and inspect the debug output on the last test case, you will see why :).
It's subtle: pre-skipping happens inside the qi::skip, but post-skipping happens outside.
(By the way, from your test case I got the impression that x[ a.b, c ] SHOULD be valid. If not, then the [] need not be inside the skip[] directive indeed)
 
llm
it is but not "a.b [ a.b, c ] [x]" ... wait for compilation to test
 
That's gonna do what you want indeed.
It's uncommon for grammars to be selectively whitespace sensitive like this though. It's a bit of a design smell for me.
 
llm
so many options/variants in Spirit, how did you need to master it?
how long did you need to master it?
the stackoverflow and boost-mailing list history is strong with you, young jedi :)
" a.b [ c , d ]" does get parsed - the blank between b and [ is not ok
a . b;
c [2];
ok C/C++ allow it - but i would try to force some sort of style so there will be no scripts with "a.b.c" "a . b .c" etc. notations
I'm not so sure about that
would you also allow blanks around '.'?
 
11:35 AM
@llm Long. In fact the first time I tried it I gave up (but that was also partly because the compiler we used (xlC++) didn't really support it well at the time.
 
llm
did that even work a bit?
 
I think not. I half remember using a custom gcc build to get it to work
That must have been around ~2002. Second time I tried was much less bad, and I came up with a very naive grammar that I had previously written using Poco/C++. It was completely centered around semantic actions instead of attribute propagation, but it worked.
Later I got the hang of attribute propagation (https://stackoverflow.com/questions/8259440/boost-spirit-semantic-actions-are-evil) and since then have done the wax-on/wax-off so often on Stack Overflow that I can basically see the sweet-spot/happy path from the start, and smell most pitfalls before they happen or recognize quickly when
 
llm
are you using Spirit for your daily work or time by time - for me its the second time in 10 Years
to recap:
-you would not use lexeme for spaces around . [ ] ( ) - because of the smell
-would reduces my lexemes down to the string
problems solved :)
-what would be the solution to not allow spaces around these?
its already working for the '.'
wife is calling (the fifth time, getting angry :)) lunch time - still in chat in ~30-45min?
bbl
 
Yeah I'll keep an eye out here. You can always "summon" me using @sehe, and I get notified like you will now: @llm
 
llm
12:10 PM
back again!
 
Hi
Now it's gonna be lunch time for me, but in the mean time I sketched an AST type that I would start out with:
    namespace Ast {
        using Identifier = std::string;
        struct String : std::string {
            using std::string::string;
            using std::string::operator=;
        };

        using Number = boost::multiprecision::cpp_dec_float_100;

        enum Operator { Plus, Minus, Mult, Div, Mod };
        struct Member;
        struct Binary;
        struct Unary;
        struct Call;
        struct Subscript;

        using Expression = boost::make_recursive_variant<
            Number,
(Somewhat jovially using cpp_dec_float to avoid requiring double/integer separation, also to avoid any floating point inaccuracy when parsing from decimal notation in input text)
A parser skeleton would like like this (obviously needs all rule definitions, but the declarations should show the idea):
template <typename It>
struct Grammar : qi::grammar<It, Ast::Expression()> {
    Grammar() : Grammar::base_type(start) {
        start = qi::skip(qi::blank) [ expression_ ];
        expression_
            = '(' >> expression_ >> ')'
            | quoted_string
            | identifier_
            | number_
            | member_
            | binary_
            | unary_
            | call_
            | subscript_
            ;
    }

  private:
    qi::rule<It, Ast::Expression()> start;

    qi::rule<It, Ast::Expression(), qi::blank_type> expression_;
See how driving it from the AST and keeping 1:1 rule compat helps keep sanity? The most complicated trick in this picture is the recursive variant. That's intrinsic complexity, and you would have to solve that same problem with any approach (although you would be more likely to use dynamic polymorphism (OOP hierarchy, so using dynamic allocations explicitly because of C++) when using something more traditional instead of Spirit)
@llm I'll be back after lunch
 
llm
12:28 PM
time for me to come up with a more complete example what i try to reach - currently got the problem to understand the Member,Call and Subscipt splitting
 
llm
12:38 PM
expression = immediate | identifier_chain
assignment = identifier_chain ":=" expression

assignment examples:
a := 10
a.b.c := 10
a.d[30] := 10
a.d[30] := a.b[20]
a.d[30].c(34) := a.b[20]
a.d := a.b
a.d[20].c(a.e*3,20,"hello") := b.c.d*20
a.e := "hello"

expression examples:
a > 20
a+10 > 30
a = True
a = b.a.c
not a
a and (b and c)
a*34+a.b.c(23)
10 if a > b else 30 // python style unary
expression and assignment can be two seperated parsers, but could be also combined - there is another graphical like language around that should keep the complextity of the boolean evaluation and the assignment low
i've developed some sort of simple graphical SFC chart engine around: infosys.beckhoff.com/content/1033/tcplccontrol/Images/…
recursive_variant type - nice
currently i call something like this "a.d[30].c(34)" identfier_chain - its something that directs me (through arrays, method calls) to an (assigneable)value
and i use the boost_1_72_0\libs\spirit\example\qi\calc_utree_ast.cpp example as an orientation - thats why i was using the utree in my samples
would you add something like True, False direktly to the Parser or as a part of the (sematic) symbol analyse after parsing?
 
llm
1:10 PM
and a identifier_chain can be just 'a' - isn't your splittup of subscript,call and identifier a problem with that - subscript and call are part of a chain in my world, or is that hard to store this way?
so 'a' is just a very simple chain
 
llm
1:41 PM
the identifier-chain is double-used for member access "forms[0].asCircle().pos.x" or as namespace "Math.Trig.Sin()"
but i have currently no good idea how to express the chaining
in the AST
"a.d[20,a*b,a[2],b(3)].c(a.e*3,20,"hello")"

from identifier_chain viewpoint

"a.d[].c()"

std::vector<(.identifier|subscript|call)> identifier_chain
and the chain can only start with an identifier
or is it a member tree with special types

a
d
c
nesting does not :(
work
-----------this is not clear to me

struct Member { Expression obj, member; } ;
struct Call { Expression fun; Expressions params; } ;
struct Subscript { Expression obj; Expressions indices; } ;

why is fun and obj an expression?
does that mean that i can write stuff like (3*5)[3]?
or (a.b.c*45).c?
------------
my manual (dirty) parser was recursive so i never got the
problem in wich chain im currently in - how can that be
solved with spirit?
 
 
2 hours later…
3:58 PM
@llm you might be interested in this:
11
A: How to calculate boolean expression in Spirit

seheHere goes a quick and dirty demo based on my old Boolean Parser answer. This is a visitor that evaluates the AST you pass it: struct eval : boost::static_visitor<bool> { eval() {} // bool operator()(const var& v) const { if (v=="T" || v=="t" || v=="true" || v=="True")...

 
llm
thank you - even more to read
 
Also very relevant:
8
A: How to verify algebraic statements using boost::spirit?

seheThe simplest thing that could work, if you ask me would be http://liveworkspace.org/code/1fvc8x$0 equation = (expression >> "=" >> expression) [ _val = _1 == _2 ]; This will parse two expressions, and the returned attribute is a bool that indicates whether both expressions evaluated to the sam...

@llm I encourage you to only scan them superficially. You'll remember what you have seen when you need it
@llm both could work almost equally well. Having hardcoded keywords for literals COULD help you write optimization passes (e.g. False and x is False)
 
llm
i currently try to get my parser example to be complete - coliru.stacked-crooked.com/a/373a4a525c73a9e8
but adding my pseudo binary_op "and" will just crash - and i have absolute no idea why, i know that the parse example is not very clean (ignoring some of your tips see double_) but i see nothing that could produce a crash
its some sort of "side" project because im still struggeling so much with the parsing
 
@llm It's up to you how you wish to represent those. A problem I see with wrapping every identifier with a redundant chain is that the AST grows complicated to work with. I would treat the nodes as a graph with relations ("namespace or member lookup on obj") instead of having special purpose datatypes for fixed compounds of nodes.
This is also why I tend to keep things generic, which brings us to your other question.
2 hours ago, by llm
-----------this is not clear to me

struct Member { Expression obj, member; } ;
struct Call { Expression fun; Expressions params; } ;
struct Subscript { Expression obj; Expressions indices; } ;

why is fun and obj an expression?
Indeed you spotted that well. I didn't want to assume that members could not be calculated (as in most programming languages). Of course, you're free to replace that with just identifier_ if your domain doesn't support that.
 
llm
4:16 PM
i try to understand the difference between member_ and identifier_ - compared to my identifier_chain
 
@llm Left recursion is not allowed in PEGs. Your rules are:
expression = term ...
term = factor ...
binary_op = expression ...
factor = immediate | binary_op ...
Now substituting: expression = factor ... -> expression = immediate | binary_op ... -> expression = immediate | expression .... So if immediate doesn't match, you get infinite recursion (expression recurses into expression immediately, and infinitely). The crash will be a stack-overflow.
 
llm
my hope was that spirit does not compile if i do something like that
 
@llm I'd draw the different AST example on paper (say for a.b[3][4,5].foo()). You're "tokenizing" it as a chain, which has the "elements" a,b[3][4,5],foo(). I'm adhering to classical tradition where the AST follows the type semantics: t1 = a, t2 = t1.b[3], t3 = t2[4,5], t4 = t3.foo()
 
llm
silly hope - Spirit is not a PEG design/trial&error tool
 
@llm Spirit does diagnose a lot, but it's limited by language restrictions. Special purpose parser generators can do a lot better rule analysis (and often even reformulate them into more optimized form)
@llm Well. It is for me. But I've already acknowledged the learning curve.
 
llm
4:21 PM
Spirit is pure magic - without knowing where the magic ends
 
@llm I've jotted these examples in a test program and filled in "straight forward" rule definitions. Most of it works pretty well already: coliru.stacked-crooked.com/a/ad435659b65bc3eb
@llm I hate that aspect of it too. It goes for a lot of Boost. I appreciate Boost and C++, but a lot of it is skirting the limits of the language making it pretty hard to co-exist with the library internals.
 
llm
just wow!
 
:) See what I mean. I like the productivity of this. I do realize that cutting through the next 20% could be nigh impossible unless you have a lot of experience. It's a mixed bag
 
llm
same to me when i teach newbies base C++ template magic - now i am the newbie :)
are you involved in the spirit development?
 
I'm not. Although sometimes I contributed tiny patches
The guys do know me - from Stack Overflow and the spirit-general mailing list
 
llm
4:28 PM
where are you from, also slowed down due to corona?
im from germany, 1 weeks in the house, nearly no contact to others - very stressy
 
Same here. I'm near Rotterdam, Netherlands
Two kids, wife working from home.
I happen to just have lost my job march 1st because of the Dutch office closing down. So, it's a weird coincidence that I'm "free". Hard to do the job hunt this way though
I try to limit the shop visit to once a week. We can get part of groceries shipped. But it's crazy crowded so everything is delayed and unreliable
 
llm
im freelancing and 2 of my 3 customers shuted down on monday - i hope it won't get worse - buyed a house 3 month ago and company my wife works for went insolvent in the first week of corona (bigger company want to buy them and canceled the whole deal)
but please send me your contacts and stuff
 
That is harsh stuff. I'm grateful that I'm not in a occupation where the income dries up immediately, and we have some reserves.
@llm What area are you in? If you want you can join chat.stackoverflow.com/rooms/10/loungec - although the majority of the members there have moved onto discord. I can probably get you an invite to that discord as well discordapp.com/invite/gRFKWwn
 
llm
4:44 PM
what is the context of the this room?
ah i see Lunge<C++>
Lounge
 
It started out as ba opinionated bunch of C++ enthusiasts on SO moderating the [c++] and related tags
They still do, although most of them are less active on SO than before (like myself)
 
llm
im very sorry to say but ~there are bugs~ in your hyperspeed implementation :)

"a.d[20].c(a.e*3,20,\"hello\")" OK: (a (d {20}))
Remaining unparsed: ".c(a.e*3,20,\"hello\")"
----
"a.d[30].c(34) := a.b[20]" Failed
Remaining unparsed: "a.d[30].c(34) := a.b[20]"
----
"a.d[20].c(a.e*3,20,\"hello\") := b.c.d*20" Failed
Remaining unparsed: "a.d[20].c(a.e*3,20,\"hello\") := b.c.d*20"
----
 
Hehe. "Most of it works pretty well already" and I'll also quote "I do realize that cutting through the next 20% could be nigh impossible unless you have a lot of experience. It's a mixed bag"
I don't consider it bugs, just limitations that don't work out for the requirements. I'll see what I can swap around.
 
5:03 PM
Okay, [this was kinda advanced wizardry](http://coliru.stacked-crooked.com/a/a74756ade857f992), but it does
1. work
2. make everything way more efficient
3. remove the need for most FUSION_ADAPT_STRUCT (I left it in for debug printing only)
 
llm
testing...
 
I'm benchmarking your CPU performance now
@llm And now with all cases enabled again and cleaned up the Fusion use by implementing operator<< better, manually: coliru.stacked-crooked.com/a/3793fcc53e91ee43
The output speaks for itself:
                                "a > 20" OK: a>20
                             "a+10 > 30" OK: a+10>30
                              "a = True" OK: a=True
                             "a = b.a.c" OK: a=b.a.c
                                 "not a" OK: nota
                       "a and (b and c)" OK: aandbandc
                        "a*34+a.b.c(23)" OK: a*34+a.b.c(23)
                   "10 if a > b else 30" OK: 10 if a>b else 30
                                     "a" OK: a
Note that it now has an AST with (almost¹) full fidelity. So you can base whatever transformations you want on it (e.g. evaluating expressions or assignments).
The ¹footnote is about operator precedence: I skimped on that for brevity. I have dozens of expression parsers that DO heed operator precedence on Stack Overflow so I'll leave that for now.
 
llm
5:19 PM
thanks alot, would i be reasonable to base the operator precedence just complete on the resulting AST inside my interpreter,so no change to the parser or is the change in the parser easier?
 
Depends a little on taste. Both are viable. I'd say it's easier to express in the parser, although it does generate a few extra "levels" of parser rules - as many as there are precedence levels, basically.
I'd prefer doing it in the parser.
@llm Cosmetic fixes on pretty printed AST: coliru.stacked-crooked.com/a/687682a327069104 (I'm sure you saw the glitches :))
Okay, I enjoyed a good old-fashioned day of hacking. I'm going to start dinner here. Feel free to @ me if you have any more ideas/questions
 
llm
thanks again, have a nice dinner
im too impressed to see glitches
 
Mental note: when making bugs, prefer to make them impressive so you can wow the customer :)
 
llm
go eat!
thx - bye, time to feed the kid
 

  last day (41 days later) »