« first day (1614 days earlier)      last day (2166 days later) » 

2:41 AM
@earl As soon as you are past tail you are doomed. If there is no way for PARSE to check each time whether its pointer is past the current tail, then disaster is unavoidable.
Seems like Ladislav's mathematical approach to series' is not just laudable but also required.
If that is the reason PARSE is so fast, well, that's ... sad.
 
3:15 AM
Protecting things from being garbage-collected, that's easy.
But maybe PARSE doesn't even do that, you know, because that would slow it down too much, performing a whole additional addition :)
Late for me and I'm tired too sorry.
 
3:32 AM
Read the top post on this blog... http://morepypy.blogspot.com/
That is really cool.
 
 
2 hours later…
5:15 AM
@MarkI I doubt the performance would be impacted terribly if it were rewritten to work like DO does. Addition is cheap.
What I've done with the PROTECT/UNPROTECT may be slow, though. I haven't checked the impact of that.
@JacobGood1 "You identify medium- or large-sized, likely-independent parts of the code and to ask PyPy-STM to run these parts in parallel." Hum. OpenMP-ish, but for Python?
 
5:35 AM
@MarkI In the current implementation AFAIK, if you have a giant series and clear it, the system is not aggressive about shrinking it. But it could be aggressive in doing so, and realloc just as expanding it is technically forced to. But you are correct...executing on the data past the tail is just wrong.
 
5:55 AM
Heya @Respectech... any news?
Rebol's whole memory pools thing... why? Why doesn't it just use malloc and such? It makes it harder to debug errors and I would bet it makes things slower. It's just overhead and winds up killing the benefit you could get with something like tcmalloc I bet. And it means valgrind can't help you when bad things happen, because as far as it knows the memory is still "good". :-/
This gives me an unfortunate idea, which is that I try building Rebol minus the memory pools and just use tcmalloc and see what happens. Unfortunate not because it's not probably better, but unfortunate because it means I have once again created more work for myself...and it will probably mean the exposure of many bugs via valgrind
 
6:42 AM
posted on April 02, 2015 by fork

[Bug] In the implementation of PARSE as written, rules blocks are processed via a pointer into the series data block. This pointer is incremented until an end-of-series marker token is reached in the data area: https://github.com/rebol/rebol/blob/25033f897b2bd466068d7663563cd3ff64740b94/src/core/u-parse.c#L668 Shrinking the size of a series does not currently reclaim or reallocate extra memo

 
 
1 hour later…
8:08 AM
All right, my test build minus the memory pooling does...indeed...let you catch this problem in Valgrind. (I was wondering why Valgrind wasn't giving any complaint before)
'>> rule: make block! 1
== []

>> append rule [(append rule ["c" "d" "e"])]
== [(append rule ["c" "d" "e"])]

>> parse "abcde" ["a" "b" rule]
==21086== Invalid read of size 1
==21086== at 0x46E26F: Parse_Rules_Loop (u-parse.c:680)
==21086== by 0x46DB82: Parse_Rules_Loop (u-parse.c:941)
==21086== by 0x46BC06: Parse_Series (u-parse.c:99)
==21086== by 0x46ECF4: N_parse (u-parse.c:1301)
==21086== by 0x40DE83: Do_Native (c-function.c:289)
==21086== by 0x40683E: Do_Next (c-do.c:878)
 
 
4 hours later…
12:16 PM
@HostileFork Quick thought: it may be enough to have CLEAR replace all the items with END!s, instead of just the current position.
 
@MarkI It's a good thought for CLEAR, but that's not the only mutator
 
@HostileFork Of course. Great valgrind post BTW, interesting reading (which I have not yet finished).
 
What I'm doing now could be very interesting if it works.
If all goes well: a smaller, faster, easier to debug Rebol.
Chasing some other bugs that @rebolek's Lest stress test seems to expose.
 
12:34 PM
Lest always had odd output (or failure to parse) on the 14th time the lest function was called a couple months ago when I was testing it
 
@kealist Yup, I'm on it. :-)
 
@HostileFork Thank you. I'm still not on the level to debug things such as this
And valgrind & other things I had tried didn't show anything too funny (as you figured out)
 
@kealist Yup, I'm hopefully going to change the game on that as in the above
Tightening some screws
 
12:50 PM
@HostileFork Thanks, I will check that.
 
@rebolek Actually, you probably shouldn't bother
I think I'm leaning to the side of fixing PARSE
Sloppy semantics aside, it might be useful (as long as it does not crash)
 
Ah, OK :)
 
But it's not the only problem. There's something else definitely wrong, but I'm doing some changes to make it easier to find out what that is...and it may or may not be like turning over a log and finding a lot of...bugs :-)
 
 
2 hours later…
2:38 PM
From f-modify.c: // I hate unicode: github.com/rebol/rebol/blob/…
 
@HostileFork I actually have comments like that sprinkled in some of my code too.
In my case, they come from a time between starting to learn about Unicode and realizing it has some good points.
In this particular case, you can actually tell that the editor of f-modify.c didn't know enough about Unicode yet -- the binary allocation in the next line is too large by 2.
 
Unicode was retrofitted awkwardly onto Rebol2 in order to implement Rebol3
 
@HostileFork As opposed to poorly in Rebol3? :)
In some ways the R2 "awkward" method is actually more functional than R3 is.
 
@MarkI If you understand Unicode well, you might want to chime in on github.com/red/red/wiki/…
 
They're not show-stoppers, and they will get fixed, but the Unicode bugs in R3 really are annoying -- specifically unicode urls.
 
2:51 PM
I've essentially figured out why Rebol does not appear to leak more than it does. There are functions for creating and destroying series. There is also a garbage collector.
If a temporary series is created and not freed before handing it out to where other places can get their hands on it, then it will still be picked up by the garbage collector.
 
@HostileFork Interesting. I understand Unicode as well as is it is possible to after studying it for more than 7 years.
Which is actually not good enough, if my drift missed you :)
 
There is no control over the "Free_Series" function--when it is used explicitly--to know or not whether it might be referenced somewhere. It has no way of knowing.
Yet some code does explicitly free, because it thinks it knows it's safe.
 
@HostileFork Wow. Here be dragons, indeed.
 
But if it forgets, there's no worry...because at the end of the program all the series will be freed.
So (a) dangerous and (b) arbitrary leaking of internal series during a run.
I'm tinkering with a series flag of "internal, must be freed manually, never shown to the GC" vs. "intended to be sent to user code, may be plugged in elsewhere, cannot be manually freed"
Trying to see if performance + safety + everything else can all be tied together here
@MarkI If you do know Unicode then I would encourage you to indeed look over whatever Red is doing on that front.
A spec for the behavior serves both languages; but Red is trying to be more "right" about it.
Perhaps, in particular, due to the heavy Chinese influence
 
@HostileFork Took a look, as usual Red seems to be taking a nice pragmatic approach.
I will share some pitfall stories, but probably in the other room, and ... well, when I can.
 
3:11 PM
@HostileFork Unfortunately, that behaviour may actually be a bug, or at least accidental.
>> [a/b/#[path! [pathword]]/c/d]
 
; Brought to you by: try.rebol.nl
== [a/b/pathword /c /d]
 
So it can handle the set-path indicator colon, but the path itself seems to be forced to end at the construction.
 
3:23 PM
@MarkI quick, spot the bug... github.com/rebolsource/r3/blob/…
The problem with Rebol trying to do memory pooling on its own in blocks of storage is there was just no way to in a debug mode know how far it was overstepping any allocation.
If I had to bet, there are going to be quite a lot of these.
Valgrind and such can tell you when you alloc something of size 4 and then step out of it, because it sneaks in a trickier allocator. But if you think to yourself "gee, I shouldn't be calling malloc so much... how about make a map from different sizes to big old swaths of memory allocated in one go..."
First of all, you're cheating yourself out of the work being done on fancier allocators which are probably smarter about that than your guessing. But secondly, no one will tell you when you reach outside your memory block unless you step off the head or tail of a pool block
 
3:41 PM
So a good solution to the Internal/Shared thing is to create internal by default, and then say you have to make things shared before returning them to user code. A lightweight check could be to make sure internal series never make it to the eval result. Provide a mechanism for promoting from Internal => Shared, but do not allow a reverse mechanism. Only permit a manual freeing of series if it is Internal and has not been promoted (demoted?) to shared status.
It's not practical to make _Internal and _Shared versions of every function, so that seems more feasible.
 
0
A: What are the rules about self-modifying parse rules in Rebol/Red?

endo64This might work: >> rule: ["a" "b" m: (insert find rule "^(64)" "c" probe rule) skip "d" "e"] or >> rule: ["a" "b" m: (insert find rule "^(64)" "c" probe rule m: next m) :m "d" "e"] >> parse "abcde" rule ["a" "b" m: (insert find rule "d" "c" probe rule m: next m) :m "c" "d" "e"] == true

 
 
1 hour later…
4:58 PM
@HostileFork Will switching the order of the first two tests on line 446 fix it? I "spotted" that one ...
 
5:25 PM
@MarkI It would, but the bigger point is just "valgrind wasn't catching this before, because it couldn't"
Leading to the "what else isn't it catching", and I'll answer that as soon as I get past boot
 
5:52 PM
@HostileFork Awesome work. Looking forward to the new uncoveries ...
 
 
2 hours later…
7:31 PM
@MarkI There are some pretty tough tradeoffs, it would appear. It's like Rebol's C codebase is caught between the desire to leverage its own garbage collector to be Java-Style-Lazy... yet still wanting to be able to make various temp series and then free them. But there's no real strategy to it. I'm trying to impose a strategy...as I said, the ability to "promote" explicit an memory management series to be managed by the garbage collector, and only let you free the ones prior to promotion.
But it gets kind of tricky in the system to decide when to "promote". Right now I'm saying you do it explicitly if you create something, but if it's a copy then the copy will be whatever the source was. So no way to get an internally-managed/freeable clone or copy/deep of a block that has been shared.
If your source was shared, so will be your copy. I just can't see any sense to going in and trying to find permutations of every function just to add flags for making internally-managed copies of GC'd series... guessing that's a solution in search of a problem.
 
7:46 PM
Please forgive me for asking another hard question of the group. You know I only do it because I have to, right?
There is explicit code in l-scan.c to check for special characters that are allowed in issues.
One of the characters that is so explicitly permitted is the comma ",".
But, commas are definitely and absolutely explicitly disallowed in both words and refinements.
Can anybody help me out with any kind of rationalization/explanation/guess as to why this was done?
Because (modulo the restriction that any-words cannot begin with a digit) otherwise all word types share the same lexical space.
 
@MarkI If people asked more questions and worked more things out, more would have been done right the first time...!
>> 10,20
 
; Brought to you by: try.rebol.nl
== 10.2
 
>> print 10.20
 
; Brought to you by: try.rebol.nl
10.2
 
@MarkI Outside of eliminating the comma's role in "visual noise" as a generic interstitial, there was a pervading thought that when reading source a comma and a period are "just too similar" to have them mean different things.
Bolstered by the use of the comma as a decimal point, the notion arose of "anywhere a comma may appear it is equivalent to a period" (except in strings).
 
7:53 PM
@HostileFork I would actually be in favour of allowing commas in word types provided they were changed to periods just like happens to numbers!
 
Yes, that is my stance, although I see it as being like case... so /STRICT would discern them
 
@HostileFork Not after they were converted it couldn't ... so no.
PS what term should I be using instead of any-word if I want to talk about just the 4 any-word types with the word "word" in them?
 
@MarkI Hm? The word ABC can be discerned from abc. So could a.b.c be discerned from a,b,c
I don't know. I want to kill REFINEMENT! in NewPath
I know that I want PAREN! to be GROUP!.
 
@HostileFork You are talking about "," comparing as equal to ".". I am talking about commas converting to periods on scan, just like backslash converts to slash in files.
 
@MarkI I don't suggest that. I suggest if you use a,b and a.b that the words be distinctly recorded, as with casing. But in symbol lookup in the evaluator they find the same thing.
If a dialect wished to discern them, then it could.
But baseline Rebol knows better...
 
7:57 PM
I think I can get away with using "words" for the 4 word things, and "any words" for all 6 ... would that be too confusing do you think (anybody)?
 
If you feel the distinction is important. My main point is to slate REFINEMENT as a two-element NewPath construct. The equivalent of to-path [none name]...that would mold out (source out) as /name)
 
@HostileFork This is the difference between "aliasing" and "equivalence classes". So far in Rebol (modulo bugs of course) there is no concept of dot-comma class, just that comma is an alias of dot. I would not be in favour of changing that, I would prefer to just extend it to words.
 
ISSUE! has always had a weird name.
If it has word status, it should probably become xxx-word.
 
@HostileFork But it's another example of Rebol foresight, today no-one would call it anything but "hash-tag", unfortunately.
 
hashtag-word!
 
8:03 PM
Both "hash" and "tag" rank in the top 5 erroneously re-used words in computing.
 
Well...WORD, SET-WORD, GET-WORD, LIT-WORD are pretty solid names. Pattern-wise it suggests a three letter name would be good.
 
Still thinking ... base-words? core-words? name-words?
I like the last one, because only those 4 word types can actually be bound to anything ...
 
One thing about it is that it is inert.
dead-word!
 
Well, it's more correct to say that only bare words can be bound to anything, the lit- set- and get- forms actually alias to the bare form when they are looked up ...
Wait ... what?
Looks like you can GET any one of the 6 word types, they all alias to the bare form.
 
@MarkI I know that there's a very purposeful reason for SET allowing (for instance) set quote foo: whatever... it can make a difference if you want FOO: to get picked up in the locals gathering.
 
8:13 PM
@HostileFork Yes, a syntactic difference only though, SET only sets the bare (or all, equivalently) form.
But I have gotten totally distracted (again!) from my main point, I am discovering some weirdities about any-word equivalencies.
I'll come back (also again!) with some more comments, but possibly not today.
 
@MarkI I'll continue to encourage you to write and map things out...there's a real need as Red is forming to have it not wind up carrying forward randomness...
 
I think I am going to adopt, with your permission, the phrase "dead word types" to describe refinements and issues.
They evaluate to themselves. They can be passed to GET but not to SET. Weird!
At least, while "live" may not be the best term for them, the 4 other word types do not evaluate to themselves.
But that may be a bug to some people!
When looked up with FIND or SELECT, unless /case is present all 6 word types are aliases for each other.
 
@HostileFork OMG. Your work?
 
@MarkI Also, if you missed the hilariousness that is "similarity"...I actually think that it's eerily useful, once you take all the junk out of the basic equals and such
@MarkI Yup, was part of Ren Garden, now exported into a separate proposals place. Feel free to send pull requests. It's all about simulated behavior.
 
8:31 PM
But, when looked up via a path, bare words appear to be aliases of all 6 others, but the other 5 only match themselves. Truly bizarre!
Is that covered in your opus @HostileFork?
 
@MarkI If one evening's work that I see I didn't finish a thought in the leading comment before pushing counts as an opus... well, no. It's my attempt to start coming to my own conclusions about stuff people have been thinking about before.
But I think there really are some important points in general when you're thinking about expression...such as the inflection of different? vs sticking in a not-equal? that means the same as not equal?. I made the analogy of how UNLESS would be far less interesting if it were called IF-NOT
 
Well, to get back to my main point for a second, I am going to disallow commas in all word types.
What I want is for there to be a community (not just a HF) freak-out, and a demand for me to change it back.
Odds are slim :)
 
@MarkI There has been support for comma and period equivalence when not using strict comparison that then differentiates when /STRICT. I don't recall the exact list of supporters on it...I think @rgchris favored it.
P.S. @rgchris After several rounds of experimentation, if you didn't catch it, I think I'm caving to the idea of for-each to go along with map-each.
 
Hello!
 
And remove-each, and really any other -each.
 
8:47 PM
Long time... How are things going, in this place?
 
@pierre Hello... long time. You've missed a few things. :-) Personal plug if you missed this in particular... RenCpp + Ren Garden, Ren Garden Value Explorer, etc.
@pierre Same as usual. Presumably you saw about the Red investment $.‌​..
 
@HostileFork Did I express an opinion on that one? I kind of like EVERY since you suggested it :)
 
@rgchris In terms of meaning shades, something that bugged me about EVERY was that it could just as easily mean "every position in series" as it could mean "every item". I think the for-each and remove-each and map-each "family" plugs better, because of the whole 99cents ea. kind of thinking. People know that each is one atomic item.
Being unable to really think of better names for map-each and remove-each, I came to think that EACH is the only answer but then it's missing a prefix. So I came back to either EACH or FOR-EACH.
 
@HostileFork Fair enough. It is strange given I've spent the last 10-15 years using FOREACH contemplating life with the extra hyphen. Not a bad strange—if it has to happen, it has to happen. Strange nonetheless.
 
@rgchris I understand u man, fo 'reals.
 
8:57 PM
It's such a key part of the language that I do start to spell 'for' as 'foreach' when composing emails/writing/etc.
Reflex.
From that point of view, it's probably for my betterment that it change.
 
@rgchris Hehe, just be grateful "-" doesn't need the shift key :)
It's still awkwardly far away though, and a pinky key.
Luckily, there's always R2/Backward.
@HostileFork It is certainly not a Unicode thing - again, unfortunately. There are lots of things I wish they'd have support for "equivalencing" out of the box.
 
9:36 PM
@MarkI I use my ring finger for hyphen
 
Well, I've been under rocks for a while... @fork, I've juste watched your video about Ren Garden: it seems son cool!
I'll Come back tomorrow using a full-size computer, abd not this small téléphone. Good night!
(soo cool, I meant)
 

« first day (1614 days earlier)      last day (2166 days later) »