Conversation started Aug 2, 2016 at 10:46.
Ell
Ell
Aug 2, 2016 10:46
Also man understanding streambufs is taking me a while :V
Probably because the entire streams library is fucked up. :P
Ell
Ell
Too many ptrs for me :P
also due to unhelpful names
egptr
xsputn
egg_pointer
execute_sputnik
Ben
Ben
Aug 2, 2016 10:53
->
@ThePhD exempli_gratia_ptr
Ell
Ell
Yeah and I'm confused about putback in the get area
@Ell it's horrible.
You don't really want to understand it.
@ThePhD Excess Putin.
Ell
Ell
@Griwes I do :D
Aug 2, 2016 10:57
cppref should have, like, explanatory pictures
@Ell Trust me, you don't.
@ThePhD It does.
Ell
Ell
Do you know about it?
Ben
Ben
What's unsafe and pointy?
a rapier
Ben
Ben
Aug 2, 2016 10:57
a pointer.
@Ell did you deal with xalloc and the related nonsense yet?
hehehehehehehehh xalloc/pword/iword
(That's for streams and not buffers, but same nonsense. :P)
@milleniumbug yeah
the most absurd part of the library, probably
maybe except some of the locale nonsense
The locale stuff is a bit rough to deal with.
And it doesn't have good defined behavior among the platforms. =/
I remember trying to use put_money and shit successfully.
Like, there's probably three people in the world who deeply understand and always remember how to use xalloc right.
Aug 2, 2016 10:59
Holy fuck, did that NOT go well in teh slightest.
@Griwes One of them being Dietmar Kuhl? :B
Ell
Ell
Egh why does putback even exist? :P
@Griwes nop
@ThePhD I'm almost sure he counts as two of them. :D
@ThePhD Meanwhile we're finally getting locale-agnostic string/number conversion functions.
@Ell Because streams don't have seek().
And because people didn't know better back then. :D
@Morwenn Fiiiiinally.
Aug 2, 2016 11:01
It took ~34 years, not even taking C into account.
Even though they do have a C-ish interface.
> After constructing and checking the sentry object, if rdbuf() is not null, calls rdbuf()->sputbackc(ch), which calls rdbuf()->pbackfail(ch) if ch does not equal the most recently extracted character.
ahahahahahhahahhahahhahhh
I forgot how hilariously bad this was.
That smacks of a lot of weird internal state.
People have worked on defining better stream interfaces already, right?
They have worked, and they have failed probably because they failed at properly separating different layers.
I mean, for me I know it's useful that std::ostream& can point to anything from memory to a file to console out, but...
Like, the actual I/O layer should not have anything to do with anything else.
Literally any I/O on any not ancient OS currently just mmaps some memory.
Aug 2, 2016 11:03
@ThePhD Rightfold has :p
There's no reason to limit that shit to just InputIterators. ForwardIterators (is that the right one? I can never remember) are 100% fine.
And suddenly a lot of problems disappear.
I would imagine some people would like it if their iterators for IO streams could be anything from random access to bidi to forward, depending on where they got it from.
Ell
Ell
@Griwes inputiteratirs are the correct approach imo
What if you have a stream giving you network data?
Eh, we might get 130L of beer for sponsorship for our next album :D
And from what I've seen, the album illustrations will be gorgeous.
sounds great
Ben
Ben
Aug 2, 2016 11:12
@Morwenn what's your band called?
Dur Dabla
Although who donates 130L beer to a band? Not your local pub, isn't it?
@Telkitty I hope it tastes great too :D
@Telkitty A local brewery of course.
@Ell The network data lands in memory either way!
You album would sound great after 130L of beer :D
Aug 2, 2016 11:15
It's going to be literally mmap'd into your process' memory in the end for sane implementations of things.
@Griwes Can't stick around forever.
Forward iterators would require that.
@R.MartinhoFernandes You can control for how long it stays though.
Not with enough granularity, no.
Ben
Ben
why are you guys talking about stream buffers?
@Telkitty Just like our concerts do.
Aug 2, 2016 11:16
If you hold an iterator for the start, you keep the entire stream, even if you might need only the first few items, or a few random ones.
Ell
Ell
@Griwes the control comes from using inputiterators and controlling the output yourself
Just grabbing what you want explicitly seems better.
A network stream is not replayable so no forward iterators. It can be made replayable, but then that functionality should probably go as generic iterator wrappers since it can be applicable to any similar construct.
Ben
Ben
@Morwenn do you have a picture of yourself with the rest of the band? :)
So in the end what we need to do to parse a file sensibly is to open the file, read the entire contents into a string (which first mmaps the file, which allocates memory and virtual address space, and then copy all that into malloc'd buffer, which allocates memory and virtual address space again) instead of just using the (useless) iterators.
@Ben There are a few ones around, but I'm too lazy to look for them.
Aug 2, 2016 11:18
@R.MartinhoFernandes Network stream is not replayable on the wire.
Once it's in memory it's in memory :/
The entire stream?
The parts the program has access to.
@Griwes Which, if you hold an iterator to the start, is the whole stream.
A programmer holding an iterator to a part of it basically says "I need this from this point onwards".
That's not enough granularity.
Aug 2, 2016 11:20
This is not a technical limitation, it's just a problem of teaching the programmers "don't do this if you don't actually need it".
You're asking for half-assed functionality that can be better provided elsewhere.
@R.MartinhoFernandes If you need just a part you need to copy it anyway; nothing changes.
@R.MartinhoFernandes I'm asking for a memory-management-level feature.
I have no way to get a forward iterator into the mmap'd buffer of an open file.
@Griwes Isn't that what the streambuf abstraction is meant to help with?
@R.MartinhoFernandes istreambuf_iterator is not a forward iterator.
@Griwes Again, because in general it can't.
A more specialized one could.
Aug 2, 2016 11:22
In general it can.
A filebuf_iterator could.
@Griwes why not mmap it and create that iterator from the pointer ;)
@ratchetfreak Terrible.
@Griwes In the same way it can also provide bidi and random-access. Where do you draw the line?
@R.MartinhoFernandes I already did, on forward iterators (because for the I/O purposes of parsing that's enough, and that's what you usually do).
Single-pass input iterators are pretty useless for that, because you literally can't peek() using an input iterator.
Aug 2, 2016 11:26
2 mins ago, by R. Martinho Fernandes
A filebuf_iterator could.
you don't need forward iterators for parsing
(It doesn't have to be specific to filebuf; just more slightly more specialized than streambuf)
you'd just need a "multiple-dereference" guarantee
@milleniumbug That's a forward iterator.
which is only a part of the forward iterator concept
yeah, but it also specifies that other iterators are valid after ++it
Aug 2, 2016 11:28
@milleniumbug How useful is it without that?
auto&& ref = *it;
I guess you can deref and pass it around again.
anyway, forward iterators are too strong, input iterators are too weak
sup lounge
@Griwes What you need is just fowardbuf and forwardbuf_iterator.
@Rerito Wow, hey :D
It's been weeks since I've last seen you around.
streambufs are still horrible.
Aug 2, 2016 11:34
doesn't peek just require that you can read the current position multiple times?
@R.MartinhoFernandes Which I argue should be the primary one. Eh.
Discording mainly @Morwenn
dumb_useless_inputbuf and dumb_useless_inputbuf_iterator would be good fits for the weird non-default thing.
@Griwes Fair enough. It just feels completely wrong to forbid completely transient buffers.
Or bufferless (though I'm not confident the stream library currently supports that)
@R.MartinhoFernandes Frankly I don't see the network stream example as convincing.
@R.MartinhoFernandes Bufferless buffers? :D
(Sorry, had to. :P)
Aug 2, 2016 11:39
@Griwes I'm thinking about the implementation of this. Wouldn't it require some messy bookkeeping?
The buffer has to track all iterator copies somehow.
Am I missing something?
It'd need buffer block refcounts.
It'd behave basically like a segmented iterator, where each segments has a control node that keeps a shared pointer to a buffer segment and to the next control node.
wouldn't a stray iterator copy being held somewhere hold up cleanup for a long time
So in the end @Morwenn, did you find a cool jerb near Brest?
@ratchetfreak It would, so don't do that. :P
@Griwes Basically, I don't disagree with this, but I think forcing everything to be forward is too much.
Aug 2, 2016 11:44
@R.MartinhoFernandes I feel like the root of this problem is that the streams library is virtually (lol) the only dynamically polymorphic piece of the standard library. :P
But the whole streambuf thing as is makes some things really hard to do.
@Griwes Yeah.
So we agree on the principle of all of this. Good. :)
@Griwes Though if you go and mimic the iterator concepts in streams, I question the stream/iterator distinction (at least as the idea of "stream" as it currently stands in the library) in the first place.
I always feel like the stream library has too much overlap with the iterator library.
(Well, with way too many libraries. Locale et al. But not something I want to focus on)
IMO it should just be raw I/O and perhaps some basic buffer management facilities.
Agreed.
Though when you have a "smart" iterator, I'm not sure what else you really need, because it already works as a buffer, too.
@Rerito Er yeah, like 4 months ago already :p
Python (and JavaScript) job in oceanography.
Aug 2, 2016 11:56
Maybe open_file() should give you a thing that allows you to "seek" in a way that seek(n) gives you an iterator? Hmm.
@Griwes Sometimes you want the buffer directly.
To pass to external code, for example.
I wonder if this could be extended to bidirectional iterators for the file case (I know it could not sanely for network streams, but for files?).
Sounds like more logic, but I think the amount of logic stays about the same.
Hmm.
Or even random access (though that'd be much harder on bookkeeping, since you'd have gaps in your segmented iterator thing).
auto file = open("asdf"); auto it = file.seek(1024); ++it; peek(it); is fully reasonable IMO; not sure if giving the option to do it += 10000; would make sense (especially since you already have seek).
@Griwes But you might want to do relative seeks from iterators without having the file object.
The fun part is that for systems that actually fully mmap this thing, the bookkeeping would be much simpler.
@Griwes If you want you can avoid the extra logic with two read modes, one where you can only read forward (call it "streaming") and one where you can read back too (name?)
Aug 2, 2016 12:02
I wonder if current standard libraries on POSIX mmap files in their entirety, or just segment-by-segment.
@R.MartinhoFernandes The logic's not really that different, so I'm not sure if it actually is more logic (or at least, not much more).
> sorry, unimplemented: mangling binary_left_fold_expr
void Testf( U );
...and now I'm grepping glibc for things.
what am I doing with my life
Hey guys I've got a problem. I have an implementation file and a header file with declarations. In the declaration file I have a template function, example:

template < class U >
#define _IO_XSGETN(FP, DATA, N) JUMP2 (__xsgetn, FP, DATA, N)
FUCK YOU GLIBC
I JUST WANT TO KNOW HOW YOU FREAD
Aug 2, 2016 12:08
In the impl file I have the definition:

template < class U >
void Testf( U ) { }

And in my msain file, i call Testf( ), and it does compile, but fails to link.
Helperino pls!
Deep-fread chicken.
I'm confused.
@EnnMichael rtfm
/* FIXME handle putback buffer here! */
Ugh.
Aug 2, 2016 12:12
put stack
820
Q: Why can templates only be implemented in the header file?

MainIDQuote from The C++ standard library: a tutorial and handbook: The only portable way of using templates at the moment is to implement them in header files by using inline functions. Why is this? (Clarification: header files are not the only portable solution. But they are the most convenien...

@Griwes lol, is that an obsolete comment or does the code not really handle it?
@Morwenn IFREMER?
@Rerito Nope, CLS. I worked for IFREMER 3 years ago. But it's like ~500m away from IFREMER.
Oh just next to my former school
Yup, just next to my former school too.
Aug 2, 2016 12:19
enib?
@R.MartinhoFernandes also lol go review some of the others FIXME's there. :D
@Griwes Not too many FIXME's, but some of them look like they'd affect some corner cases.
And one of them is just "well we don't really check for safety with NULL".
The GNU C Library is free software;
right, no wonder the code is so 'pretty'
I give up trying to figure out if or where glibc calls mmap in the context of file IO. :/
@Griwes Is the dynamic polymorphism bit really so bad? After all, there's a lot of code that handles the base classes of std:: streams and then uses those.
Aug 2, 2016 12:27
it's a C library ...
@ThePhD Yes - it stops you from having different iterator categories for different streambufs and shit.
Granted, they probably do that because it's the only way right now, but I would imagine a lot of developers are thankful they don't have to nail their choice down to a specific type of stream (see the FILE* fiasco of many C APIs which for some reason only take a FILE* file pointer or a const char* fname c-string, and usually have to be hacked up later to deal with a section of memory or otherwise).
> First, and not limited to concepts, we need to treat a member template as dependent if its signature depends on template parameters of its enclosing class […]
really? that deserves looking into
@Rerito Yes :)
I had some classes in Télécom Bretagne too.
image processing?
Aug 2, 2016 12:35
correction: this deserves looking into by someone else
Sob.
Not even 3 days and I already thought of another feature sol2 needs.
Someone save me.
@R.MartinhoFernandes Now what about the other direction? How do you do writes? And more importantly, how do you connect that with async IO? Or is the async IO layer just the layer that the iterators build upon?
Euuugh.
*it = (something); gross
that time of the month special!
coz anything could look like a panda
Aug 2, 2016 12:42
I guess for "synchronous" writes you could make it possible to write through those iterators, and that's okay, but async IO needs to be an actual thing (though it probably should be the layer below).
Would need a way to flush all the changes done through iterators.
@ThePhD how is it any more gross than something = *it;?
@Griwes The way IO works right now, when you read something, it generally moves ahead.
I know that.
And I also dislike the way IO works right now. :P
And when you write something, it also moves ahead.
So iterators are kinda borked because they generally point at a specific position.
Because I consider the way IO works right now to be ancient.
44 mins ago, by Griwes
auto file = open("asdf"); auto it = file.seek(1024); ++it; peek(it); is fully reasonable IMO; not sure if giving the option to do it += 10000; would make sense (especially since you already have seek).
There's a bit more of discussion and random attempts at online design in the discussion above.
If you could write an IO layer beneath that that allowed you not to be forced to move forward or backwards when doing a read or write that'd be the first part of making iterators more viable, right?
Aug 2, 2016 12:45
1 hour ago, by Griwes
It'd behave basically like a segmented iterator, where each segments has a control node that keeps a shared pointer to a buffer segment and to the next control node.
These are the two messages most important for understanding what I'm getting at, I think.
Oh. Well, okay.
So, at least to maintain a bidi iterator,
you'd need to have a buffer for your current stuff, a buffer for stuff behind you, and a buffer for stuff in front.
Yes.
If you wanted you could go further with random access, but then bookkeeping goes nuts.
Yeeah... true random access is really hard. I think bidi is the most sane you can get?
I think so too.
I'm going to once again throw myself an interrupt from current personal project on the table and try to implement something like this.
(...despite the last one being thrown like on Sunday. :D)
@Rerito Nope. It was the the research master degree. I don't remember which classes exactly but it was boring and I was bad.
Ell
Ell
Aug 2, 2016 12:49
I still don't see the rationale of putback
It's not even guaranteed or anything
One could just not support it on a stream
@Ell You want to parse stuff.
@Griwes What's the current one?
Your memory is limited, so you can't just read the entire file.
And your API is so braindead it doesn't have peek().
That's when you need to start putting things back.
Ell
Ell
I mean, putback shouldn't be in the stream :P
Obviously.
Ell
Ell
Aug 2, 2016 12:51
But even then they would have known that
@Morwenn Oh! T'as joué de la flute alors ? :D
Ell
Ell
I think there I something I'm missing
IMHO it's just one of the signs that the API's foundations are braindead and/or ancient.
@Rerito Belle métaphore, mais non. On n'était pas obligés de valider toutes les UEs x)
Du coup j'ai eu le master de recherche à la moyenne générale.
Avec 2.5/20 et 4/20 aux cours de Télécom.
@ThePhD Vapor, sitting on top of an UEFI bootloader and a brand new greenfield kernel following it, sitting on top of a CPU toyed with in Verilog (where I'm currently mostly fighting with the language), sitting on top of Despayre (which's my build system thingy), sitting on top of constant improvements to my general purpose libraries.
Ell
Ell
Aug 2, 2016 12:52
I'm writing an LZMA2 decompression stream BTW
I might've forgotten something. Sorry.
@Griwes That's quite the stack.
@Ell Well, if you've never seen code that thinks that getc (+ ungetc after failed peek...) is a right interface to use when parsing text, then you'll keep missing it.
If you've seen such code, you are probably hating it and hoping to forget it ever existed one day.
@ThePhD Yep.
Hahaha nice
My current stack is {SCHOOL WORK HAHA FUCK YOU}, chat wubsite, sol2, gladell
Aug 2, 2016 12:59
@Griwes Not sure. I'd need more time to think about it than I can currently dispense with.
I can't imagine asynchronous iterators.
@ThePhD Wow, that's quite shallow.
@R.MartinhoFernandes I'm leaving the bottommost pits of the stack out...
@R.MartinhoFernandes Fair enough.
But do ping any time if you have a random idea about it.
 
Conversation ended Aug 2, 2016 at 13:00.