« first day (630 days earlier)      last day (4318 days later) » 

11:03 PM
Thanks for the answers, guys. I am trying to conceptualize how iterators are implemented.
 
Ell
pointers in a vector aren't they?
argh god not being able to get comfy is incredibly frustrating!
 
@Ell Not necessarily. They can be pretty much anything as long as they conform to the iterator requirements
 
Ell
well I know its not specified but why would you use something else?
 
@Ell You can use more complex structures for debugging in release mode.
 
@Ell Debug versions of the iterators.
 
11:06 PM
What does the C++ standard say about passing standard containers as function arguments? For instance, if a function takes std::vector argument by copy, does the function get a deep copy of the vector?
 
@Specksynder Yes. If you copy a standard container like vector you copy everything inside it too.
 
Ell
oh yeah. I'm a douche for not thinking of that
 
@Specksynder Yes. They are explicitly value typed- and CoW and such state-sharing implementations are explicitly forbidden.
however, taking by value != taking by copy
 
@DeadMG, I am confused by your last statement...
 
welcome to move semantics, biatch
 
11:08 PM
It's one of the reasons why move semantics was invented so something like std::vector< std::vector<T> > isn't horribly inefficient when resizing.
 
Ell
return value optimisation could optimize a copy out
oops ignore that
 
if you write void f(std::vector<double> x), there is no guarantee that x is actually a copy of anything
it could have been moved in
 
@DeadMG, ok, its implementation dependent?
 
no
 
Example of a sitation DeadMG is talking about: f(g()); where g() returns a std::vector<double>.
 
11:09 PM
move semantics are very Standard.
by necessity
 
Ell
move semantics seem elementary now. its difficult to see why they weren't added earlier
what other languages have move semantics?
 
@Ell Do you have a time machine? :-P
 
@Ell None, AFAIK.
 
@Ell AFAIK, none, since C++ treats objects like values.
 
most are simply enforced-reference
 
11:10 PM
Thanks guys!
 
Ell
I wonder what other semantics will come in the future :L
 
Other languages like Java treat almost everything as references.
 
Ell
yeah
I prefer c++'s way of thinking by far
 
eh
it's pretty unlikely that another kind of semantic will be introduced
 
Ell
just wish there was a language with c++'s semantics that wasn't so low level
 
11:11 PM
In C++, user-defined objects and primitives like int are treated very uniformly for the most part.
 
since we already have reference and value (move and copy), there's none more that I know of to implement
@Ell C++ is as low level as you make it.
 
@Ell C++ let's you mix high-level and low-level as much as you want.
 
Ell
I suppose
 
what's really pissant about C++ is the implementation awfuls
headers, declarations, that BS
 
Ell
oh also compilation model
 
11:12 PM
It's just that if you want to do high-level, you won't have to pay for a drop in performance
(if you do it right of course)
 
Ell
I just don't like the standard library
also can someone explain what a "stream" is?
 
@Ell How so? It's not spectacular but the levels of WTFness are relatively low.
 
Ell
is it a tool through which we read and write bytes
 
Ell
or does it represent those bytes?
 
11:14 PM
@Ell A stream, basically, is the concept of data that you don't have yet.
 
It's just something that you can push data in/out of.
 
it represents some data you have, and some data you might have in the future
 
Ell
I would just like a wider standard library, filesystem and sockets and the like
right okay, so can you get the size of a stream?
 
no
 
@Ell Yeah, but do remember that C++ is designed to be completely agnostic to the machine.
 
11:15 PM
how can any implementation possibly know how much data you might have in the future?
 
The Standard doesn't even talk about a "computer", but rather an "abstract machine"
 
"Get me the size of all data the user will enter into the console in the future."
 
Ell
I thought so, I'm just getting confused with streams and buffers
 
yeah
 
And for those that can tell the size, streams are not the proper place to put a get_size() method anyway.
 
Ell
11:16 PM
so a streambuffer is what?
 
because the Standard streams suck, horrifically, in every conceivable way.
@Ell Nothing more than a cache to improve efficiency.
 
Ell
so is a stream buffer a buffer to a stream? so the stream only writes in chunks instead of one byte per time or whatever?
 
depends on the stream
it's really a completely implementation detail and should never, ever have been exposed to the caller
 
Ell
hmm okay
 
@Ell It's an implementation detail, althought almost all implementations write out everything in chunks
 
11:18 PM
unless you're implementing a stream, buffering is not your problem (except flushing it, of course, in some niche cases).
 
Writing one byte at a time through the OS is slow as hell
 
Ell
and how about I want a stream to an std::vector<char>? (aka to a buffer of bytes in memory)
 
std::stringstream.
 
Ell
how would I do thst?
right
 
honestly, the Standard streams are super terrible, I wouldn't take them as advice on how to do anything
 
Ell
11:19 PM
that's what irks me in c++ as well, I think there should be a byte type
 
@Ell Well, really, char is two or three problems in one.
first, all the primitive integer types suck cock
 
@Ell typedef char byte;
 
two, all the string support sucks cock
 
Ell
@insilico how bad would it be if I started using that?
 
@Ell Absolutely fine.
 
11:21 PM
@Ell It would be better because it would be explicit
"This is a stream of data, not characters"
 
the Standard defines that sizeof(char) is 1, and that effectively, no C++ program can do sub-char addressing or arithmetic or storage.
 
Ell
I might do that then
 
so, whilst technically, you can say that char might be UCS-2 or UTF-32 or something strange
 
The problem is that in Unicode char is damn near useless for storing code points.
 
in reality, it's pretty much always the 8bit byte
and there exist no significant hardware platforms where you cannot use char as a byte.
 
Ell
11:23 PM
I suggested once somebody should make a Character<ASCII> and character<utf-8> type etc.
 
nah
 
@Ell Not really.
 
Ell
what would the problems of that be?
 
well, mostly, it's a complete waste of time
the Standard needs to make a few minimal guarantees and then be on it's way
 
Ell
why? then a string is just a list of those
 
11:24 PM
right
 
@Ell First of all, all UTF-xx encodings are variable-width.
 
but it isn't just a list of those.
you can't just get an arbitrary sequence of UTF-8 bytes and call it a UTF-8 string, because it isn't.
 
Ell
but you always know what encoding something is in?
 
no
 
There isn't a 1-1 correspondence to the glyphs the user sees on the screen and the number of code points encoding that glyph.
 
11:25 PM
you only know what encoding it is in if you have verified that it is actually valid for that encoding
 
Ell
then how can you possibly decode it if you know not what encoding its in?
 
you can't.
that's why Windows Codepages and other things were such a complete disaster.
also, your Character<UTF8> dealie would completely not help anything, at all
 
Ell
I think I know too little on the subject to say anything really
 
And on Windows it's doubly disasterous because Unicode really was a 16-bit fixed-width encoding some years ago.
So the Unicode functions on Windows was written with this assumption.
 
who says you can't juggle these all at once... it's not like you have to jump through hoops:
 
11:27 PM
I don't think it's especially disastrous for Windows
they updated all their Unicode functions to handle surrogate pairs just fine
it's the user applications which have not been updated correctly
 
@DeadMG That's why it's doubly disasterous.
 
the primary problem is that Windows does not really support UTF-8, and Unix does not really support UTF-16
 
The OS is useless without the applications.
 
so it's hard to pick an encoding that works for you across all plats
@Insilico Most of which are fine by now. Not to mention that surrogate pairs form a massive minority of all existing UTF-16 text.
 
@DeadMG True, but not enough understand Unicode IMHO.
 
Ell
11:29 PM
how about a character class which can hold a character of any encoding
 
@DeadMG Which is why I think string libraries should be agnostic to the UTF encoding.
 
@Ell Definitely no.
 
Ell
okay :L
 
@Insilico I agree. The internal encoding should be abstracted.
the problem with that is that you can't abstract the input encoding as string literals
 
@DeadMG True, which is why I've settled on using UTF-8 as the encoding for string literals.
Because then you don't have to use the ugly-ass L prefix.
 
11:30 PM
be prepared to bite VS's nasty habit of converting source files to Extended ASCII
 
@DeadMG Just make sure you save the source file as UTF-8 without BOM and the VC++ compiler won't touch the literals.
 
Ell
I still don't understand where all the trouble OS - you make every string have an associated encoding member, then treat it as such when using it
 
@Ell ...
 
Ell
sorry Ignore me
 
as I already said, because you can't just stitch together a bunch of bytes and call it UTF-8
 
Ell
11:32 PM
I clearly need to do some reasearch :L
 
their encodings are variable-length, multi-byte.
 
Ell
I don't understand why though
 
you could only convert it from Unicode codepoint to UTF-8.
 
Ell
yesh but
 
@Ell Let's introduce some terminology. A "glyph" is what the user sees on the screen.
Each glyph can be encoded by one or more "code points".
 
Ell
11:33 PM
right kk
code points?
 
Each code point can be encoded by one or more "code units".
 
a Unicode code point is basically the end numerical value.
so you have a Unicode codepoint for stuff like e, 9, ™, etc.
 
Ell
code unit?
 
And PILE OF POO.
 
the problem is that you also have a Unicode codepoint for things like, accent
so you can have U+something "e", and U+something accent, to produce one glyph- e with accent.
 
Ell
11:34 PM
so what would the string of bytes be to represent "a"
 
@Ell See, a code point can take up to 21 bits to encode.
 
@Ell Depends on the encoding.
 
You obviously can't fit a code point into a byte.
 
U̸̢̙̟̘͊ͥ̊̎͢Ň̴̟̪͖̟ͪ̈́̑͜Ȉ͖̱̼̼͊̎ͯ͝C̥̠̻̳̺̺͙̊̈̈͟ͅŎ̸͑̔̀̚҉̣̙̹͇͚̯Ď͈̠͔̟̦̎ͤ̈̒ͤ̇̌ͫ͟͠E̔͏̷‌​̪̥̭͍
 
That's where the UTF-xx encodings come int.
 
Ell
11:35 PM
with utf-8
 
sbi
> Ex-Nokia staff to build MeeGo-based smartphones — theverge.com/2012/7/7/3143099/…
 
although Unicode's first 127 members are the ASCII plane and all Unicode encodings encode ASCII as, well, the raw values.
 
Bit of fail.
 
So in UTF-8 each code units is 8 bits.
In UTF-16 each code unit is 16 bits.
 
@Ell Then the same as in ASCII.
 
11:35 PM
and UTF-32 each code unit is 32 bits.
There are other Unicode formats but they are practically irrelevant.
 
in UTF32 32bits is obviously way more than a codepoint needs
so here you have 1:1 correspondence between code units, and code points.
 
Ell
right okay, so it would consist of one code point consisting of one code unit which was the code unit of a?
 
but for UTF16 and UTF8, that's not true.
 
In UTF-16 code points may be represented by 1 or 2 code units.
 
@Ell Yes.
 
11:37 PM
In UTF-8 code points may be represented by 1 to 4 code units.
 
@Insilico Unless you work for Microsoft. :v
 
depending on the final value of the code point, the number of code units and their binary values changes.
 
(remember that in UTF-16 each code unit is 16 bits and in UTF-8 each code unit is 8 bits)
 
if you are encoding an ASCII codepoint, then UTF8 == ASCII.
 
Ell
so for æ
 
11:38 PM
but above that, it becomes more complex
@Ell Isn't that in Extended ASCII?
 
Ell
it would be how many code points? assuming utf-8
 
BMP non-ASCII characters are up to two bytes in UTF-8.
 
@DeadMG it's in Latin 1, yes
 
@Ell 1 codepoint. I think.
 
@Ell That's one character, so 1 codepoint.
 
11:39 PM
but for any given glyph there is no way to know.
 
@CatPlusPlus a bit more... the BMP is 64K
 
hmm can a theme in windows 7 not change the taskbar appearance?
 
(BMP = Basic Multilingual Plane)
 
Ell
so would it be one code point consisting of a single code unit?
 
some Vietnamese glyphs are like, 5-6 codepoints long.
 
sbi
11:39 PM
> Yo mumma is so fat, not only does she have to book two seats on the airplane — they have to be two windows seats!
 
all themes i have seen so far look the same wrt the taskbar. they just change the wallpaper and colors
 
@CheersandhthAlf No, BMP goes up to 0xFFFF.
 
@CatPlusPlus that's 64K, my cat
 
@JohannesSchaublitb I'm sure you can change it. (The color of the taskbar, I think)
 
@CheersandhthAlf Either way, up to two bytes.
Don't make me think.
 
11:40 PM
would like to change the form of it
 
@Ell For a glyph in the ASCII plane.
but Extended ASCII is not in the ASCII plane.
 
Or maybe 3.
 
@CatPlusPlus necessarily two bytes and necessarily more as UTF-8, because quite a bit goes to redundancy
 
if æ is in the BMP then it would be 1 codeunit in UTF-16
but .. up to 3 in UTF-8, I think.
 
Ell
so let's assume it is not in the BMP
 
11:41 PM
Wiki describes 6-byte UTF-8, and I'm too lazy to binge through Unicode standard.
 
@DeadMG Yes, it's up to 3.
 
2 codeunits UTF-16, up to 4 bytes UTF-8
 
So whatever.
 
@CatPlusPlus Capped at 4 bytes.
 
@DeadMG I know, but it wasn't.
 
11:42 PM
the UTF-8 encoding scheme can in theory be extended up to 6 bytes but Unicode explicitly caps it at 4 bytes.
 
(Although UTF-8 decoders handle 6-byte sequences anyway to deal with malformed UTF-8 strings)
 
@Ell The trick here is that the first byte of those several UTF-8 byte carries more than a few bits.
it has a special binary format and indicates how many more bytes are to go.
 
UTF-8 (UCS Transformation Format8-bit) is a variable-width encoding that can represent every character in the Unicode character set. It was designed for backward compatibility with ASCII and to avoid the complications of endianness and byte order marks in UTF-16 and UTF-32. UTF-8 has become the dominant character encoding for the World-Wide Web, accounting for more than half of all Web pages. The Internet Engineering Task Force (IETF) requires all Internet protocols to identify the encoding used for character data, and the supported character encodings must include UTF-8. The Internet Ma...
 
and UTF-16 codepoints which are surrogates also have special binary forms.
that's why you can't just go mucking around swapping them around for lolsies
 
There, it even includes a table about what each byte means.
 
11:44 PM
MSB in first byte will always be 0 for ASCII characters.
So Latin1 crap is already two bytes.
 
Ell
right so the first byte of a utf-8 glyph has a special format saying how many bytes follow?
 
codepoint, not glyph
each glyph can be several codepoints.
 
@Insilico Shouldn't that be for(typename std::vector<T>::iterator i...?
 
Every byte has a special format.
 
@FredOverflow (assume T is a known type)
 
11:45 PM
in Western scripts they are almost always single codepoints (after normalization) but for arbitrary glyphs they can be more- sometimes a lot more.
 
Ell
oh kk codepoint
 
and rendering them is a complex topic
that is really best never known about if you can avoid it
 
@Ell But, effectively, yes.
 
Ell
so where is it encoded how many codepoints there are in the glyph?
 
@Insilico Ah, okay. Whenever I see <T> I'm thinking template parameter.
 
^ How to win a TV debate -- with a gun!
 
Glyphs are graphical representation.
 
@Ell The Unicode Standard defines an algorithm for glyph iteration, I believe.
or mb that's just extended grapheme cluster
in any case, you as a string class implementer do not really need to care about glyphs
they are mostly dealt with by the text rendering APIs
 
All combining characters after a normal character go to that character, and that's about it, I think
 
11:47 PM
@Ell Technically you can have as many code points representing a glyph as you want.
 
ZALGO
 
That's why ZALGO is possible.
 
upshot is that a character is a string
 
and a couple support algorithms if you're writing a Unicode UI
 
And a couple of goats.
 
Ell
11:49 PM
are glyph and character the same thing? apart from one is the graphical aspect and one is the conceptual
 
yeah
 
@Ell Just avoid the use of the world "character" and you'll be fine.
 
You already answered your own question.
 
Ell
aka, is what describes a glyph also what describes a character?
 
but now you can see why there is a problem when using the word "character"
the user thinks of a character as a glyph
Unicode-aware APIs tend to treat a character as a codepoint
 
Ell
11:50 PM
right
 
and a bunch of morons treat a character as a code unit
 
Ell
but surely that is incorrect?
 
Character is codepoint and associated properties.
 
@Ell Indeed it is.
 
Ell
I mean treating a character as a code point?
 
11:51 PM
No, why would it be?
 
Ell
because a Character is composed of one or more code points?
 
depends on your perspective
 
Ell
isn't it?
 
the user sees a different glyph
 
11:52 PM
@Ell Just remove the word "character" from your vocabulary when talking about Unicode.
 
but the properties of the character are identical
 
Glyph might be, but combining characters are well, characters.
 
well
it's best to simply never, ever talk about characters.
 
It's an abstract thing.
 
glyph, codepoint, code unit.
 
Ell
11:52 PM
right okay i won't
 
they exist, they're well-defined
 
Ell
hmm. if we took that a c-string is an array of characters, would the Unicode equivalent be an array of glyphs? or an array of codepoints?
 
array of code units
which satisfy the encoding- so for example, if you have an array of UTF-8 code units, they have the correct binary encodings and such.
 
@Ell Internally, an array of code-units.
Externally, ideally, the interface would be in terms of code points (or a string in some prescribed UTF encoding)
 
hmm the major reason for allowing "typename" outside of templates were macros right?
 
Ell
11:54 PM
hmm I don't understand that
 
@JohannesSchaublitb What about templates and macros?
 
arguably, you could say that as a C-string, they have glyph, codepoint, and code unit as all the same thing.
but the Unicode string is an array (or rope) of code units
 
C-string is a byte string.
You can look at it however you like.
Byte arrays have no intrinsic interpretation.
 
which are interpreted and manipulated by the string's users as an array (or rope) of code points, ideally, independent of which format the implementation chooses.
 
11:56 PM
That's why we do protocols and encodings and keep track of them and whatnot.
 
Ell
would an accent be a codepoint? let's say the two dots on ö be one codepoint and the o another? or would they be represented as two code units in one code point? can I have an example of where something would be split up into two code points?
 
In digital typography, combining characters are characters that are intended to modify other characters. The most common combining characters in the Latin script are the combining diacritical marks (including combining accents). Unicode also contains many precomposed characters, so that in many cases it is possible to use both combining diacritics and precomposed characters, at the user's or application's choice. This leads to a requirement to perform Unicode normalization before comparing two Unicode strings and to carefully design encoding converters to correctly map all of the valid wa...
 
@Ell It can actually be represented both ways.
normalization can convert from one to the other
 
I mean, you know that you are not within a template so why use "typename"?
 
Common combinations have their dedicated codepoints.
 
11:57 PM
but a macro author does not know where its code will be injected
 
some combinations however cannot be normalized in complex scripts, usually East Asian
 
For efficiency, most likely.
 
@JohannesSchaublitb If you're using macros containing C++ keywords in them, you might have a bigger problem. :-P
 
Ell
@deadmg so "normalization" is from two code units to two code points and vice versa, or two code units to a combined character single code unit?
 
11:58 PM
however
accents are the canonical example of a combining codepoint
 
@Ell Depends on the normalization algorithm.
 
Normalisation doesn't deal with code units.
 
Ell
normalisation deals with...?
 
@Ell Code points.
 
Code units don't exist outside encodings.
 
Ell
11:59 PM
right.
 
The only reason why code units exist is because native 21-bit integral types don't exist.
 
all Unicode algorithms operate on code points.
code units are just about which format you chose to store those codepoints in.
 

« first day (630 days earlier)      last day (4318 days later) »