@NickAlexeev Looking beyond the simplicity of the language's design, and its readability, mostly it'a good at string processing because strings are a built-in type with a rich set of methods and no need for garbage collection.
In C#, C and C++ strings all need much more careful memory management. In Python, memory lalocation and deallocation is automatic.
@CoolCloud No, this is the Python room. That's not even a question about programming (so not SO material), but about Operating Systems. Try SuperUser.com and Windows user sites.
@CoolCloud When joining a new chat room on Stack Overflow, please read the description at the top-right corner of the page. Generally, as in this case, it will contain either the room rules directly or a link to them. Here are the Python room rules. It's also good to just, you know, read the name of the room and assume what is probably welcome based on that.
@JosephSible-ReinstateMonica That professor is probably named "The Internet".
It's a common cargo cult trick. Look at it this way: at least they're trying to do some reading and improve their code! Even if they're doing it wrong.
Under the hood, it's going to be using mmap on POSIX systems (e.g., Linux, macOS), and MapViewOfFile on Windows.
So any background information you want to know that isn't covered in the Python documentation can be learned by reading about the OS-level functions that do the heavy lifting.
Just last night I explained some python to a C-strong good friend of mine. Needless to say he wasn't crazy about name binding behaviour, but he wasn't as difficult as some people
@AndrasDeak just let it happen :) That's usually what I tell c programmer. Don't try to understand it, just let python do it's magic. Let go of your need to know every little detail of how things work and focus on getting things done
Programming productivity is measured in WTFs/minute. When the language itself and/or its basic syntactical constructs are producing a significant number of WTFs/minute, that's a problem.
@Hakaishin Which leads to horrible programming practices. You don't write good, correct code by just throwing things at the wall and seeing what sticks. Even if you "prove" it runs in one circumstance, that doesn't prove it will work in others.
But, I'm with you up to a certain point. I very much like abstractions, and I don't think about low-level memory allocation or whatever every time I write code.
But if you don't actually understand what's happening, the moment something starts to break down or behave in an undesirable or unexpected way, you're SOL.
And that's an uncomfortable and undesirable situation to be in when you're a software developer. Or... any kind of... anything.
Obviously, I can and do work at multiple levels of abstraction. There are times when I'm probing signals with an oscilloscope, which is about as low level as you can reasonably get. I write assembly for fun (and read it sometimes when debugging complex problems or compiler issues.) But I obviously don't spend very much time working at these low levels of abstraction because one cannot be very productive there.
Totally, my point is just there are many things in python which I never had to learn how python does them, because they just never break down. I agree that you have to know the lower levels the leakier the abstraction, but python is a very good abstraction that nearly never breaks down :)
At that point no language can save you, right? I'm probably just missing your point because I don't have anything to do with these nooks and crannies of IT.
@CodyGray About twenty years ago there was a brief discussion about making Python source code case-insensitive, but even back then the weight of potential code-breakage and (worse) undetected bug injection to existing source code outweighed the desirabiliity of the change.
@CodyGray pip has some weird things. Like the pip search CLI which has been disabled for like a year because the backend it uses is under constant DoS. There's a searchbar on the pypi webpage that works...
@holdenweb Yes; unfortunately, that's the kind of thing you have to get right from the beginning. It's not something you can introduce as a breaking change. Too much breakage.
@AndrasDeak Hm, that's unfortunate. A good lesson in why we cannot have nice things, I suppose. Not that we needed any more lessons of that.
@holdenweb Adding a pre-parse step where you run the code through a formatter that converts everything to lowercase is hardly enough to merit the name "fork". :-)
Some people expect programming languages to be case-insensitive. After all, the first use of the word "after" in this sentence has the same meaning as all other uses.
String literals aren't source code, they're data :P
@holdenweb Ooh.. nitpick. No, it doesn't. The capitalization serves a semantic purpose: it indicates the word begins a new sentence.
@AndrasDeak That's true. If you needed to handle string literals, you would need to have a smarter parser. Or, create a requirement that string literals don't belong in source code files and require that strings always be defined as external resources. You know, for your own good. :-)
@holdenweb Do you have any guesstimates about the number of people expecting that? It would be really weird for me. And seeing all the questions on SO where no two variables are spelled the same, but they all refer to the same name... ugh
There are cases where the beginning of a sentence is not necessarily indicated by a period or other sentence-ending punctuation mark. Consider a sentence beginning inside of a quotation mark.
Now please don't mind me, this is very informative
@holdenweb Import hook + AST preprocessing should do, right? I imagine this actually could be doable in, say, a 100 line module. Give or take a few lines for cross-version AST compatibility.
@Hakaishin I... don't get this sentiment. Name handling in CPython is practically just pointers. C folks should have little problem understanding it. Seeing this sentiment repeated so often makes me wonder whether the learning material is just bad or where else the issue actually is.
@holdenweb It's tempting indeed. Is Py3.9 okay for a demonstrator?
i am wanting to check if a string `startswith` one among the string patterns predefined.. Preconditions: 1. the predefined string patterns are in the order of hundreds in quantity 2. the check operation is loaded into a very complex system, which calls the check function a few hundreds of times in under a second. Question: i feel like my implementation is kinda primal, so wanting a review
prelist = ["a","b","cc","dddd","fewhundredmore"]
def checklegality(word):
for element in prelist:
if word.startswith(element)
return True
return False
I wonder if something like github.com/qntm/greenery can construct a regex that matches a set of strings. I know it can do the opposite, but that's obviously easier.
If re.compile is smart enough to construct a trie that merges common prefixes of your prelist, then there are definitely scenarios where it will run faster.
A trivial example: prelist contains ten million unique instances of "abcdef" + randstring(), and also "foobar". Given the input "foobarbaz" A loop-and-check algorithm will require ten million and one comparisons. A trie will require two.
It looks like this is responsible for turning ab|ac|ad into a(b|c|d), but it only does it if all branches share the same prefix. Not suitable for our purposes.
I'm decently sure such transformations are NP'ish, and everything but a simple heuristic would immediately burn away all the speedup you could possibly get.
If any regex expression can go between the pipes, then yeah there are probably a bunch of catastrophically bad cases. But if only sequences of literal characters can be between pipes...
i am loving this discussion, because usually a debate is like when trying to find out whoās right & whoās wrong, whereas a discussion like this serves to establish what is right and at what use cases.
Timing results: str.startswith: 14 sec re module: [still compiling] regex module: [cancelled it after around 30 seconds] super trivial self-made state machine: [still running]
(Measured using two strings that start with 1 million "a"s and then end in either a "1" or a "2")
I have an idea for an approach that turns a trie into a super fast C function, but my knowledge of ctypes is 1% of what it needs to be for me to test it
I see the Steering Council "strongly suggests" that the upcoming 3.10 changes to enum should be reverted, and PEP-ped for 3.11. Another blow for backwards compatibility!
I wonder if memcmp(ptr1, ptr2, n) is O(1) when ptr1 == ptr2. It would be very inexpensive to do a quick referential equality check, but perhaps they don't bother because it would slow down the common case by a femtosecond.
@Kevin Maybe in the C code the method does an identity check with self. That would be worth doing, as the cost is typically amortised across many cycles of string comparison and it allows an immediate True return.
Plenty of "look, I reimplemented memcmp for my CS 102 class", not very many "look, here is how the Linux kernel implements memcmp on core i7 processors"
For the record, the CS 102 submission does not do an identity check
I'd expect kernel programmers to be more alert to the potential for optimising the comparison between a thing and itself. But that's just my foolish optimism.
I also expect that, but I also expect that sometimes their 300 IQ analysis results in "actually, it's not worth doing, for reasons Kevin cannot possibly comprehend"
"If you cross-reference the L1 cache against the speculative multithreader, you'll see that the Coriolis effect causes a destructive guru meditation error, and the kernel has to restore itself from tape backup"
And Steve closes the tape archives early on alternating Thursdays so he can get ready for his LARP night, so we can't do identity checks then
@Kevin Precisely. I'm a firm believer in respecting the black box interface of sufficiently reliable services (though I'll sometimes take them apart out of interest). I long ago lost the arrogance of believing my intuition would beat the efforts of people who'd actually done lots of hard work on a problem.
@Hakaishin It depends on what "flavor" we're talking about here tbh. There as many flavor of Lisp as there are star in the sky (I'm probably exaggerating, but not too far off)
the syntax may look the same, but most people won't make a clear, detailed comparison of the token, syntax, grammar etc and just paint them all with the same brush
for example, the most documented Lisp flavor would be Common Lisp.
If you know it, you technically should be fine learning other Scheme/Lisp stuff, but that's not always the case...
The syntax is probably fine even with the parentheses being overused, but that's probably a question of preferences.
@Kevin same. For me it's also because there so many question I have but most won't get answered right away...and when I learn something, I really want to "know" what's going on, even if I don't know the internal, at the very least, to know what X give Y as result, instead of "trying and failing"
Every language other than Lisp should just suck it up and write a better parser. But Lisp has the justification that the user needs to be able to define new syntactical elements for the language at runtime, and the more parentheses they have in the base syntax, the less painful it will be for the user
I don't know; I feel like most language have a very simple syntax for "printing" text like "hello world", and then, it just give users/learner the impression it's actually as easy as that, but it isn't most of the time
@Kevin at least we got auto pairing for parentheses these days...imagine back in the day when you had to remember if you put the right parentheses at the end or start before testing something you made
Not sure how exactly it works, but it's not like you can introduce any new syntax. Everything is essentially just a function call. So adding a "keyword" is equivalent to defining a new function.
@Hakaishin Probably it compromises on "unambiguous". There may be expressions that could theoretically compile to multiple different valid syntax trees, and Python picks one basically arbitrarily. And there may be expressions that are syntactically valid, but the parser incorrectly rejects them as invalid
@holdenweb The unicode type has three different internal representations, depending on the largest character ordinal. PEP 393 lists the three kinds as 1 byte (Latin-1), 2 byte (UCS-2) and 4 byte (UCS-4).
Or perhaps it refuses to compromise on ambiguity. It can check for these failure states, and use context clues from the surrounding source code to make the correct decision. But then they're compromising on "0 additional tokens of lookahead"
@Kevin I never quite got the point. If you insist on simple syntax to make it easy to define new syntax... isn't the new syntax then not automatically restricted to be just as simple? Otherwise, the advantage is gone pretty quickly.
@MisterMiyagi I'm guessing the designer's preferences come into play at some point, and their own view of "simple" probably isn't what we would all call that.
@Hakaishin Yeah. All other things being equal, I'd prefer python-without-noise over python-with-noise. But if I had to choose between "python without noise" and "python with noise, and also with a suite of powerful syntax-redefining tools"... Not so cut and dry
I have been meaning to explore the question of "can you construct a Turing complete system using only string replacement?". MacroKS could be one possible outcome of that.
@Kevin you can already do that though. sed and ed technically are Turing complete :) same for other "string" replacement dependent langs (most of them are esoteric langs though)
Unfortunately, I am essentially the Blub programmer that Paul Graham describes in his article, so I can't give a truly satisfying example of how NoisyPython could be more powerful than Python. Maybe if I had stuck with those Lisp tutorials for longer. Alas.
I could say "NoisyPython could implement call/cc" but I don't know if that's actually true. It would merely be a diversionary tactic so I could escape unnoticed while everyone confusedly reads that Wikipedia article four times over
@Kevin I wish all people I talk to would immediately understand, accept and incorporate into their language my pet peeve I have with what ever we are talking about. Like NoiseyPython is a good example. I wish everybody would call meat, dead animals when talking to me about veganism :)
I'm amused that en.wikipedia.org/wiki/UTF-32#Analysis is just three solid paragraphs explaining why UTF-32's principal advantage isn't worth the effort
section last edited by user "fixedWidthFormatH8er"
hmm, wondering how to know if a file has a newline at the end, So I thought of reading the very last char/byte like this:
with open("test.txt", "rb+", encoding='utf-8') as f:
f.seek(-2, 2)
# -1 or -2, may change depending on whitespace characters at end of the file
var = f.read(1) # read one byte for a number
f.seek(-1,1)
print("last character:", str(var, 'utf-8'))
would this work for doing this or?
trying it but not sure if there a better way (the advantage here is that the whole file isn't read but just a single char/byte I think)
@ParitoshSingh There's "UTF-8 strings", which are just a sequence of bytes and the iterator knows how to interpret the UTF-8 size indicators (e.g. if the first bit is 0, its a 1byte/"ASCII" character). That's very compact but it can practically only be used for iteration.
There's also "List of UTF-8 characters", where the List (actually Vector) type automatically adds padding so that each element is the same size. A 1 byte character would just get padded with 3 more bytes, for example.
@Kevin "It is extremely rare[citation needed] that code wishes to find the Nth code point without earlier examining the code points 0 to Nā1." Laurel.
also, didn't know you could do a function within a function in python...not sure what's the use of doing that, especially if it's within the same Class
to me, Class are what you should probably use the most when structuring all your functions. So I just don't understand why you would need/want to use a function inside a function inside of a class instead of two separate function inside a class
I don't employ nested functions too much but I'll play devil's advocate and say that it's usually a good idea to use the tightest possible scope for your variables. In other words, if you never use the variable outside a function, then you should define that variable in that function.
The flipside of "choose tight scopes" is "write functions that are useful in multiple contexts". So there's some pressure to move functions to highly visible scopes, in case some other part of the code wants to use it