« first day (2778 days earlier)      last day (2170 days later) » 

wim
8:00 PM
open(fname, 'rb')
 
...that should be the same in both versions, though?
 
DSM
Well, that won't work, because bytes object don't have an encode method.
 
wim
opening it will work
:P
base 64 is now a binary transform:
>>> import base64
>>> base64.b64encode(b'ananas')
b'YW5hbmFz'
 
DSM
;-) But yeah, open the file in binary mode, and then do .. what wim just said.
 
yeah. i tried using base64 to encode it—but received a similar "this is not json serializable" error, indicating that there's something pretty different
 
8:03 PM
@AmagicalFishy It's hard to help without knowing what your code is doing. We have pieces and fragments like "file", "base64" and "JSON", but no real idea what you want to do. A Minimal, Complete, and Verifiable Example would probably help.
 
(the file's already opened in rb mode; at least i think it is. lemme check the tempfile docs real quick)
good point @Aran-Fey lemme read some stuff and i'll get back w/ something closer to the problem i'm actually experiencing
 
wim
@roganjosh how you call the code readable that uses a mixture of 2, 4, and 8 spaces indent :P
 
How is this question getting upvoted stackoverflow.com/questions/50536087/… Am I missing something very obvious or is this a confusing question
 
wim
Actually, now that I look at it again, they seem to be using 2**n indentation (where n is the indentation level). Maybe there is method to the madness! ;)
 
@wim It's a good method; the more nested the code gets, the louder a PEP8 linter will scream.
But yeah, it's a stretch to condone the indentation in that answer.... it's runnable, but I can't wholeheartedly defend it
 
DSM
8:07 PM
@AMF: You can't seralize bytes with json. What you can do, though, is decode the result of b64encode into a string, which can then be jsonified safely.
 
So, is there a way to decorate a function to receive an array of some specific dataclass?
 
Wim, what test scale would make you happy for a benchmark? Number of keys and number of entries?
 
wim
@roganjosh I would not benchmark this at all unless it was identified as on the fast path of some performance critical workflow.
 
I'm actually curious as to whether I can extend Akash's approach and if there will be a break-point where less-clear syntax wins
But you answered a question asking for the "best" approach. If it can't win on clarity of code, what else do we benchmark "best" against?
 
ok, i think i'm closer to what's going wrong. Is there a difference in:

{
"one": [1, 2, 3, 4],
"two": {
"sub1": [1, 2, 3, 4],
"sub2": "eight"
},
"three": 3
}

in Python 2.7 and Python 3.5?
like, do they handle this dictionary of stuff differently?
 
wim
8:14 PM
In the absence of any other information, "best" to me means the same as "pythonic" - simple, correct, readable. but opinions differ.
 
@AmagicalFishy The only difference there is that python 2 strings are different from python 3 strings
(In python 2 they're a weird amalgam of bytes and strings, and in python 3 they're actually strings)
 
I'm only curious about this because I've watched the debate over the singleton with PM 2Ring over several days. It's nothing more than curiosity for me, I'm just trying to work out where your own boundaries lie :)
 
@AmagicalFishy There is a very small difference in dictionaries: in both versions, dictionaries with string keys are in arbitrary order, but in 2.7 that arbitrary order will usually be the same across separate runs, while in 3.5 they'll usually be different across different runs. Either way, if your code relies on either behavior, your code is wrong, so this rarely matters. The difference in the strings within the dicts is a lot bigger.
 
ah, hm. I bet that's what's causing my problem. do you happen to know if the difference in str would make serializing... weirder? (it sounds like, from what you said, it should actually be easier to serialize strings in python 3)
Ah! luckily, my code doesn't rely on the order :v
 
@abarnert you can set your environment variable, I believe it was $PYTHON_HASH_SEED to a specific variable to guarrantee consistent behavior, no?
 
8:16 PM
Because, if performance is no issue, why would you use a new approach to answer a question when it can be solved with for loops and it becomes less readable
 
Well, it's almost always be easier to serialize strings correctly in Python 3, but it's often easier to serialize things incorrectly (but it still works for pure ASCII) in Python 2.
 
It can happen that your code "just works" in python 2 (but in reality it has latent bugs) and python 3 throws an error instead
 
@OneRaynyDay PYTHONHASHSEED. And IIRC, setting it to 0 usually even gives you identical ordering to Python 2.7 (if you were using unicode in 2.7).
 
yeah. something like that is probably what's going on
 
@abarnert I see. I remember having to set this to guarrantee reproducible behavior in some research environment
 
8:19 PM
ah, alright. i'm going to construct a minimum-working-example and, if i can't figure out what's causing it, i'll ask an official question :D ty for your help
 
wim
@roganjosh not sure what you're getting at
 
@OneRaynyDay Yeah, but for that, you usually don't want 0; you want to generate a random number that you can publish in your paper and set the environment variable to.
 
wim
also think mine or the ChainMap answer are the most readable there, of course I'm biased.
 
Yo, anybody know the syntax to assert the type of list of dataclass derived objects?
 
@abarnert I think you're right; it wasn't 0.
@Mikhail assert with isinstance?
 
8:22 PM
So, if I have a list of them, do I need to iterate the whole list and assert that?
 
yes because python contains heterogenous "things" inside of a list, roughly speaking
 
Seems like an O(N) just to assert the type?
 
@AmagicalFishy One of the things that often catches people porting 2.7 code to 3.x (or dual-version) is the small handful of bytes-to-bytes and unicode-to-unicode encodings. And I think you mentioned base-64 earlier, which is one of them. In 3.x, you can't encode bytes or decode unicode. But the change is always simple once you know that's the issue—e.g., instead of s.encode('base64') you do base64.b64encode(s), which works correctly in both 3.x and 2.7.
 
you can do np.array(..., dtype=yourtype)
which will assert upon insertion, sort of.
 
@wim You're a more experienced programmer than me, but you came back to chat to announce that you used a new method, that you'd never used, to solve a problem that can be solved with a really basic approach for someone new to programming. But you reprimand PM for what you consider an anti-pattern (I'm using that term loosely).
 
8:23 PM
Yeah, keeping them in an nd.ndarray isn't a bad idea
 
np.ndarray you mean
 
But is there some more paradigmatic way to make a list of @dataclass derived objects?
 
wim
PM asked me my opinion. My opinion was that using a metaclass overcomplicated things unnecessarily. I sincerely hope he didn't feel "reprimanded"
 
there are many, but there is only one Right Way; and that requires more context I think
 
wim
What is the really basic approach? are you talking about using:
if k not in d:
    d[k] = ''
 
8:26 PM
Even though np.array is actually a factory function, people loosely talk about the np.array type instead of the np.ndarray type all the time. Even Numpy's tutorials. Even the repr for an array is a call to the factory, not the class constructor.
 
@abarnert to be honest, I don't really blame them because the nd in front is a bit redundant
 
I mean, I have a list/array (1 gigabyte) of @dataclass derived objects that I need to assert the type of when I'm passing it to other functions. Not sure whats the best way to keep the type signature as I'm refactoring...
 
@Mikhail Python does not have a "homogenous list" type; you just use a list and don't stick incompatible types in it. You can statically check it by using type annotations, but you can't, and generally don't want to, dynamically check it.
 
unless you start thinking about np.matrix which is n=2 with extra stuff, but bleh
 
@wim maybe "reprimanded" is not the right word, but you're holding a downvote over his head. And again, I only want to discuss this in terms of what I see in chat, nothing personal. I'm just struggling to see the difference between the two situations on a fundamental level.
 
8:28 PM
@Mikhail mikhail you're a great C++ programmer from what I remember but this is python and the dance is a bit different
 
if k not in d:
d[k] = ''
Will fail because the keys can't be known in advance
 
wim
I never understood the "ndarray" thing. Maybe they didn't want to stomp the stdlib array module name.
 
The matrix docs used to say that you rarely need it, and it's only there because Python doesn't have separate matmul and mul operators. It's weird that now that Python has @ (and it works on ndarray), the docs for matrix don't try to warn you away that much.
 
wim
@roganjosh that's just replacing the setdefault call (inside the context of the keys already having been found).
you have to iterate the entire list of dicts first, regardless, right?
 
@wim But the array factory function stomps the stdlib module just as much as the type would.
 
wim
8:30 PM
yeah
and they apparently didn't mind stomping any and all
 
@abarnert Okay, so what is the type annotation when you need to assert that your list of objects (or any other container) is of a the same custom class...
 
wim
ndarray just sounds like array with a speech impediment
 
@wim np.assarray
 
@Mikhail List[MyClass]. More generally, see Generics.
 
Agreed, I've taken that code a bit out of context. So if a solution exists that uses that code, which I think is much more readable, would it not be prudent to test its speed first since it definitely wins on readability?
There should be a good reason for the OP not to use a for loop instead of set().union(*lst)
 
8:33 PM
@wim There are languages where nd as a syllable onset cluster is valid. Of course English-natives sound like they have a speech impediment when trying to speak those languages, so…
 
Whoa. I have a very minimum working example:

import json
json.dumps(b'123ABC')
This works fine in Python 2.7 (which returns '"123ABC"'), but 3.5 throws an "... is not JSON serializable" error
 
@AmagicalFishy JSON only knows a few types—strings, ints, etc.—which does not include arbitrary byte-strings. The only reason it works in 2.7 is that bytes and str are the same type.
The right way to handle this depends on what that b'123ABC' represents. If it's ASCII text, use json.dumps(b'123ABC'.decode('ascii')). If it's Latin-1 text, or UTF-8 text, or text in your default encoding, just change the decode argument.
 
Ah: it's actually the contents of a .wav file, encoded to base64
 
wim
@abarnert ... nigerian ?
or maybe some indian languages
 
8:40 PM
OK, that's fine—base64 output is ASCII, so decode('ascii') is what you want. (Unless you need dual-version 2.7/3.4+ code, or 3.3+, and the performance cost might matter.)
 
aaah, that is awesome. it's a value in a dictionary, but i don't think that should matter much (i'll just decode it directly). lemme give it a shot
 
@wim I'm pretty sure "Nigerian" means either the local dialect of English or the local dialect of Hausa, not any language of its own.
 
whoooooo
 
playing with pexpect
 
that was it @abarnert thank you kindly. :D (everything'll be run from docker so i can pretty much neglect 2.7)
 
8:42 PM
@wim all I can take from this whole thing is "don't teach newbies about singletons but I'll answer a question using a method I've never encountered, in an experienced programming life, to solve a simple problem". Now I'll put this discussion to bed from my side :)
 
wim
@roganjosh I didn't say I've never encountered it. I've encountered it many times.
I just didn't see any convincing use-case for it until today.
 
What? "Just used dict.setdefault for the first time in my life"
 
wim
Yep
 
"used" != "encountered"
 
@wim Anyway, I was thinking of southern Chinese languages. One of my fellow students in grad school was a native speaker of… now I can't remember, except that to me the name of the language sounded a lot like Ndndnd from Futurama, but to her it sounded completely different of course, and I think it was considered a dialect of Mien in China but a separate language in Laos and Thailand.
 
8:45 PM
Eh, so the argument has boiled down to this.
 
the only use case for setdefault is if you don't want to import defaultdict
 
Also, Laos was called something "Thousand Elephant Land" in her language, which was something like "Ndzaangz", which my ears just gave up on.
If you want something that will raise KeyError after it's built, but not while you're building it, I'd use setdefault rather than building a defaultdict and then copying it / changing its __class__ / hiding its __missing__ / using a custom defaultdict subclass with a flag to flip back and forth / etc.
 
wim
np.ndzaangz sounds like some crazy function that only divakar would know how to use
 
I don't get it. Perhaps PM 2Ring had never used a singleton in his life, and had encountered it, but decided to use it to answer a question.
That's enough internet for me for one night. cbcb.
 
wim
@roganjosh Perhaps. So what?
 
8:50 PM
rbrb even. Gosh I'm tired :/
 
@wim perhaps it needed a different name to have the factory as array
nobody initializes using the ndarray constructor, so the name is rarely needed
 
wim
right but the type could be called array, the initialiser could be called array, and the low-level initializer that is not for public use could be _ndarray or whatever
 
@wim With the rising-falling tone it's some crazy function that only divakar would know how to use, but with the falling-rising tone, it means "erase my hard drive".
 
Hmm? How would you have two arrays which are different? Or did you mean hooking the current array into the the __init__ of the new array class?
 
I suspect the only reason Numpy doesn't use array for the type is that its design goes back to before builtin type/class unification. The same way int used to be a factory function for types.IntType, etc.
 
8:53 PM
hmm, right, lots of baggage in numpy (and I know none of that)
 
So, I have a function signature like do_correlation(connections_in : List[Connection]... but it looks like PyCharm's code hinting can't infer that connections_in[0] is a Connection object, is there some trick to providing it with more information?
 
@Mikhail Does Mypy infer it? In a simple case, it definitely does. So what we need to figure out is whether (a) your case is actually not simple enough for type inference (and then we can figure out why, and what to do about it…), or (b) it is, but you need a newer version of PyCharm (that may exist yet).
For example, in this code, Mypy will infer that c is a namedtuple with an attribute named a:
Connection = collections.namedtuple('Connection', ('a', 'b'))
def spam(cs: typing.List[Connection]) -> None:
    for c in cs:
        print(c.a)
 
So, it can infer the type when its not in a List[T], don't know about mypy
 
The point is that Mypy is the reference implementation for Python's static typing rules, and PyCharm is a separate implementation that's better in some ways, but more complicated and not quite as guaranteed to be complete and correct. So you need to figure out whether the problem is with your code, or with PyCharm.
And if you can't provide a MCVE for others to debug for you, you have to run it through Mypy and report back.
 
\o rbrb it's Friday. Have a nice weekend :D
 
9:12 PM
@MooingRawr bye
 
9:36 PM
So, opening and closing PyCharm seemed to have fixed its inability to infer types
Now my problem is that type information is lost when using tqdm: aka for con in tqdm(connections_in): :-/
might have to stop using tqdm
idk, is there some other progress bar?
 
You could add type hinting to tqdm (either keep a fork, or submit it upstream). Or, more simply, just typeshed the one function you use out of it.
I'd assume that function just takes an Iterable[T] and returns an Iterable[T] or something like that, right?
 
Yeah, I also noticed you can manually invoke the tqdm print functionality
 
wim
for all these problems, does typing information actually give you any benefits?
 
I have a lot of code to verify. Pushing 12,000 lines. Right now its just passing lists of lists, or other random containers.
 
The only benefit I've seen is when you're porting 2.x code, and you didn't bother to keep track of which strings and bytes and which are strings, adding the annotations as you figure out each one is the only sane way to keep track of them.
Beyond that, it's sometimes useful as documentation, but I rarely bother running a verifier on it. When I'm just passing around lists of lists of whatever… that's usually because I want to be passing around lists of lists of whatever—often because I'm handling some kind of dynamic data, like a JSON response that's far more complicated than the part I need, or an HTML tree from a flaky site, etc.
And when I see other people's code with annotations, 90% of the time they've made things far stricter than they actually need to be, for no benefit. Like a function that takes any iterable of integral values, and in fact in most non-testing use cases you're going to be passing it a generator expression as that iterable, but someone annotated it as taking List[int], so you either have to fix that, or disable type checks on it.
 
9:47 PM
That's a new greeting "Hello Stack Overflowers"
 
@Simon Where do you see that?
 
So answerers are StackOverflowers, and askers are StackOverflowees?
 
Probably they didn't mean it in the way I took it (i.e. we program stack overflows). :p
 
10:04 PM
Stack Overflower? I hardly know 'er!
 
wim
Get her StackoverFlowers for mother's day
 
10:30 PM
@Mikhail wim said progressbar2
 
10:47 PM
@abarnert The guidance I was given for type hints was "return types as specific as possible and parameters as permissive as possible", which, so far, works ok.
I would agree though that the main gain I get from it is documentation
 
wim
I wonder what % of ignacios answers are these code dump only things
 
11:07 PM
@Arne That's good guidance, but in practice the people who most like type hinting seem to be the ones least apt to follow that advice. Basically, people who'd rather be coding in Java or Go or some other language where the static type system is just strong enough to get the worst of both worlds (frequently manages to prevent you from writing the obvious code you want, often fails to prevent you from writing code that's wrong but compiles anyway).
 
I work close to a team that writes scala, and one of them linked me an article that basically claimed "meaningful variable names are code smell, you should have typed them well enough that you can call everything a,b,c,.. and still have readable code." Reading that was very upsetting.
The following discussion where I tried to claim that this might not be true for python, and typing is not what 'finally makes it a proper programming language' even more so =(
 

« first day (2778 days earlier)      last day (2170 days later) »