« first day (2760 days earlier)      last day (2413 days later) » 

15:00
can I free a python variable? so it stops consuming memory
@Aran-Fey Is that dirceted at me? Sorry, I'm doing stuff in the kitchen. Back shortly. My main intention was to show the other answerer a better example of how to answer. But feel free to hammer, and I'll delete it.
@PM2Ring Yeah, sorry, that was for you. I'm also a bit distracted at the moment, arguing with 3 authors of 3 bad answers about why their answers are bad...
@Aran-Fey No worries. I was hoping the OP would respond to my comments, so I could address his issues better in my answer. But that code-only answer kinda derailed me.
15:18
@Neoares you can use "del var" to delete var
ok, thanks
can someone explain me this? (regarding a pandas dataframe)
>>> df.memory_usage()
Index                458361112
killed_by            458361112
killer_name          458361112
killer_placement     458361112
killer_position_x    458361112
killer_position_y    458361112
match_id             458361112
time                 458361112
victim_name          458361112
victim_placement     458361112
victim_position_x    458361112
victim_position_y    458361112
distance             458361112
map_ERANGEL           57295139
map_MIRAMAR           57295139
dtype: int64
all the columns except the ones I generated with pd.get_dummies have the same byte size? ._.
well I think I fixed it passing deep=True as parameter :)
cbg, has anyone set up their own yum/apt-get mirror before?
alright I'm back you hooligans
what's going on
I'm pretty sure I have to ask management to change a configuration option in the production environment, and if they say "no, we won't do that" I might say "then the project just won't ever work" and then chaos will ensue
15:48
Hmm I think I could say "Ok, then I will require 300 additional man-hours for the project" which is more palatable than a hard no
Semantic question: I'm looking to raise an error if the correct answer to a multiple choice question isn't in the provided answer choices - I'm trying to follow "use the most specific Exception constructor that semantically fits your issue" - any suggestions?
I'm thinking ValueError, or AttributeError (since it's a problem with an instance attribute).
@Kevin Make it 500, just to be safe.
You could always define your own WrongAnswerException, deriving from the Exception class.
Purists might argue that getting a wrong answer on a test is not a truly exceptional circumstance, and so shouldn't raise an exception at all
Or, hmm
ValueError sounds like a good fit, but a custom error class like Kevin suggested sounds even better
If it's like "choose one: a, b, c, d" and the user enters Q, maybe that's exceptional
Or, hang on. If the correct answer isn't among the choices? Like "Q: 2 + 2 = ? A: a) 1, b) 8, c) Wyoming, d) 99"?
That would be reasonably exceptional. UnanswerableQuestionException.
@Kevin Precisely. I'm writing a program for creating many versions of worksheets/tests/exams, and I'd like to avoid such a question being created.
15:57
cbg
@idjaw oh darn... you're onto me runs out again
For some reason have had "One Punch Man" on in the background and I really quite like the end theme tune...
No idea what it's saying... so I'm hoping when I bother to look it up - the translation isn't really rubbish... but it's nice and cheery, got that electronic sound of arcade games to it...
Ahh... and if animelyrics.com/anime/opm/hoshiyorisaki.htm is anything to go bye... it's a bit of a love song... I can live with that...
PUP
Those lyrics can't be right, there's no "hai-yah!" anywhere
@idjaw I'm not here...? :)
That's the beauty of foreign language songs, you have no idea what they mean and you just go off the melody and the phonetic of the words and the tone. It's like listening to classic music, no need for words. The downside is running into the risk of what you said, the translation being rubbish.
One Punch Man would be a good SO contributor. He's rarely the fastest gun in the west, but when he does show up, he can solve the problem with a single action.
DSM
DSM
16:11
The first line of that translation is right, anyway, so it starts well. :-)
@KevinMGranger Just because he can, does he violate Zen with horrible one-liners?
@KevinMGranger not sure why I bothered putting it on (the show) but glad I did - it's absolutely hilarious...
I agree. I seem to have a knack for enjoying anime which turns out to be a parody of anime tropes, but then i don't realize it because I don't watch other anime
@DSM pandas definitely doesn't have some helper function buried somewhere that addresses things like stackoverflow.com/questions/50217968/…? Variations of that come up fairly often and I'd have thought while it's not exactly desirable - it's not going to be a completely uncommon set of starting data...
DSM
DSM
No, there's no built-in explode, AFAIK.
16:22
@JonClements no but we have a LOT fo dupes
hmm, I can't seem to find one, but those've been asked before, certainly
cbg. all
Yeah... if there's one with a nice generic answer that happens to include a helper function ors omething...
I hope some answers say "fix your X"
huh this one's a bit different, it would seem... it's looking for an itertools.product-y solution across columns
16:29
All fairly similar base cases though...
TIL cpython lets you reassign __builtins__ to a custom dict-like object
which makes a horrible solution to this question possible that I have to refrain from posting
yeah, there are a few variations, I suppose someone should write up a dupe (me)
While I'm learning about VM internals and frame objects, I realized that it wouldn't be too hard to do tail-call-elimination for at least the simplest cases. Has no one done it because the simplest cases are just as easy to do iteratively, or because they're against encouraging that kind of thing, or because no one's bothered yet?
@KevinMGranger think GvRs written stuff up re: Python about why it's not there or has to be supported by implementations
But it's perfectly possible to implement TCE if you wanted
Oh, I absolutely wouldn't put a guarantee in the language. Heck, I might not even document it
The only consideration is that it makes tracebacks confusing for those not expecting it, I think
16:34
@KevinMGranger GvR's take is at: neopythonic.blogspot.co.uk/2009/04/… - looks like his thoughts on it were: So let me defend my position (which is that I don't want TRE in the language). If you want a short answer, it's simply unpythonic. Here's the long answer:
(lots more details in that post though)
> First, as one commenter remarked, TRE is incompatible with nice stack traces: when a tail recursion is eliminated, there's no stack frame left to use to print a traceback when something goes wrong later.
yeahp
I wonder if there are any code-rewriting decorators out there in pypi... hmm
There's an insane decorator that turns a recursive function into an iterative one
@wim I strangely want a version of that, that doesn't have an obvious loop look to it... so it just looks like it's on some massive never-ending quest of eating...
16:38
Ah, then it's already done. TCE is the art turning a recursive function into an iterative one :)
wim
wim
has a bare except:. stopped reading.
that's what got you? hahaha
There's a name for the kind of gif that depicts a real scene that's been cleverly edited so there's no obvious endpoint. Something trite like "cinematograph"
16:43
I've heard cinemagraph, but for a picture where it's completely still except for one small part. Like a cozy fireplace scene where only the steam coming off someone's hot cocoa is moving.
wim
wim
it will be hard to do with that floor though
@KevinMGranger That's it.
jpp
jpp
Interesting meta discussion here on a Python question (how to use gold dupehammer correctly): meta.stackoverflow.com/questions/367459/…
In future, I think I might just "replicate duplicate vote comment" rather than wield the hammer when there's a potential conflict-of-interest. Seems safer.
On a related note, since we have some canonical writers here, I think they should absolutely be encouraged to mark as duplicates of their own Q&As
Despite making that "how to loop user input?" question for the explicit purpose of being a canonical dupe target, it's very rare that I use it during hammering, for exactly this reason
@jpp I can support that
or you can ask for dupe closure here for other hammerers to see
@jpp FWIW getting out of your way to hammer things to your own canonical is never a good idea. I know you're talking about something else because I read your meta, I just wanted to make this clear.
16:53
I'm glad to see other people use it, however. Bringing to fruition my fiendish plan to produce a valuable thing that my peers derive objective utility from
jpp
jpp
@AndrasDeak, Understood. I'm not in that category yet (no canonicals to my name!), but I understand your point. Some others seem to do it "routinely" [as noted on meta].
yup
I posted a canonical because I had seen a lot of similar questions. I haven't seen one since and it's possible that part of the people struggling with that find my Q&A in google and don't ask in the first place
@KevinMGranger @abarnert mentions techniques for (kind of) implementing tail call optimization here
I'm sure I've seen stackoverflow.com/questions/50219036/… a few times and I'm sure my comment's right but... ... I've gotta go meet a mate quickly at the train station... anyone know of a dupe?
17:05
something like that :)
DSM
DSM
@AndrasDeak: it took me a few reads, but I think you mean "going out of your way".
@jpp Re: that meta thread; I would encourage you to think more objectively in terms of quality; because the only issue with closing as a duplicate of your own content—you tend to get biased towards which q/a is superior
and I can see that happening with you, being the rep-hunter you are ;-)
for the simplest case where tail-call-aware code was already written, I think you could modify the AST to replace the return and call line with a yield, where the returned yield value is the next set of arguments, and wrap the body in a loop. I'm sure there's something I'm missing though
Speaking of loops...
A looping Escher: i.imgur.com/cn91fky.gif
(and then have a wrapper function that keeps using that generator until it reaches tbe base case)
@PM2Ring "help me, I've fallen and I can't get... uh, up? I think?"
DSM
DSM
17:08
Does that reference work for non-NorthAm types?
jpp
jpp
@cᴏʟᴅsᴘᴇᴇᴅ, In this particular instance, someone asked a better method than str.contains, and unutbu's answer is the only I can find on SO. But your general point is accepted :)
wim
wim
it's fine to close dupe to your own answer
@DSM cursory research seems to show Life Alert operates in Europe too, although I don't know if they used the same commercial
wim
wim
perfectly natural since those are the posts you are very familiar with already and can find in seconds
Are you sure? I have trouble sometimes, since I've written so many :(
wim
wim
17:12
I am sure that this is my opinion on the matter, yes :)
jpp
jpp
I find it hard to find my answers. But I find it much easier to find my questions (there aren't as many).
wim
wim
I close dupes to my answers all the time. There was only one occasion where somebody had a problem with that (and it was @JonClements , years ago)
they're easy to find because you search user:me some keywords ...
@wim Indeed. I don't keep all of my answer code on my HD, but I do keep stuff that I think's important or is likely to be useful in the future, with the SO link in the docstring, so I can find it quickly when needed.
@DSM bah, I do, thanks. I have to concentrate to get that right otherwise it's 50-50 :/
wim
wim
17:15
though, I would advise to don't go digging for years old questions to close hammer. ("let sleeping dogs lie").
Also,
I generally try to do a dupe search before answering a question, unless I feel it's very unlikely that a dupe already exists. So if a similar question comes up further down the track, I know I've already done a dupe search, so I can be confident that my answer is a good target. Of course, someone else may have written a better answer in the mean time, but in that case, they should've found my answer in their dupe search and then subsequently hammered my old answer with their superior one. :) — PM 2Ring 49 mins ago
I realise the latent benefit of increasing the reach of your content as well - this is something I haven't taken advantage of until now
I consider gold badge holders to be fungible. If Alice closes a question, that action should be judged exactly as if Bob had closed it. Regardless of whether the target question was written by Alice or Bob or Carol.
@AaronKyleKilleen okay?
17:18
@wim OTOH, we need to do something about high scoring "dinosaur" questions that are Python 2 oriented and which don't apply well to Python 3. Dupe hammering them to a modern question seems like a good approach to me. As long as the new solution really is superior, of course.
wim
wim
@PM2Ring My preferred approach is just editing the old one
especially if the old user hasn't been seen for years, just edit it don't be shy
@wim Sure. But even then, I prefer that to happen with a little bit of peer review, unless the edit is trivial. It only takes a moment to pop in here & say, "Guys, I'm updating this question with this text + code, any suggestions?"
Ah, been a while since I got the new user hat trick: comment saying "this did it, thanks!", no accept, and a follow-up question having nothing to do with the original problem
more like garlic bingo
17:33
I wish we had a canonical Q for "why doesn't x = [1,2,3]; for i in x: i *= 2 mutate the list?" so I could give some direction to Python replace text
wim
wim
@PM2Ring nah, just edit (that's just LBYL vs EAFP). it doesn't mean there's no "peer review" - other users can feel free to add further edits, or roll-back your edit.
@Kevin I feel there was one something like 'other languages have variables, python has names'
Well I can only ask 1 question a week and I did it soooo I need to know here — Goten Black 1 min ago
Ooooh boy
I finally got an accept from the cycle + islice guy. It was touch & go there for a while, though. First, he shifted the goalposts, explaining that the input data was more general than what he originally specified. Luckily, that was easy to handle. Then he insisted that my new code gave wrong output for one of his new sample input sets. But I finally got him to realise that his expected output for that particular set was faulty. :)
Real tempted to reply "I understand, ask your question a week from now and I'll take a look :^)"
migrating from py2 to py3 is much more painful than I anticipated
17:37
Is that really a thing? a one-question-a-week ban? Or is it, like, his teacher that's forbidding him to use SO more often
IPs can get throttled I think
I think that's a SO thing.
@wim How will they know to do that though, if you're updating some ancient question? Yes, it will be visible to people watching the "active" questions, if they're quick, but I normally just look at the "newest" questions.
wim
wim
@PM2Ring I'm saying you can post it here if you want, after your edit, so they can see it in the finished form that you've proposed
@wim Rightio. That sounds good to me.
17:39
I'd have to check the whole question ban discussion on meta, but I have a vague recollection that there is a "one question per week" that allows users to ask questions in an attempt to improve their participation and eventually lift the ban entirely.
@Code-Apprentice What's the most painful part? Separating text from raw bytes?
jpp
jpp
Do we have a canonical on naming variables after classes?
wim
wim
having to discuss beforehand seems vulnerable to this problem:
Sep 5 '14 at 3:42, by Jon Clements
Put all requests in writing to the "Dark Council at the sopython crew" - its committee and sub-committees will review it, and in about 90 years we may decide upon it? :p
@jpp DO you mean like you have a class named Person, should you do person = Person() ?
@wim Fair enough. And there's also the practical problem that formatting a mixture of text and code is borked in Chat.
@PM2Ring I think that is the main reason many of our tests fail...at least the ones that are left
jpp
jpp
17:42
i mean list = [1, 2, 3]
that and someone had the bright idea of sorting lists of dictionaries in some of our tests
@jpp Ah, right. Shadowing built-in names.
@jpp PEP8 suggests list_, but even then you're nearing the edge of insanity
4
jpp
jpp
Yup, we see several of these each week. Would be nice to be able to close as a canonical.
wim
wim
I never liked list_.
Always prefer something contextual like user_ids
jpp
jpp
17:44
I like lst if there's no context.
I think in the past I've voted to close with the "typo" reason when the problem is overshadowing builtin types
@jpp They're usually benign though, we don't often see code where shadowing is the actual cause of the bug. So I normally just tell the OP (or answerer) not to do that via a comment.
Although that's usually more for questions where the person did the assignment like eight hours ago and their REPL session is still going and they completely forgot about it
jpp
jpp
@PM2Ring, Fair enough, though I have also seen instances where it is the root problem.
But I vaguely remember seeing a decent answer on the topic. The trick is how to find it. :)
wim
wim
17:45
Wow, rate-limit of 1 question per week is frightening. We already deal with a fire hose of crap. Imagine how it would be if the worst users weren't rate-limited.
@jpp Oh sure. And when it does happen, the error message can be really confusing, especially if you're a newbie, but even to old-timers it's a bit misleading, unless you've been bitten by that error before.
jpp
jpp
@chrisz, thanks :)
I would only want to dupe vote if the name issue was the entire cause of the problem, and if the problem had a proper MCVE. But you usually don't see that level of diligence from the kind of people that have this problem
wim
wim
@jpp you should change your avatar from a tortoise to the hare
17:47
@chrisz awesome, I needed that. I got an upvote for an old typo answer, so now I have something to hammer it with
@jpp There's some relevant info + links here: sopython.com/canon/98/typeerror-x-object-is-not-callable
No mcve -> close as no mvce. OP says "well it must have been something in my session" -> close as "can no longer be reproduced"
wim
wim
you have already posted more answers in 3 months than I have in 7 years.
jpp
jpp
Here's a funny idea. Are we shooting ourselves by downvoting & VTC? If we downvote, less people see the post able to VTC. Then the post lives longer.
Paradoxically, UV & VTC might close it quicker!
Downvotes have a gravity of their own. Who doesn't click on a post with a score of -7 so they can watch the train wreck?
17:49
Yea I usually go into the downvoted questions just to VTC
jpp
jpp
I guess the point is.. we all know where it is heading (closed as X, Y or Z), but isn't it better to get there in 30 mins rather than 3 hours?
that way OP has extra guidance too (remember questioner doesn't see close votes)
This isn't that useful as a dupe target, but it's kind of relevant: it talks about how to restore the original name in an interactive session after you shadowed it. stackoverflow.com/questions/16523789/…
I think the OP is happier to have it open for those 150 minutes, in the hopes that some rep hunter gives them the codes even though it's a bad question
Hi guys... anyone knows how to deal with lz4 files with a huge JSON array (100GB+ uncompressed, 20GB compressed) in Python? I would like to do some processing in Pandas+Dask but pd.read_json doesn't seem to support lz4.
@PM2Ring I like including that in the list of dupes though, it gives them more background to the issue than just why it's happening.
Also gives a fix
17:53
And since I like to play "Good Cop" I often open highly downvoted questions so I can VTC & reduce the blood loss.
@RobertSmith Looks like pypi has a LZ4 library. Remains to be seen whether it works on a 20GB file, but it's worth a shot.
bah I clicked on a python2 doc link
@Kevin I think it is quite possible the read the file that way in blocks, but how do you process data in Pandas?
@RobertSmith I think you're legally allowed to stab people who create 100 GB JSON files. ;)
I wonder if some memory mapping would be possible
17:59
If you can't process a JSON file in 15 minutes, you're legally allowed to leave.
Actually, it was a 350+ GB file... I managed to remove a lot of fields to reduce it to a 100GB+. Should I stab myself? :-P
Print out the file, then use the blocks of paper to wall them into a crypt in your wine cellar
FWIW, there's json.iterencode(), but no corresponding json.iterdecode(). ButI guess that's reasonable, since you can know how long you have to wait for the closing brackets in the stream that you're decoding
If the file is small, they should have no problem breaking out. That's justice.
@RobertSmith Having never used Pandas, I'm not sure.
Is this the pythonic way of handling an exception in a generator without breaking out of the loop?
18:02
I think the issue here is not actually reading the file, but transform it into a useful structure such as dataframes
@toonarmycaptain getting an error message instead of an int looks bad
@toonarmycaptain I don't think there's much point in having a try block that contains nothing but an unconditional raise.
@Kevin MCVE :P
It is also tricky because the file JSON file has nested objects, so I can't just convert it to csv
Ah, does the real code have more stuff in the try? That would make more sense.
18:03
@RobertSmith If you have nested objects how do you want to turn it into a dataframe?
I know that LZ4 has a streaming API available. But last time I looked there wasn't anything usable from Python, although I might not have been looking in the right places. I spent a little bit of time reading stuff about the algorithm, but decided it was too much like hard work and gave up on it. :)
You need to know the structure of the JSON, but if you know the structure you may be able to cut it up
ANd even if you can get the LZ4 streaming to work you still have the problem of stream decoding the JSON, unless there's some logical structure you cn use to cut it up into manageable pieces, as Andras is suggesting.
@AndrasDeak Sure, that's absolutely a problem. I can "flatten" the JSON, but I first wanted to make sure there was a way to transform it from lz4 to a DataFrame+Dask.
my point is that in order to figure that out you need to know the structure
18:05
As for using memory-mapping techniques, @AnttiHaapala may have some suggestions.
I think I'd be inclined to skip the exception stuff entirely here and just yield None or some other non-Exceptiony signal of failure
wim
wim
@toonarmycaptain no
yeah, I don't think anything other than None would work as a yielded value for signalling a problem (if any)
wim
wim
why would you raise an exception which you catch immediately? that makes no sense
I'm not sure I understand? How do I raise the exception and handle it while continuing the execution without try/except.
Yes, I want my actual code to do more, was just trying to do an MCVE. Instead of my printing function I'm constructing a bunch of class objects for questions in the project I mentioned earlier.
18:08
@AndrasDeak Yes, cut it up might not be a problem. However, let's assume I already have a non-nested JSON file but it can't be loaded in memory. Is there some distributed way to do some processing on that compressed file?
I don't think you can have a raise inside a generator, and continue executing after that raise occurs, unless you catch it inside the generator.
There's no next(on_error="continue") call you can make or anything
if there's something lz4 that works with bytearrays, you can hopefully use stdlib mmap
So the easiest way to continue execution is: don't raise anything
Maybe I'm missing what @PM2Ring and @AndrasDeak are saying by "unless there's some logical structure you cn use to cut it up into manageable pieces"
Perhaps you could yield a (value, error_message) tuple that either yields (something, None) or (None, something) depending on success or failure respectively. Or perhaps you could have an errors parameter that you pass in, which the generator appends to
18:10
mutating generator? :/
I was expecting a parser to work using lz4 chunks even if you can't read the whole file.
Oh are we recreating Result from rust? I'm listening
If you're just referring to having a file with a regular structure ready to be collected, yes... that's the case here: { { obj1 }, { obj2 }, ... }
what library? There's this for instance python-lz4.readthedocs.io/en/stable/…
wim
wim
@Kevin 👎
18:14
Yes, that is the one I was looking: stackoverflow.com/a/48368277/460147
Thumbs down if you like, but they're still the best alternatives anyone has offered, because nobody offered any other alternatives :^)
wim
wim
don't yield errors in generators. if you need them to be handled, pass in an on_error callback. this follows the model of os.walk.
@RobertSmith Sure, that's possible. But how do you stream-decode the actual JSON data? The matching close brace or bracket for the opening one is 100GB away.
ah, so many pieces to fit...
@PM2Ring Streaming usually remove the wrapping brackets and goes through its inner objects
At least, that's how it works when you stream objects in jq
18:17
That's fine if your inner objects aren't too big. Can you guarantee that? If so, you should be ok.
(jq is really slow in this case, by the way)
wim
wim
@Kevin accumulating them in a list means they can't easily be handled until the generator is exhausted. you might not ever consume the generator completely.
Yes, inner objects are small enough
@Kevin # so if:
wim
wim
and returning tuples with "ok" thing to check every time is hell. if you like that, may as well move to golang.
18:18
Oh, okay... then I should give up on using DataFrames and instead focus on creating a generator with all the inner objects
Well, in that case, I suppose chunk = file.read(size=chunk_size) is good enough
the processing can start there and collect what I need. I just thought I could use a more high-level approach
(groupby, specifically)
note that neither PM nor I are pandas experts
but I'd be very surprised if there was some magic along these lines
yeah... Well, I will wait a little bit to see if someone comes up with an idea to make it work in Pandas
but looks unlikely
# so if:
for x in generator():
    if error:
        # do error stuff
    else:
        # instantiate class
18:22
I just hope that this lz4 binding is fast enough :-P
isn't good, what's left?
What is on_error?
I'm actually ok with appending to a list, when I iterate over a bunch of questions from text or Blackboard etc formatted input, I can add the questions to a database or pool to generate worksheets from, and then just say sorry I'm not allowing you to use xyz questions because they're stupid.
I'm not entirely convinced that a generator is useful/necessary here, to begin with. It's not the best possible encapsulation to put error generating code and error handling code into separate functions, if you only intend to use them together anyway
def gen():
    for x in whatever:
        if y:
            signal error somehow
        else:
            yield z

for thing in gen():
    if detect error somehow:
        do error stuff()
    else:
        do non-error stuff

#... Is not strictly better than

for x in whatever:
    if y:
        do error stuff()
    else:
        do non-error stuff
The latter code solving the problem of "how to signal an error?" by not actually having to signal anything
wim
wim
@toonarmycaptain on_error is a callable, specified by the caller, which accepts one argument - an exception instance. if you want to use that to accumulate errors in a list and deal with them later, you can.
18:34
The idea being that there's no need for the generator itself to know how to handle the error. But the caller of the generator should know how to handle it (or somewhere up the chain of the caller's caller, etc). So the caller supplies a callback that the generator can call when an error arises.
Jul 17 '15 at 12:55, by PM 2Ring
In accordance with the ancient programmer proverb: Never test for an error condition that you don't know how to handle.
are there any django experts here?
i'm so confused :l
@cᴏʟᴅsᴘᴇᴇᴅ Done
Welcome, Fishy. Have you read the room rules yet?
hammered, thank you
i have not, but i don't see a rules link :(
@AmagicalFishy It's invisible on the mobile interface. Room rules: sopython.com/chatroom
wim
wim
@AmagicalFishy depends ... are you asking about the gypsy jazz guitarist, or the old western movie?
hey, they made a remake of django with jamie foxx
"Django: Unchained", a true Tarantino cult classic
wim
wim
yeah, great movie
Tarantino been strangely quiet since then
oh wim you a fan?
18:44
didn't hateful 8 come out after django?
wat? There was this H8ful 8
though, I'd want to forget that it exists :F
wtf that movie is great!
I've not seen it. Not sure if it's worth seeing or not
Are you sure you're not thinking of the one with adam sandler?
Worth seeing but not the best Tarantino
18:45
I did just see Paint Your Wagon recently, it's put me in a western mood
my wife fell asleep in the movie theater watching H8, and after she woke up she wished she could have slept through the rest. :D
no movie with adam sandler is great
Adam Sandler's career was a mistake
Adam Sandler was a mistake
@AmagicalFishy true dat.
@cᴏʟᴅsᴘᴇᴇᴅ or the other C-word :P
wim
wim
18:47
@AmagicalFishy oh yeah. didn't see it yet, thanks for the reminder.
@AnttiHaapala Which one? :p
I'm not sure what you mean by callable specified by the caller here.
I'm not sure if it's relevant (famous last works), but the error would be picked up in a function called by the generator, but I assumed I'd have to just pass it back, or ignore it.
Here's the script. The TextQ class takes the question data, to be assigned to a database, formatted in an output document, etc.
@toonarmycaptain Something like this.
def gen_reciprocals(seq, error_callback):
    for item in seq:
        if item == 0:
            error_callback(item)
        else:
            yield 1.0/item

def an_error_occurred(value):
    print(f"An error occurred trying to find the reciprocal of {value}")

for x in gen_reciprocals([1,0,2], an_error_occurred):
    print(x)
1.0
An error occurred trying to find the reciprocal of 0
0.5
Now the caller is free to specify how errors should be handled, without having to modify gen_reciprocals
hey, is there any way to get help on linux parameters? for example apt --help works, but apt list --help does bring apt --help content instead of apt list specific parameters.
@Teomanshipahi that's not related to python at all
try reading the manpage
wim
wim
19:01
@AndrasDeak apt is written in Python ;)
Mostly I just google stuff and try commands that look like they'd be useful. That's pretty much how I do everything, really.
my bad :P
@AndrasDeak replace apt with python, and you're good.
uh-huh
if we're going technical this isn't even on topic for chat.SO
@Kevin So really the significant difference is encapsulating/factoring out the error handling action?
19:03
Yeah.
wim
wim
I'm smoking crack. apt is c++, it's yum that's python.
@PM2Ring interesting
I've used lz4 too, the python library just had compress buffer + decompress buffer
wim
wim
I <3 lz4
it's so fast
19:06
speaking of yum and python, does anyone know a nice guide for making your own mirror locally?
@RobertSmith there is an iterative json decoder, good luck using the standard one.
If I do that, and I want to log the error (you know, so I can go back to check that it's not picking up things it shouldn't eg poor formatting I can handle differently), aren't I violating some convention/rule by logging an error far away from where the error is occurring? Tracebacks etc
Why does Django not pass through all the files to my form when I use a file input w/ multiple-True?
though, you'd perhaps want to consider rewriting it to msgpack if possible :D
I'm trying to construct a modelform whose instance have all the same info except that which is determined by a file—so the user can fill out a single form, select multiple files to upload, and hit "submit" (creating multiple objects in the DB, one for each file)
19:08
our eventual usecase was lz4 + msgpack, though for much smaller pieces of data - these were tweets - but the stuff we were doing - filtering tweets in C - that worked quite well there.
so we'd write filters in Python DSL, and the tree would then be converted to C code that would be compiled with TCC at runtime and linked against the utility functions :P
wim
wim
@toonarmycaptain The callback should receive an exception instance. The exception instance has a traceback attribute.
@AnttiHaapala That would be the equivalent to using python-lz4, right? stackoverflow.com/a/48368277/460147
Hey @Antti You've used the heapq module before. This question's got me puzzled. The OP is using heapq with a crazy class that has inverted __lt__ and __gt__ methods, to get maxheap behaviour instead of minheap behaviour. But it still behaves like a minheap. Any thoughts? stackoverflow.com/questions/50220019/…
Sure, I could just negate the .val attribute, but that will only work with numbers, and I'd prefer to have something that works on strings too.
@PM2Ring total_ordering
DSM
DSM
19:24
"Please take a few minutes to enjoy our transformation journey." <- from the introduction to NumberFirm's last company-wide email
@AnttiHaapala I just implemented all 6 comparison methods. Still no go.
@wim so instead of yield x, y, z (normal exec) I am doing yield x, y, z, MySpecialException() [or yield None, None, None, MySpecialException()]
@PM2Ring You have to fix the comparisons. self.val > other -> self.val > other.val
@PM2Ring are you using Python 2
wim
wim
@toonarmycaptain don't yield errors. I don't know how to say this any more clearly.
19:27
@PM2Ring return self.val > other -> how about comparing against other.val :D
@Aran-Fey Ah. Of course.
lol
kevin'd by aran fey
and antti'd by myself
Beautiful! Thanks @AnttiHaapala! Now I feel silly for not realising that. I guess I'm still a bit rusty from not coding for a couple of months.
@DSM "mhmmmm, metamorphy"
19:45
@AnttiHaapala I really don't like total_ordering It's a band-aid solution. And as the docs warn:
Note

While this decorator makes it easy to create well behaved totally ordered types, it does come at the cost of slower execution and more complex stack traces for the derived comparison methods. If performance benchmarking indicates this is a bottleneck for a given application, implementing all six rich comparison methods instead is likely to provide an easy speed boost.
Oh well, the OP's disappeared.
after lunch, cabbage
Why would you eat cabbage if you just had lunch?
it's an after lunch snack
makes up for the pizza I had for lunch
20:02
Does anyone know if CL. is a developer of SQLite3?
@PM2Ring those should be wrapped in c
wim
wim
@PM2Ring well what do you expect, it's in the junkyard (functools)
20:14
@wim Oh, right, I'm not yielding it, I'm just running that function without yielding in that iteration. Now I want my custom exception to put a stacktrace somewhere. I guess I have to use logging for that.
how will you know there was an iteration that didn't yield?
or are you happy with getting the next item?
@wim I never expect multiple levels of Python function calls to be fast. Decorators may be cute, but I wouldn't use them for time-critical stuff. And comparison calls of a heap implementation count as time-critical in my book.
That's why decorators should just rewrite the AST ;)
@PM2Ring Your 6 rich comparison methods don't work right. You still have to compare the .val attributes
@KevinMGranger That'd be popular. :D Although it'd be nice as an optional thing, I guess.
20:23
cibuildwheel is great. Plug a few lines into Travis / Appveyor configs and get Linux, Mac, and Windows wheels built.
@Aran-Fey Yam. I C&P'd the old versions.
I'm finally getting MarkupSafe a comprehensive set of wheel.s
Thanks, @Aran-Fey! And thanks again, Antti.
wim
wim
20:37
@KevinMGranger fun idea, any precedent for that in stdlib?
the only ast rewrite I've seen in the wild is pytest contextual assert
Probably nothing in stdlib, and that's the only example I've seen too
@AndrasDeak I'll have my callback function do something. I'm happy with running through all the items, then saying xyz didn't work.
rbrb all
I see! rbrb
wim
wim
import ast
import inspect


def more(f):
    source = inspect.getsource(f)
    parsed = ast.parse(source)
    parsed.body[0].body[0].value.right.n = 2
    code = compile(parsed, 'potato', 'exec')
    f.__code__ = code.co_consts[0]
    return f


@more
def inc(n):
    return n + 1
20:54
+1 for using potato as a throwaway name
Are code objects modifiable? You could just modify the consts tuple and replace the 1 with 2
wim
wim
nah I think thats readonly
wonder if this works on pypy

« first day (2760 days earlier)      last day (2413 days later) »