« first day (3807 days earlier)      last day (1145 days later) » 

2:08 AM
Gah! Second time in about 4 years where I got bit by datetiime_var.timestamp() of a naive datetime implicitly interpreting datetime_var to be in the local timezone. To force getting a UTC timestamp requires datetime_var.replace(tzinfo=timezone.utc).timestamp().
 
3:01 AM
Just wondering what the convention might be for this variable, should it be put in all caps? FILE_BASENAME = os.path.basename(__main__.__file__) The variable is at the top of my script and it obviously won't change.
 
From PEP 8 "Constants are usually defined on a module level and written in all capital letters with underscores separating words. "
so if it is a constant then its fine
 
ok cool thanks
 
3:36 AM
@Kevin My favorite, the extJS JavaScript framework had is_disabled as a getter for UI widgets, so I was tempted to write code like if !widget.is_disabled() != False
 
3:56 AM
Is it just me or is anyone else seeing "Stack Overflow for Teams" ad on their left panel recently?
It lets me close the ad, but I am not sure if it was always there
 
 
1 hour later…
5:03 AM
@python_user Yeah a bit, they must have changed their advertising mix.
 
 
3 hours later…
8:14 AM
Does someone know a good dupe for "Why does {1:1, 1:2} contain only one item?"
 
8:45 AM
@PaulMcG I already have my submission for the Cock-up Cup in our weekly standup. Apparently, Redshift is happy to parse totally infeasible dates if you set the format as "DD.WW.YYYY" instead of "DD.MM.YYYY". I think you're hard-pushed to beat that; we've had to throw entire studies in the bin off the back of that one :P
I mean, it would be handy if there weren't 5 datetime formats used across the tables, but they also store ints as strings and throw a nice ol' comma in for units of 1000+ so... it's not nice data
 
 
3 hours later…
11:23 AM
Cabbage all
Does anyone know if Flask interferes with fork() calls? I'm assuming it doesn't, but thought I would ask the room
 
I've never been able to properly experiment with fork() because Windows implements it weirdly ;_;
Fork Lite©
 
Daemonisation is definitely one place where Windows is a bit more sane than Linux.
 
Or maybe it can't fork at all...? I'm getting AttributeError: module 'os' has no attribute 'fork' right now, but I'm pretty sure I could fork in at least one of the boxes I've owned in the last ten years
Maybe it was a janky pypi module
 
(The hoops you have to jump through definitely starts feeling like some esoteric eldritch incantation)
 
Well I've demonstrated what my opinion in this matter is worth, but I'll go ahead and say I don't know of any reason that Flask would prevent or prematurely kill a child process created by fork
 
11:31 AM
It's appreciated though - this very much feels like a sanity check
It just means I should start looking more intently at uwsgi eyes narrow
 
Aren't forks first-class processes? So you kill them the same way than non-forked processes.
@Kevin I usually hear it say that fork is a linux thing
Usually in the context of stackoverflow.com/a/20222706/5067311
 
I thought Windows doesn't implement fork at all?
 
yes, that's what we're saying :P
 
This is the whole reason why you need the if __name__ == '__main__' guard on multiprocessing
 
also what I was saying :P
 
11:37 AM
Can we pretend I was giving useful original thought anyway? :P
 
independent verification!
 
The core issue is that I have a daemonised process that works when it's kicked off from python or the command line. If I kill the python/terminal/etc, then it's adopted by init fine and keeps running. If I go via my Flask (and uwsgi), then when Flask is terminated - it's taking the daemon with it.
 
<3 thanks for indulging my ego.
 
@OldTinfoil what's the difference between "kicked off from python" and "via my Flask"?
 
The code is identical
 
11:39 AM
but yeah, from the nothing I know about web servers I find it likely that whatever is managing the flask processes is to blame
 
Except that Python is either python code, or via the command line interpreter
Flask is the same code, but being invoked when someone visits a page, rather than me slamming the enter key with pretentious satisfaction
 
@OldTinfoil how do you daemonise the process?
 
Is the forking done by a literal os.fork?
 
It's being daemonised via some code I've been maintaining for a few years, based off the jejik daemonisation code
It does the double fork dance, detaches from stdinputs, etc. I've augmented it so it used pidfiles like a sane person
 
@OldTinfoil so in principle it's the same as nohup other_process &?
 
11:43 AM
Working assumption: WSGI is a good parent and reaps its children instead of making them orphans.
NB: sys speak is harsh.
 
Nah, it double forks so that it gets adopted to init, so doesn't need to be nohupped. It will no longer use stdin/out/err from the parent process. It'll have a different process group id and the parent process is definitely becoming 1 (ie, is getting adopted properly)
It's especially important to reap orphans if they've become zombified
I'm coming to the same conclusion too MisterMiyagi, but I can't work out how it's keeping a track of the daemonised process as it should be getting freed.
 
On a totally unrelated note, I'm pleasantly surprised that one of my async "this is how you do open heart dentistry on yourself" frankencodes is being re-used for various other Q&A. That's another, oh, hundred well-paying code maintenance jobs created.
@OldTinfoil Sadly, it's been too long since I've created my own daemons.
 
I think I've sacrificed all the right goats in the right order. But one never knows if there are some entrails that one has forgotten to tidy up
 
Not really sure how all the stuff works exactly.
 
(If it is the case that uwsgi is being a "good parent", then I will accept defeat and find another way of starting the daemon process from the web interface. I don't want to interfere with uwsgi doing its job well)
@MisterMiyagi Thousands of years in the future, people will curse your name as their hearts are sown into their mouths.
 
11:51 AM
@OldTinfoil Yes! Immortality! laughs ominously
They can fight against the robotic offspring of my self-parsing-parser. commits
 
Oops, I've been informed of an answer I wrote three years ago that was producing obviously incorrect results the whole time
That's another hundred well-paying code maintenance jobs created... Oh, who am I kidding, it's five at best
 
I have one answer to give me my "massive" stash of points. It's essentially "RTFM", here's a link and a brief example.
No way that's creating any jobs.
 
I have promptly redesigned the code to produce correct output, noticed my code was now functionally identical to my higher voted competitor, and self-deleted. From the point of view of the commenter, he told me about the problem and I self-deleted in a huff without bothering to fix anything. Commenter, please understand.
 
@OldTinfoil you should update it with a youtube tutorial and tiktok
 
If you ping someone in a comment and then immediately delete the parent post, do they still get the notification? Sometimes I'll do that on the off-chance they see it in the fifteen second time window.
@AndrasDeak Youtube is passe, the new hotness is putting your documentation in a discord
 
11:59 AM
@AndrasDeak I am as charismatic as being slapped with a brick filled haddock. Makes me nauseous thinking about it!
 
Love to spam the community with "Kevin joined the discord! Kevin left the discord!" every time I need to check if the API uses pascalCase or not
Fortunately I have only suffered this indignity for unimportant HTML5 game frameworks
 
@Kevin I thought you were being silly
But of course that's a thing
 
If the situation doesn't improve by v1.0, I'll write a discord bot to pirate everything and put it on readthedocs
But surely the maintainer will get tired of the electric shocks notifications by then. Surely.
 
On one hand, I expect JS devs to have a little more basic sanity than PHP devs. On the other hand, they're crazy enough to work with a language that has to function under 3+ mutually exclusive interpreters/environments
 
wonders what arcane truth DenverKevin9 just (removed)
 
The truth that I can reverse engineer the intent of a 404'd meme image just by looking at its url
 
from what I can tell modern PHP is not insane at all
 
Also: just typing "denver" in my search bar brings up xkcd.com/979 as the only suggestion. Scary.
 
12:29 PM
Theory: server-side JS became popular because it was a diaspora for JS devs who couldn't bear to hear "The customer says it doesn't work in IE6" one more time
 
@MisterMiyagi not mine
 
denvercoder9 pops up as a suggested query for me, but it does that for anything I've typed into the search bar more than once
 
@AndrasDeak I blame caching Siri.
 
@AndrasDeak I've also heard that, but I have to scapegoat somebody
 
I'm also using a non-tracking search engine
 
12:36 PM
I've half-heartedly installed a tracking blocker extension or two, but I'm more or less convinced that the billion dollar advertising industry will rowhammer me if they have to in order to determine whether I prefer pepsi or coke
 
@AndrasDeak I keep meaning to come up with a plethora of tools to suggest to people
 
I mean it's just duckduckgo
 
If it searches like a duck and tracks like a duck...
 
Does anyone else ever fantasize about constructing network traffic analysis tools from raw sand and copper nuggets in order to determine if the entire internet tech stack has been compromised by ken thompson hacks, or is that just me
 
Firefox also has an integrated tracking blocker, which works like a charm. Embeded tweets and instagrams and facebooks don't show up on websites.
neither do facebook avatars in chat
 
12:44 PM
I have about three hundred facebook-related domains in my hosts file that I've redirected to localhost. They probably promptly countered my measures by releasing a 301st tracking domain, but I'm leaving it up for the principle of the thing
 
I've started just using the pihole server as a dns blocker & caching system
It's far from perfect, but it's still doing some decent blocking
 
12:59 PM
@PaulMcG hey, I got bit by that a month ago. Juniors kept complaining about certain test cases that would break in CI (which runs containers that use UTC) but would work on their workstations. Found a scary amount of bugs that way that had already been deployed. whoops.
three of my formative experiences as a dev: text is hard, packaging is hard, dates are hard.
5
 
Especially these days. Ha!
 
I'm glad the only datetime code I'm responsible for just needs to be accurate within 72 hours or so
widget.creation_date: mid-march-ish
 
haha, now I want to add "[...], maybe" to my output. just the natural consequence of defensive programming, really
 
I still practice diligent paranoia whenever anyone in here asks a datetime question
I think I averaged about 0.7 disclaimers per advice unit when the last guy wanted to put timestamps in his db
"I tested it and it looks like it works," he said. "I hope it also works for your users in UTC 8+<your timezone>," I thought, but I didn't say it because it was nearly time for lunch
If nothing else i tried to provide him tools for un-shooting his foot later
 
1:39 PM
*creates new package* oh, pyproject.toml still isn't standard *goes back to square one*
Not sure if this is Python packaging or purgatory...
 
@MisterMiyagi but pip uses it, does it not?
I could define build-time dependencies with it, despite setuptools
 
1:59 PM
@AndrasDeak yeah, some tools use it and others just... don't.
flake8 being one contender.
 
2:18 PM
I am trying to add a % sign to float cols in a pandas dataframe.
Cols data type is float64.
Several cells are empty.

I get an error:
ValueError: Unknown format code 'f' for object of type 'str'

Anyway to stop considering empty cell values while adding a % sign?

My code:
df[['col_1', 'col_2']] = df[['col_1', 'col_2']].applymap("{:,.2f}%".format)
 
2:29 PM
Hmm, I'm surprised that an empty cell would cause an error with that message. Can you make a MCVE?
 
2:56 PM
@Kevin I'm surprised if only some elements in a column are str
 
Yeah IME dataframes are picky about polymorphism
"waah, None isn't a float". Silence, library. They both inherit from object, that should be good enough
 
3:31 PM
Thinking about the halting problem again lads
 
Still haven't finished yet?
 
I say it'll take him 10 years
 
My response to that question is undecideable
 
Woot! My RSA token started with 314... today is going to be a good day.
 
today is your day. Be there, and be squared
 
3:36 PM
Random trivia: My fanperson'ism for Rust has skyrocketed after learning they have a never type that expresses whether something does not finish. I expect for Python this would remove about 20% of newbie questions on SO.
 
I started by thinking about the Ken Thompson Hack. A compromised compiler can invisibly compromise a wide selection of compilers and diagnostic tools, but it must use a whitelist or heuristic to determine which source codes generate those tools, and where to insert its trojan. Unless it can solve the halting problem, there must be at least one compiler whose source evades this detection and compiles into an uncompromised executable.
It's easy to write a "pathological program" that foils a halt-detector, but only if you have access to the detector's source code. You can't foil a KTH-compromised compiler in a similar way, because by definition you don't know the real source code of any KTH-compromised program
 
squints at Kevin are you still talking English?
Joking aside, I didn't follow the second message of the two - can you explain it in smaller words?
 
3:53 PM
@Kevin people buy old shipwrecks and dismantled bridges for low-background-radiation applications. You could try salvaging a compiler that was compiled before the inception of the KTH. And dig up contemporary hardware while you're at it.
How hard can it be to compile a chain of legacy compilers on legacy OSes running on gradually less legacy hardware?
 
Relatively straightforward until you get close enough to a modern platform that the CIA decides to perform the Five Dollar Wrench hack on your knees
 
they never like competition, do they
 
@OldTinfoil Hmm, I'll try
 
and he was never heard from again
 
@Kevin @AndrasDeak Thank you for your response.
What is MCVE?
 
4:04 PM
In short, I think your column has object dtype which is bad form
we need an MCVE to see what your data looks like to tell you how to fix it
 
ok. thank you.
let me send my code over.
give me 10 - 15 mins.
 
^^ also look at stackoverflow.com/questions/20109391/… for help in creating the problem.
 
@LinuxUser only send your code if I can run it and get the data you have. Otherwise spend a lot more time on it to have something small and self-contained.
 
(I often find that fix the problem when crafting an mcve, so it's a good practise to get into when you get stuck)
 
4:20 PM
Halt-detectors and compilers compromised by the Ken Thompson Hack are similar kinds of programs, because they both analyze source code and try to determine some quality about their behavior. A halt detector asks if the program will halt; a KTH-compromised compiler asks if the program is a compiler, and where in the source code it can insert evil instructions before compiling it.
It has been formally proven that you can never write a bulletproof halt-detector that works on all possible source codes. An adversary with knowledge of your detector can compose source code that your detector will fail on. Likewise, an adversary with knowledge of your KTH-compromised compiler can write source code that will be compiled without any evil instructions being inserted into it. [citation needed]
KTH-compromised compilers are very very good at hiding information about themselves, so it's hard to write a program that conclusively avoids being compiled with evil instructions. I'm currently brainstorming about ways that a clueless adversary can foil a KTH-compromised compiler, or at least be a nuisance
 
Is there some common name for "range including its end"? I've just realised that naming my function xyz_range might imply the end isn't included.
 
Thanks for coming and writing that up! Much appreciated
 
I believe you can flip the argument around and say that it's impossible to write a program that is immune to all KTH-compromised compilers. If all else fails, the evil compiler writer can read about your miracle program in the news, and add if source_code == miracle_program: insert_evil_on_line_ten()
 
@Kevin Since this means you are deciding whether a KTH will decide whether your program is a compiler, isn't that like a double halting problem?
 
Yeah. a KTH definitely cannot correctly detect the compilerness of all programs. But it would be a frightening proposition to tangle with even a KTH with a 50% success rate on "regular" programs that an average programmer might write
 
4:31 PM
As a hunch, there aren't that many compilers around that are worth targeting. Checking whether the first N bytes include a GCC, LLVM, or <whatever Windows uses> preamble/license should be a viable heuristic.
Especially in the case of GCC. Looking at you, GPL.
 
the C2 wiki suggests that 99% of useful compilers use lex/yacc, so it would be quite easy to detect anything that's trying to parse a language
... But perhaps if you were writing a KevinScript compiler, the KTH would think "ok, this is definitely a compiler, but I've never seen the target language before. where do I insert the evil?"
So that's one avenue for becoming a nuisance -- Take N hours to write a custom language, and then a CIA junior developer has to spend N*2 hours writing a custom evil inserter for you
Now we just need a program that generates one million custom languages per second...
A different kind of timing attack: obfuscate your source code somehow so its compilerness is possible, but very expensive, to decide. A good and pure compiler will compile it in 1 ms, a KTH will churn for fifteen minutes
avoid if source_code == miracle_program: insert_evil_on_line_ten() style detection by encrypting* with a different key every time, so simple string comparison fails
 
Okay, so could this be mitigated by using something like the public key infrastructure? The "goodies" write a known compiler with a particular checksum. Theoretically, a human could read the bytes and verify the compiler is legit.
Admittedly, I wouldn't want to be that human.
 
(*or something like encryption that still makes it possible for the compiler to hand you an ostensibly useful executable at the end. Homomorphic enryption may or may not be a thing here.)
@OldTinfoil I'd say, mitigated yes, definitely foiled no. This is true of almost all counterstrategies.
"bad guy foiled in all circumstances where the bad guy isn't willing to spend a trillion dollars and ten googleflops of processing time" would still be A+ work though
 
4:48 PM
I guess if we're talking about a state level actor, poisoning the PKI would be relatively trivial in comparison
 
"make the bad guy pony up" is basically what SHA-512 does, right? It's always breakable as long as you can spend 2^512 seconds brute forcing it.
 
I mean, that's the principal of all security ever invented. Finding that line between security and convenience that has an acceptable level of risk.
Door keys are fine, but it's possible you and your neighbour buy the same lock and happen to get the same/similar key for unlocking. Chances of that happening were low
 
@AndrasDeak @Kevin

Here is my code:
https://codeshare.io/aVz778

Error is on line 25.

I am trying to compute mean on values containing %.
Thank you for your help as always!
 
Mm hmm, the A++ approach we're looking for here would let us do required_resources_to_crack_our_program = entire_world_gdp + 1
 
That's a really big prime
 
4:54 PM
@LinuxUser Thanks for the mcve, I'll take a look now
I don't know if this is the idiomatic way to do this, but how about this:
mean = df.append((df.agg({"% Complete": "mean"}).rename("Mean")))
mean["% Complete"] = mean["% Complete"].map("{:,.2f}%".format, na_action="ignore").fillna("")
print(mean)
I took fillna off the first line, so map knows that there are rows it should ignore. After the formatting is done, it finds all ignored NaN lines and turns them into empty strings.
 
yeah, the problem is that you're using an empty string as a placeholder in a numeric column, changing the dtype from double to object
 
(Oh also na_action didn't work on my machine until I upgraded pandas so you may need to do that)
 
@Kevin @AndrasDeak
Thank you sir!

Solves my problem.
Kudos to both of you!
 
One thing I'm concerned about -- at the end of this, I believe the series will still have a dtype of object, because at one point it contains both floats and strings, and it's not smart enough to notice that it's 100% strings at the end. I wonder if there's a way to coerce it into having a dtype of string.
... assuming dtype str is even a thing. Maybe that's just for numeric types? I don't know, I am a humble yak shaver
 
@Kevin there are new-fangled string dtypes in pandas, but originally it was dtype
it's fine if the end result has object or str dtype, because it contains strings with percentages
 
5:08 PM
Ok, cool. I suspected the end result dtype wouldn't be a super big deal. It's not like we're trying to do fourier transforms on it or anything, it's probably just going to get written to an excel spreadsheet
 
Is there an equivalent of chr for bytes? ord works for both str and bytes, but chr does obviously not.
 
pandas.pydata.org/pandas-docs/stable/user_guide/text.html#text seems to indicate that the newfangled string dtype is mostly for the benefit of people that love strong typing
My feelings towards strong typing resemble an understated taupe, so I will leave it to the reader to decide if it's worth using
@MisterMiyagi I usually do bytes([x]) which is functional but it does not spark joy for me
 
Was about to say ^
bytes.fromhex perhaps :P
 
I wouldn't hate a bchr() in the stdlib, devs please do the needful
 
@Kevin It'll do. As long as it keeps me from falling to the easy path of supporting only str, it's fine.
note to selve: figure out whether parsing "sequences of frozensets" has practical applications
 
5:15 PM
TIL that ord() works on byteses. All this time I've been doing the_bytes_with_length_one[0]
 
My assumption is that it's unintentional, TBH.
Too much work to forbid.
 
Myself, I'm weakly confident that they went out of their way at least a little bit to make it work. To the source!
builtin_ord has separate cases for strings and byteses. That's approximately what I was imagining.
I notice that their error messages only mention strings. Surely ord(b"ab") should not crash with string of length 2 found, because b"ab" is not a string. This is an outrage!
I'm chalking this one up as "intentional, but undocumented, but stable for the last decade so you can probably go ahead and depend on it, but maybe don't if you're coding a space shuttle"
 
There are some... things there that I rather would not have seen.
 
5:31 PM
Baseless theory: possibly this code is a throwback from 2.7 when it made sense for ord to accept both str and unicode arguments. Then they ported it to 3.X by making the minimal changes necessary to accommodate the switched-around type names. It didn't occur to them that the newly-christened bytes wasn't quite as string-like as it used to be, and maybe shouldn't be ordible any more.
The one time they have carte blanche to cause backwards incompatible changes, and they don't do it... Grumble grumble
 
@Kevin It's just you :P
@Kevin Lucky indeed. I had to work on code that maintained biometric stream tracking and reporting across international teams who travelled. That was, shall we say, interesting.
@AndrasDeak Yeah, took flake8 out of my local pre-commit hooks because of that.
 
It's good that it's just me, because nobody can come to my secret island where I build verified-to-be-uncompromised code analysis tools. Any one of you could be a sleeper agent from the gubmint.
 
5:54 PM
@Kevin Probably overcooked.
mean = pd.to_numeric(df['% Complete'].str.strip('%'), errors='coerce').mean()
df.at['Mean', '% Complete'] = f'{mean:0.2f}%'
 
Neat
 
@piRSquared @Kevin @AndrasDeak If I want to code like you guys, from where should I take training? :)
 
The streets B-)
 
Try to answer questions on Stack overflow (-:
 
hello!
 
6:00 PM
cbg
 
The learning process I have employed so far is "be gradually less bad over the course of 20 years"
 
i'm having trouble with gunicorn, flask and dotenv
 
Recommended if you have 20 years that you aren't using
 
for whatever reason it's just not loading my .env file
i have a load_dotenv() in my config.py but it doesn't seem to be doing anything
or
do i have to load_dotenv() in each and every file i want to use the values from the .env file?
because it seems like those i saved in flask settings work but those not there don't work
 
No, you need to load them into your config when you start the app
It's a bit too abstract at the moment to help properly sorry. Are you sure that you're loading your env vars at all?
 
6:08 PM
well
after some testing, it only loads them for config.py
accessing those vars outside of config.py doesn't work
which is disappointing
 
Oki doki, sounds like you're most of the way there then. Do you have a Config class in your config.py file?
 
no, not really
 
So how do you register your config params on the app object?
 
app.config.from_pyfile('config.py')
 
Ok. I don't do it that way so I'll need to check the source for a min. I'll get back to you
 
6:11 PM
nah, i'm sure the problem isn't flask, but python-dotenv
it seems to only load the environment variables for the module it is invoked on
 
It can't be if you're telling me that the vars are available in config.py (which is exactly where you want them)
 
not all of them are assigned to a constant
because there were some i wanted to access directly rather than from flask's app config (because there are some things that are done outside the flask app context)
i think i know what i'm going to do, even if it's a bit hacky
 
Then have you tried load_dotenv(find_dotenv()) to traverse the directories?
 
yes, but i think i figured out that isn't the problem
 
Well, you're not making it easy for me because you've basically raised an issue and decided you know what to do about it, so is there a point in me trying to dig into this further?
 
6:14 PM
well, the thing is
the .env variables seem to be only accessible from config.py
accesing os.getenv to a variable defined in that .env doesn't seem to work
if i do it from outside config.py
which is disappointing
 
Ok, well I'm sure you're on top of it. You've basically repeated yourself verbatim so we're not going to move forwards
 
hmph
it seems i didn't notice it when running it via flask run because flask run loads .env files to the local env
 
Ok
 
which leads to another question actually
is there a good way to get an .env file to my local environment?
like, load all those variables as env variables
 
hello, is there is a way to query items with id bigger than 5? fo example ?
ex:
`x = class1.query.filter_by(id>5).all()`
 
6:21 PM
@F.O.Senn something like load_dotenv(find_dotenv()) as I suggested and you brushed off, perhaps?
x = class1.query.filter(class1.id>5).all()
 
@roganjosh thank you so much !
 
@roganjosh i tried that, but it didn't work
 
Specifically with find_dotenv()? What do you get from for k, v in os.environ.items(): print(k, v)?
 
i'll try that
 
Is it finding another .env file first, for example? (someone at work had this exact issue yesterday that they'd accidentally created an empty .env that happened to be found earlier in the search path of find_dotenv())
 
6:26 PM
huh, it prints the vars i want in config.py but not from other .py files where load_dotenv isn't being called
 
my productivity today is very cold... sub-zero
 
Well it won't do, because you need to use load_dotenv() in every file that you want them that aren't in the context of the app
If you'd let me finish my search, I would have told you how to access them within the app context and also mention how to make it work outside of the app context. This is not "disappointing" it's how namespaces work
 
There's an entry in Charles Darwin's diary where he says he can't get anything done today and he hates the world, and he wants to stay in bed and have a pint of ice cream. I may be paraphrasing at the end there. Thinking of this is some comfort to me when my productivity is best measured with a microscope
 
That's it :-)
 
6:36 PM
wonder how far back in human history you'd have to go to predate the universal notion that "I'm so stupid"
 
the first amphibian sets foot on land and is like "this is an excellent idea" and it's all been downhill from there
 
Everyone has berries and I have pine cones, "Ugg so dumb"
 
I didn't realize that happened on a mountain!
oh no wait... I'm an idiot
 
the second amphibian be like "why can't I do something as cool as when my dad set foot on land, there must be something wrong with me"
 
I kinda understand the reasoning why dict(k=v) does not take the previously defined value of k, but gosh, I long for that functionality
 
6:49 PM
k = 'i'
v = 1
print(dict(k=v))
print(dict([(k, v)]))

# {'k': 1}
# {'i': 1}
 
In [49]: k = 'q'

In [50]: dict(k=3)
Out[50]: {'k': 3}

In [51]: v = 3

In [52]: dict(k=v)
Out[52]: {'k': 3}
perhaps a versioning thing?
I'm on 3.8.5
 
<- 3.8.8 but I'm seeing the same as you
I'm saying pass a list of tuples instead
 
oh no wait! Lists. Damn! I'm an idiot. Yes. I'm aware of the iterable constructor syntax, but I just wish I could toggle the **kwargs syntax, which would be impossible by definition
 
I pass **dict(zip(list_o_varnames, list_o_values)) all the time
 
@piRSquared Fourth line should be print(dict([("k", v)])) if you want "k" as the key, vs. the value of the var "k".
 
6:57 PM
paint a picture of what you're looking for and we'll torture it into existence.
@PaulMcG that was the behavior I was trying to highlight
rbrb
 
7:25 PM
{"k": 1, "k": 2} and dict(k=1, k=2) should produce the same output. Only three quarters joking.
 
7:45 PM
>>> dict(k=1, k=2)
  File "<stdin>", line 1
SyntaxError: keyword argument repeated
>>> {"k": 1, "k": 2}
{'k': 2}
 
8:00 PM
I think that's his point
 
 
1 hour later…
 
2 hours later…
11:05 PM
@smci jeez; what was scipy targeting with that implementation I wonder?! It's nice and stable, but when is it gonna win out?
 
11:25 PM
I've got a pandas question that's really irking me. It feels like it's a Pandas 101 issue but I can't seem to implement it. I have some really noisy data and buried within it are blocks I want to slice out. I basically just want to grab the portions between the first instance of “delivery” and the last instance of “delivery” (inclusive) - removing things on either side of these two events - for each driver on each day.
df = pd.DataFrame({'datetime': ['2021-03-10 08:00:00',
                                '2021-03-10 09:00:00',
                                '2021-03-10 10:00:00',
                                '2021-03-10 11:00:00',
                                '2021-03-10 08:00:00',
                                '2021-03-10 09:00:00',
                                '2021-03-10 10:00:00',
                                '2021-03-10 11:00:00'],
                   'driver_id': [1, 1, 1, 1, 2, 2, 2, 2],
                   'status': ['garbage_data', 'delivery', 'interesting', 'delivery',
 
Can you cumsum equality with 'delivery' and take rows with odd counts?
 
I've found this but I don't know how to get the equality check for "delivery" in to a groupby. I do have a working solution suggested by a team member but it requires 2 new columns
 
hmm, "each driver and each day" is probably what makes this difficult
 
Funny you should say that. This is what was suggested:
df["pre_delivery"] = ~(df["status"] == "delivery").cumsum().clip(upper=1).astype(bool)
df["post_delivery"] = ~(df["status"].iloc[::-1] == "delivery").cumsum().clip(upper=1).astype(bool).iloc[::-1]
The final piece being df.query("not pre_delivery and not post_delivery").drop(columns=["pre_delivery", "post_delivery"])
Which is not bad but it just feels like I'm missing something
 
there's probably an eleganter way than this:
In [13]: df[((df.status == 'delivery').cumsum() % 2 == 1) | (df.status == 'delivery')]
Out[13]:
              datetime  driver_id       status
1  2021-03-10 09:00:00          1     delivery
2  2021-03-10 10:00:00          1  interesting
3  2021-03-10 11:00:00          1     delivery
4  2021-03-10 08:00:00          2     delivery
5  2021-03-10 09:00:00          2  interesting
6  2021-03-10 10:00:00          2     delivery
my simple cumsum suggstion misses the closing "deliveries" so I had to include those manually
 
11:40 PM
.... <head explodes>
 
trivially more elegant by only computing that boolean series once
 
How can I not break that? Wow; ok. I think you've sussed if for me
 
if drivers get interleaved it will break
but your MCVE had them ordered :P
 
Thanks Andras. I need to re-evaluate this a bit. I didn't think the solution was going to look like that but you've covered my basic concerns
 
(and the suggestion you showed would probably break all the same)
no worries
 
11:42 PM
I can pre-sort the df on driver ids and dates in any order I choose
 
yeah, I figured
 
Yeah. This is a classic case where my brain has gone down a rabbit hole of groupby but it's a keep-it-simple, stupid case. Just sort the data. Massively appreciated!
 
I can easily think outside the box because I don't know what's in the pandas box :D
 
Can't argue with that one! Keep your brain pure; no need for the bajillion pandas recipes to be baked into your brain! :P
 

« first day (3807 days earlier)      last day (1145 days later) »