« first day (4273 days earlier)      last day (658 days later) » 

12:18 AM
Is there some good way to check if a python library has all the doc strings? Does mypy or pylint enforce any check like this?
 
12:30 AM
Not exactly what you're looking for, but I've written a function that checks the public members of your module against the entries in the sphinx inventory
Code here, shouldn't be too hard to fill in the blanks
 
What do you mean by "all the"?
i.e., what rule do you want to use, to decide which things should have a docstring?
 
Avv
hello guys,

for zeromq library (0mq), it uses `socket.bind("tcp://*:%s" % port)` to assign a port and an IP to a socket, but I am not sure what is `tcp://*:` please?
 
Avv
thank you
 
1:09 AM
@KarlKnechtel I think the code I posted just takes everything it can find in your module, even if it has a name that begins with _
 
 
8 hours later…
8:55 AM
a = {"name": "geetha", "age": 20}
b = a
a["name"] = "drishan"

print(a["name"])  # output : drishan
print(a)  # output: {'name': 'drishan', 'age': 20}
Why a's value is getting changed here?
 
Because assignment does not create a copy
 
ok. Is there any way to force it to copy?
 
Do you know how to create copies of dictionaries in Python?
 
Is it about copy() ?
 
@owgitt Yes. It's named copy.
 
8:59 AM
@vaultah and @MisterMiyagi Thank you
 
you are changing a the usual mistake I make is not knowing it changes b
 
True. Did you actually intend to ask why b (the one not explicitly modified) is changed?
Because that's what me and probably vaultah as well assumed and answered.
 
if this was an interview you all did not make it, its a gotcha question
jk, anyways for why I logged in, is there any library for easy time series manipulation?
 
Sorry. There was a typo. Second and third lines supposed to be :
b = a
b["name"] = "drishan"
Anyway you guys answered my question. Thanks a lot
 
If I have a time series like this, how can I make sure I shift the other rows based on sliding window (assume the window size is 2 in this example) and wait time?
data = [
    [datetime(2022, 6, 28, 9, 0), datetime(2022, 6, 28, 9, 10)],
    [datetime(2022, 6, 28, 9, 5), datetime(2022, 6, 28, 9, 10)],
    [datetime(2022, 6, 28, 9, 6), datetime(2022, 6, 28, 9, 9)],
    [datetime(2022, 6, 28, 9, 7), datetime(2022, 6, 28, 9, 11)],
    [datetime(2022, 6, 28, 9, 8), datetime(2022, 6, 28, 9, 15)],
]

df = pd.DataFrame(data, columns=['start_time', 'end_time'])
    that is my (expected)output I am kind of lost on how to make it to

           start_time            end_time
0 2022-06-28 09:00:00 2022-06-28 09:10:00
1 2022-06-28 09:05:00 2022-06-28 09:10:00
2 2022-06-28 09:10:00 2022-06-28 09:13:00
3 2022-06-28 09:11:00 2022-06-28 09:15:00
4 2022-06-28 09:15:00 2022-06-28 09:22:00
 
9:15 AM
pandas has some datetime methods that might work well, like datetimeIndex combined with setting the right windows.
ah but you're already using pandas, so that's less than useful advice
 
I did come across that, I have found how to add the time difference, I just can not seem to find how to make the "sliding window"
 
Not sure what you're trying to achieve here; how does the window size affect things in practice?
From the looks of it, you want to eliminate overlap between entries that are not adjacent?
 
you can think of windows size as means of "available concurrency", so if its 2, it means I can process 2 rows at a time, if its 3 I can process 3 rows at a time
so if my window size were 5, then my input and output will not change (for the above)
@MisterMiyagi in a way yes, I want to make sure the overlaps are avoided for a given window size
 
I think I'm still missing something here, why in your expected output, row 3 does it start at 9:11 instead of 9:10?
 
Hm, so for a window size of w, you need to enforce that df.iloc[n].start_time >= df.iloc[n-w].end_time?
 
9:21 AM
row 3 initially had to start at 9:06, but since my window size (concurrency limit) is 2, it had to wait for 4 more minutes before it gets picked up @IvoMerchiers
 
I guess that one way or another you'll be looking at an iterative (loops) solution, simply because the logic is as such.
 
I guess loops it is, I am trying something with collections.deque to figure out if I can get this thing working
 
A regular list should be fine. Just modify the "current" element to suit your needs.
 
@Jake but at 9:10:01 you only have one thing running, so I would expect row with nr 3 to be able to expect that
 
There is nothing in there needing double-ended operations.
 
9:24 AM
But yeah, a loop is probably the way to go here
 
is there some name for such problems? at least I can look up something? but in any case, thanks all
@IvoMerchiers yes it should be what you claim it is, I failed to take this into account
 
If you're operating on the initial list, then something like this should work:
window = 2
for idx, (prev, this) in enumerate(zip(data, data[window:]), start=window):
    if this[0] < prev[1]:
        this[1] = prev[1] + (this[1] - this[0])
        this[0] = prev[1]
 
damn, that is elegant, I guess I can use this as a source, thanks a bunch
 
That's not optimal in some cases, though.
If you want to be thorough, you have to check the entire previous window instead of just its first entry.
 
25k - 400k elements is what I am looking at, but its not something that will be run frequently, so I can afford the loss, and will make the "compare all previous" change as well
 
9:36 AM
@MisterMiyagi kind of looks like you should be using idx - that or not sure why the enumerate... ?
 
@JonClements Oh, you're right! I only realised the elements could be modified directly when I was already halfway through writing the code.
 
figured that might be the case - was just checking :)
 
the more I look at this solution, the more I realize I was looking at wrong things, I looked at stuff like pandas.pydata.org/docs/user_guide/window.html and pandas.pydata.org/pandas-docs/stable/user_guide/timeseries.html, and this solution just does that in pure python
 
10:21 AM
Hey guys, I am trying to do some webscraping. I am working on 1 page and I want to access another one by a link. In HTML as you know I have a line as follows: <a href="https:www....."> without any other information. How can I access and click on the link on that line?
 
10:32 AM
Depends on what library you're using.
 
I am using requests,urllib.request and beautifulsoup so far
 
Ok. Beautifulsoup can find the href inside an A element fairly easily. I can look it up in a couple minutes.
 
Thank you for your help, I also added a screenshot!
 
What should happen if the page has more than one A element? Just get the first one?
 
yes!
I think I found the answer:
listing_a = all[0].find({"a"})
url_listing = listing_a.get("href")
with all[0] being the first listing out of all the listings the webpage has, and then I save the URL into another variable. I thought about it and I don't need to actually open the URL, just save it to access it later on
 
10:56 AM
obj.get("href") does roughly the same thing as obj["href"], so that should be fine to use here. Essentially, it gives None if the link has no href, rather than crashing with a KeyError.
 
Thank you for your help, I will keep it in mind!
 
I don't think I've seen .find({"a"}) before. I didn't know you could use a set for the first parameter. If you're only searching for one name, it's probably unnecessary. .find("a") would do the same thing.
But it is written: if it works, it works
 
11:30 AM
@Jake The non-loopy version that you probably won't want to use now
def add_window(df, window):
    df['group'] = df.index // window
    df['group_start'] = df.groupby('group')['start_time'].transform('min')
    df['group_end'] = df.groupby('group')['end_time'].transform('max')
    df['next_start'] = df['group_start'].shift(-window)
    df['lag'] = np.where(df['next_start'] < df['group_end'],
                         (df['group_end'] - df['next_start']).dt.seconds,
                         0)
    df['lag'] = df['lag'].shift(window)
    df['lag'].iloc[1::window] = 0
I also probably wouldn't want to use that, but I created it so it could be a thing
 
Does that really work? The original problem specification sounded very "depends on earlier iterations" to me.
then again I couldn't understand in three minutes of reading so I started doing something else :)
 
It does give the exact output they asked for, but I too and not entirely convinced there isn't a hole in it
Well, the exact output plus a zillion superfluous extra columns that need dropping
 
Kevin's already established the validity of "it runs on my machine" in this chat screenful, so it's fine
 
@roganjosh Whoa, that's scary nice!
Wondering if it actually runs significantly faster.
 
In the process of doing that, I once again realised that I sleepwalk into the nonsensical units of negative datetimes. I sure do hope I've not got something in production with whacky numbers
Yeah, I was also curious about whether it was faster, but my distraction-from-actual-fires time was up
Not negative datetimes, negative *timedeltas
 
11:42 AM
What's wrong with negative timedeltas?
 
It adds a full day portion of seconds on
 
ah
But "ten seconds ago" is a very well-defined timedelta. It might just break the rest of your code then.
 
It's nice and simple to spot in small examples like this, but it's not something I actively think about enough in my day-to-day
 
12:36 PM
@roganjosh thanks for this roganjosh, it does give me a SettingWithCopyWarning , probably in df['lag'].iloc[1::window] = 0 but I can find a way to fix it, provided I up my pandas game haha, but this is impressive
 
huh, it's not giving me any warning
 
let me check, but in the mean time I am on 3.10.4 with pandas 1.4.1
 
Oh, actually, it is, I just expanded the printout to show all columns
 
:), it is from df['lag'].iloc[1::window] = 0
 
It's spurious in this case but I'm not actually sure what it wants from me there
 
12:41 PM
I never understood that warning, I have a couple of low effort pandas scripts I wrote, and so far this has not affected anything
I do throw in loc and iloc expecting it to go, but at one point I just give up, should learn what it actually means if I am going to do some serious pandas work
 
df.loc[1::window, 'lag'] = 0 removes it. I find that syntax less intuitive, though and I wouldn't be surprised if it was slower
 
How would it be slower if pandas can tell the final intent instead of doing so in 2 steps?
More checks perhaps?
 
Yeah, I've just thought better of it
 
I was trying the other way round, and was getting a TypeError
 
It's always [row index, column index].
the pandas-weird part is that df[index] gives you a column if it can
 
12:50 PM
put it that way it seems clear to understand
fwiw, roganjosh if you add [datetime(2022, 6, 28, 19, 8), datetime(2022, 6, 28, 19, 15)] (append to the input), your solution seems to shift that time as well, Miyagi's solution does not do that
fixing that is my work, you guys have done more than I asked for :)
 
Like I said, I wouldn't be surprised if this needed a loop.
 
yeah, Miyagi's solution with a few custom changes seems to get me where I want, I was just pointing out
massive respect for the pandas attempt
 
It just needs a correction factor to the lag time to account for non-overlapping windows
I don't see why it needs a loop when only adjacent groups matter and you can propagate the lag time forwards
Whether or not a loop would be more sensible is a different question, I just want to prove to myself that it isn't strictly a loopy problem
 
if only python loops were fast
its fast enough most of the time, so ¯\_(ツ)_/¯
 
@roganjosh because of "propagate the lag time forward". But as I said I didn't fully understand the specifications, it just smelled like something that depends on earlier rows.
 
1:05 PM
Yeah, it does, but only one "group" back
 
But if one group bumps up the next group, can't that bump up the group after that?
I don't want to waste too much of your time here, just trying to explain my vague notions
 
Yeah, but then you run cumsum() up the lag column to propagate it forwards
 
prior to df.loc[1::window, 'lag'] = 0, you have the maximum pushback generated by the previous group. I wipe out all but one of those values, then cumsum it forwards
The problem now is that there is no negative lag for when adjacent groups have a gap in them, so that needs to be accounted for
 
1:09 PM
Jake's corner case does raise another interesting point, though. Since that completely isolated timestamp that's added isn't even close to the other time windows in its own group, it probably shouldn't be pushed back, but the lag is applied on the group level
 
cumsum sounds like a weird mathematicians sex practice :D
 
1:52 PM
hlo
 
2:14 PM
In between model runs, I had some time to go a bit wild. Closed both corner cases @Jake. If you set as_group=False then intra-group pushbacks for outliers will not be pushed back if it's not needed. The gauntlet was thrown down by Andras, I blame him for my "creation"
 
Today I wrote a sql GROUP BY clause with 132 fields. I could clean it up, except the deadline was 16 minutes ago, so...
 
There's a bit of creative license in its creation story. It's a wildly over-exaggerated response to "I think it might need loops" :P
 
2:47 PM
Cbg. It's been a while.
 
Well hi there :-)
 
Hello, Dr Snape
 
Afternoon @Kevin, @vaultah
I have a new job. With lots of Python and lots less distracting admin. Which is nice
6
Nice to see Kevin still dominating star collection in the side bar :D
 
I've slowed down a bit, but I get by
Update to my XY problem from yesterday: starting from Jon's approach, I've put together a working prototype for finding the important curly brackets in a format string. pastebin.com/aspUpwZA
I tested it on exactly one input, and now declare victory
 
@JRichardSnape Long time no see... that does sound good :p
@Kevin Sounds reasonable to me :)
 
3:02 PM
100% success rate, may as well retire on top
I looked at the source code for Formatter.parse, and I don't think I've missed any obscure corner cases. I'm not 100% confident though, because it uses a recursive state machine that hurts my brain if I think about it rigorously.
 
Think is for fools anyway... :p
 
<grunt of agreement>
 
4:01 PM
TIL that liminal is a word. I mean, it should be, given that subliminal is, but I don't think I've ever heard it used
Reminds me of a podcast I listened to a few weeks ago where they were debating what a "honcho" is. Like, can you have a Deputy Honcho? I'll never know/I might know in 10 seconds now that I've reminded myself
 
4:16 PM
@roganjosh Holy Panda, Batman!
tips hat in awe
 
Thank you, thank you :) After I'd done it, I realised I'd attempted almost exactly the same about 6 months ago and made a right hash of it because I was rushing for a deadline. I think, deep down, I had some underlying redemption arc that needed to play out
All culminating in praise from Miyagi! The highest honour!
 
hey guys, I am a newbie at DL using Tensorflow, anyone here knows any good courses with which I could learn LSTMs ( and preferably, how to use Images in them?)
 
4:52 PM
I've heard the term "liminal", but only in the context of "liminal spaces". Thing like elevators, airports, hotel hallways. Places that only exist to get you to more interesting places.
I recall seeing the term floating around on social media for a year or two, and then "The Backrooms" made its debut
Wikipedia says that The Backrooms inspired the liminal space trend, not the other way around. That's not how I remember it. It seems fitting that a creepypasta about the unstable regions of spacetime would have an uncertain origin.
 
It just struck me as really strange - the person said "we're in a liminal position", which didn't make sense to me. It's odd that liminal can have generic use and yet subliminal has a pretty specific meaning related to psychology
 
and apparently superliminal is a video game...
 
Is liminal itself liminal in that it's a boundary between entirely different concepts? So meta
 
5:08 PM
I've played superliminal. Fun premise, decent execution.
 
5:21 PM
I'm looking for examples of python using magic strings. So far I have open('r'), int.to_bytes('little'), and array('u')
 
Perhaps struct.pack() and unpack()
 
Hmm, that's not so much a magic string as it is a mini-language
It must be something that could be replaced by an enum
 
The encodings are strings-as-enums, and especially the error handling strategies.
 
I've always disliked datetime.isoformat's timespec
 
What is a "magic string" exactly?
 
5:28 PM
codecs.encode(obj, encoding='utf-8', errors='strict')
 
meh, I consider encoding to be a value from some internal database (so not really "magic" IMO)
 
@JonClements A string that has a special meaning in a certain context. Like passing the string "w" to open makes it do something different
@MisterMiyagi Nice! I'm unsure if I'd consider encodings magic strings, but the errors certainly are
 
Ten minutes ago, my definition of "magic string" was something like "a string whose exact value is irrelevant as long as all concerned parties agree on it". open("r") could have just as easily been open("q") in some alternate timeline, and you'd still be able to write the same programs.
 
logging.Formatters have the three "%", "{" or "$" format styles.
 
@vaultah Wow, I sure had a hard time finding that function in the docs. Fits the bill, thanks
 
5:33 PM
How about the mode argument for compile ?
 
multiprocessing has the spawn, fork and forkserver spawn methods.
 
But such a definition would include mini-languages (struct.pack's symbol for unsigned long long could have just as easily been Z instead of Q) and internal-ish names ("utf-8" could have been called "unicodeEight"), so perhaps it's too broad.
 
Ok, I think I have enough material. Thanks all
 
tkinter's geometry manager lets you glue widgets to the top/bottom/left/right edge of the grid if you use the arguments "n" or "s" or "w" or "e". But you can also use tkinter.N or tkinter.S, etc, so you don't have to be magic if you don't want
 
I'm still confused after reading several sources. What's the distinguishing factor for the 'r' in open('r')? On its own, it does the same thing as open('s') in that it throws an error because of positional arguments; it doesn't start doing something unexpected. When specified correctly i.e. open(the_file, 'r') and open(the_file, 'w') I'm not sure what's "magic" about the different behaviour?
Otherwise I can't distinguish it from a string argument that only accepts a specific subset of strings that are tied directly to its resulting behaviour. That doesn't strike me as magic
 
5:37 PM
I was just too lazy to write open(mode='r')
 
But it still doesn't strike me as magic. I have a map server, so you'd specify config = Config(map_profile='truck') and it will only accept "car" or "truck", otherwise it's an error. Is that a magic string I've created?
 
@roganjosh magic as in magic number perhaps
 
I just looked up "magic string" on wikipedia and apparently it means something different than I thought
> a magic string is an input that a programmer believes will never come externally and which activates otherwise hidden functionality. A user of this program would likely provide input that gives an expected response in most situations. However, if the user does in fact innocently provide the pre-defined input, invoking the internal functionality, the program response is often quite unexpected to the user (thus appearing "magical").
 
I understood "magic string" as similar to magic number except it's string
 
5:41 PM
Tkinter has about 70 attributes that simply refer to a string similar to their own name: pastebin.com/J2V2wPFj
 
To rephrase my question: I was looking for function parameters that would be annotated as a typing.Literal
 
These sorts of curiosities, I chalk up to tkinter being the threshold between two languages with very different paradigms. You could almost call the module... Liminal.
 
@Aran-Fey Wat. That sounds like an accident waiting to happen.
@Aran-Fey You might want to grep though the typeshed.
Apparently, warnings also has some string constants for its actions arguments.
 
HTTP multipart requests actually use a magic string. You just insert a separator string and pray it doesn't occur anywhere in the actual payload
 
 
1 hour later…
6:56 PM
ah, yes, the internet, designed to withstand a nuclear attack
 
if it can handle magic string constants, it can handle nukes!
 
7:28 PM
... As long as the nuke isn't near Cloudflare HQ.
 
With Aran's first example of magic strings, I honestly thought we were just teetering on the edge of something really bad
def open(file_path, mode='r'):

    if file_path != 'r':
        return some_buffered_reader(file_path)

    immolate_filesystem()
 
The ol' "halt and catch fire"
 
@JRichardSnape cbg and congratulations :)
 
7:51 PM
@Kevin I meant to pick this up at the time but got distracted. Was this an effort to remove dupes?
 
8:21 PM
https://stackoverflow.com/questions/72792619

still looking for a canonical. this is asked constantly
 
That's a tough question to answer though. It's unclear if it should be a global constant or a return value or a parameter for both functions
And if it is a return value, is it an argument for function2 or does function2 call function1
 
8:39 PM
Is there ever a variant of these that actually is clear in what is expected? This is such a nonsensical request that it could mean tons of things.
 
Close - Needs details or clarity
Just as we can all easily inject the context into the question (we all know what the OP almost-certainly wants) we just encourage this kind of help-desk behaviour
And adding a dupe will not help anyone; it's searching on "our" side and the OP almost certainly won't be satisfied with the answer
 
stackoverflow.com/questions/10354163 This seems like a useful and underappreciated dupe target, e.g. for "I tried to set up a Tkinter callback and it ran immediately and doesn't run when I click the button"
@MisterMiyagi no, because nobody who understood what to expect could have the question any more. The fundamental problem is the misconception about a) how variables (and scope) work; b) how function calling works
I think e.g. "what strategies exist for communicating between functions?" is coherent phrasing for a reference
 
I don't think it is because I don't think they understand the fundamentals. As you were writing that, I left a comment to that effect
 
8:54 PM
as long as the site has the design problems that it does, I am perfectly willing to (ab)use dupe-hammering as a weapon against FGITW; and I think that pointing people in the position of "incapable of asking a coherent question due to missing fundamentals" at *some* form of reference is superior to turning them away.
Even better if the question is a plausible excuse to introduce said fundamentals.
(I did vote to close, though.)
 
"at some form of reference is superior to turning them away." I didn't. I voted to close and gave them a lead. That's enough, surely?
FGITW is just out of control. The fight is lost. Let them gain their internet points and.... spend them on... stuff?
I mean, honestly...
probably im gonna use globals because i need get a value from another function like this: "rnd = Random.new().read(16)" — CodexSC 42 mins ago
 
@KarlKnechtel Nah, I don't think that works. People can't - and honestly, they shouldn't have to - make the connection that y = some_func is equivalent to command=some_func. If the question is specifically about tkinter, the answer should be specific to tkinter as well
@roganjosh No spend, only collect!!
 
I see what you did. You put the real Aran's internets into a high-interest saver and then run around in a smurf for your day-trading. I really need to get my internets in order
 
9:12 PM
Well, I wouldn't exactly call it high interest
 
 
1 hour later…
10:32 PM
Is there some succinct way to assign typed kwargs as class attributes?

class Thing:
def __init__(self, a: int = 0, b: int = 0, c: int = 0):
self.a = a
self.b = b
self.c = c

I.g., is there anyway to replace all of the self.* = * with something more succinct?

I'm aware of the approaches from SO questions like the linked one below, but I want to be able to see the types and default values of the attributes during class instantiation.
https://stackoverflow.com/questions/8187082/how-can-you-set-class-attributes-from-variable-arguments-kwargs-in-python
 
@JacobBumgarner depending on your class, dataclass
 
Wonderful, thanks as always Andras. This really simplifies things
 
no worries :)
just be aware that dataclass has some weird behaviours in esoteric situations, and it might even have issues with multiple inheritance (my memory of this is hazy)
Apr 14, 2021 at 11:26, by Aran-Fey
I need some help figuring out a workaround for my dataclass problem. I have a class decorator that creates/replaces the class's __init__ method. This works fine when combined with @dataclass as long as you take care to use the decorators in the correct order. The problem arises when a child-dataclass is created: Because dataclasses don't chain-call parent __init__s, my class doesn't work correctly. Here's a small demo of that
 
11:16 PM
Wow, that moron posted a paste with a time limit
 
So selfish :/
 
https://stackoverflow.com/questions/72793805 look, it's the *same* question *again*
"how do I get access to the value computed in this function?"
 
What irritates me about dataclasses is that they make a bunch of assumptions about your class and force restrictions on you, instead of being a general-purpose tool for reducing boilerplate. For example, inheritance is barely supported; if you inherit from something it must be a dataclass and it mustn't have a custom __init__. And none of that is documented, you just find out the hard way when it goes kaboom in your face
99% of the time, all I want is a tool that generates an __init__ method for me. But @dataclass does more than that, it also gives you an __eq__ and overwrites your __repr__. And the __init__ it generates doesn't even call super().__init__(). It's designed for classes that hold data and do nothing else, and I don't understand why. It could've been so much more
 
11:47 PM
^ basically nearly every new feature in Python 3 of late
"It could've been so much more"
Also the "I don't understand why"
 
The stdlib as a whole is a hot mess, if we're being honest. The best thing about it is the sheer size
Lots of bad batteries included
 
can't wait to be back on working on the Rust project I left hanging for nearly a year now
 

« first day (4273 days earlier)      last day (658 days later) »