« first day (4313 days earlier)      last day (860 days later) » 

00:43
Would you rather target with .get() the variable/textvariable of a widget or the whole widget and ask with get() for the value, please?
01:33
@AndrasDeak--СлаваУкраїні I think there's an important conceptual issue for beginners there and the hammer is faster. and people will keep asking
@AndrasDeak--СлаваУкраїні I hammered this one, but actually it should be marked not reproducible given the example
(OP might also be misunderstanding the requirement for the code in the first place)
01:56
@AndrasDeak--СлаваУкраїні yeah, that's fair, but it'd be nice if more software was designed with human empathy in mind would make a better world
@AndrasDeak--СлаваУкраїні yeah, and if concrete identities are decoupled it would also allow the versioning of identities, should the person behind it choose to (e.g. change of organization they associate themselves with, be it employer or group, or if they got married and want to indicate a change of surname)
02:12
duplicate (was tagged 2.x only) stackoverflow.com/questions/45893519
I've been working on establishing stackoverflow.com/questions/3380484/… as a canonical. Does this look like the right place to add an answer that lists what sequences are considered escapes by Python?
(and explain on a beginner level what escape sequences are, why an un-doubled backslash in the string literal causes problems, etc)
02:57
also the OP of that question mixes the 'r' prefix while adding double backslashes
03:12
@KarlKnechtel I can't believe nobody (except the OP) caught that the OP was on Linux and the slashes should have been forward slashes.
03:29
Well, OP did in a self-answer, but accidentally corrected it while pointing at a different problem
Yeah, that question is a whole big pile of worthless. I added my delete vote, hopefully someone else will come along and take it out of its misery.
 
1 hour later…
04:37
Did something change with html.parser between 3.8 and 3.10? See this question, my answer, and the discussion below it. OP was getting a "ValueError - not enough values to unpack" when using html.parser with BeautifulSoup on 3.8. The same code worked fine for me with 3.10, but gave me the same error on 3.8. When I changed from html.parser to lxml, the error went away. Any clues why?
This is the only thing I've found so far, in the 3.9 release notes:
> The unescape() method in the html.parser.HTMLParser class has been removed (it was deprecated since Python 3.4). html.unescape() should be used for converting character references to the corresponding unicode characters.
stackoverflow.com/questions/56689741 should be duplicate of stackoverflow.com/q/24085680 (I misidentified it in the comments)
Hammered.
05:11
stackoverflow.com/questions/56907943 Please find better dupe for this
05:40
stackoverflow.com/questions/4025482/… This is useful and popular, but I have many concerns. (For one, it isn't actually tagged as python; not clear which languages have this issue)
stackoverflow.com/questions/47539025/… Here is another example of the stackoverflow.com/questions/56907943 question. It's the same problem as with backslashes appearing twice, but for close quotes.
We seriously lack a proper reference for understanding the difference between a string's contents and its __repr__.
stackoverflow.com/questions/66955142/… this is another example, I think, although it is bad
it might be appropriate to write a canonical from scratch this time.
so there are two canonicals that I think are missing here:
1. "Why can't I replace all the backslashes in my string?" (the string doesn't actually contain all the expected backslashes; OP wrote something like '\m\n\o')
2. "Why do I see backslashes in a string that shouldn't have them?" (more general than the "why do backslashes appear twice" canonical; the classic example here is a string that contains both double and single quotes, so Python uses single enclosing quotes for the repr and escapes the contained single quotes)
Also, see my comment here: stackoverflow.com/a/67488119/523612 this seems to happen disturbingly often.
stackoverflow.com/questions/64993076/… why on earth did this get +5 :/
06:20
Cbg. How do I retrieve Pandas dataframe rows that are consecutively equal to the values of a list?
06:48
I have now asked the question on the main site.
I just bookmarked stackoverflow.com/questions/12723751. It's surprisingly difficult to find this canonical for "how do I replace all instances of one specific character in a string?". I largely expected that we didn't actually have it due to RTFM-attitude.
07:03
stackoverflow.com/questions/50580002 I think this might be the correct target for the "why can't I replace all the backslashes in my string?" questions, with some fixups
07:17
Thinking about it further: I actually like it as the target for both questions above, at least once I've written a good answer for it ;) After all, the repr of a string uses (a subset of) the same syntax as string literals; backslashes mean the same thing in both contexts, just that Python will use a normalized representation.
The example is solid; it would be a huge overreach to call the question "unfocused"; and it admits a beginner-level explanation. I think I will go with this one.
07:48
TIL: if you use an octal escape in a bytes literal with three digits representing a value >= 256, Python will take it modulo 256. In a string literal, it just uses code points 256..511 normally.
 
1 hour later…
09:11
Even this didn't work, though direction hinted very well. Only when I added an empty buffer commit and manually fixed folder contents on which re-based there was finally correct check-sums.
Still were errors in the end of re-base, yet it worked.
git reset --hard was damaging something compared to manual backup copy, but thanks for hints!
git reset --hard is my legacy at work for the most damage done. I lost 3 days' work for a colleague with that suggestion. That follows me round forevermore
that's quite a good way to lose friends :p
Hey, it did exactly what he asked for on slack. I wasn't to know he hadn't committed in 3 days!
09:21
On balance, though, I'm glad that that's my legacy and I haven't managed to do something worse in over a year since (or at least that they know about!)
Gitgate
I think the worst I've ever done is cost a company I worked for about £3million quid. Was a direct marketing company and they used to purchase data lists and we use to filter out duplicates across them and remove dead people and people who had moved or people who didn't want to receive marketing...
And err... I borked the system slightly... it still did apply all the flags... but ultimately on output - it just ignored 'em...
I don't think I've lost substantial amounts of money. My misunderstanding about context managers and RDBMS connections took out our central cluster for all customers for a while but I managed to fix that pretty quick
The client was furious... especially as it was Which? which are consumer rights champions... :p
Did you make the front page of their publication? :P
Nope... just the front page of my manager's wrath after him having been pulled into the managing director's office... sighs
09:27
@JonClements how exactly did this end up costing the company money? lawsuits?
unrelated: I just found stackoverflow.com/questions/280435/escaping-regex-string. it currently has about 60 duplicate links; it's probably missing like 60,000 more
@KarlKnechtel unnecessary printing and posting costs, reputation damage and potential fines
Also useful, but not very well written and not actually about regex: stackoverflow.com/questions/2241600/python-regex-r-prefix
Funniest thing is... a week later I got promoted...
also useful, but possibly the wrong factoring for the idea-space: stackoverflow.com/questions/12871066/…
It all ended like one of those 90s American shows with us all laughing + freeze-frame. He was happy after he rewrote it because it was better and he knows not to go three days without backing up. I learned that people were happy to blindly follow my suggestions (I rewound within literally 30 seconds as I realised what I might have done but he'd already done it) so I'm more cautious about that now
it's a good lesson to learn though albeit it the hard way :p
"blindly" perhaps isn't the right word. But with less scrutiny than they perhaps should have. We still have our Fail of the Week on our Friday standup and I get some epic ones in, which is helping to show that 5+ more years of experience doesn't make me any less fallible. In fact I can usually have a bigger impact because I have access to more fundamental systems
11:08
stackoverflow.com/questions/2241600 Is there any decent version of this that isn't in a regex context? I.e., something that's specifically "Q. Why does '\y' work but '\x' not?"
 
1 hour later…
12:15
I have a folder on a network drive (let's call it \\file-server\shared) from which I need to choose an arbitrary file and rename it, without race conditions. (Multiple PCs will be doing this at the same time.) Is a Path.rename guarded by an except FileNotFoundError: enough?
On 2nd thought, I may have abstracted the problem a bit too much. The goal is actually to open a file that isn't currently opened by anyone else. Renaming the file just seemed like an easy way to do it
what a strange goal, hmm. i mean, doesnt this sound like a usecase for a simple lock?
ie. refuse the temptation to access the file directly, have your file behave like a shared resource that has a lock while the file is accessed, only released when the access is done
Well, I think my question is basically how to implement the lock
How can I tell whether the file is currently locked? And how can I atomically lock and unlock it?
i see. so, main thing being, its on a network, and the diff pcs are running their own scripts, essentially?
but all trying to access this one file somewhere
The same code, running on multiple PCs, all accessing the same network folder, yes
with the usual caveat that im essentially talking without any basis or practical experience doing these kinds of things: my knee jerk reaction is that there needs to be a single middleman that's doing the file access and managing locks, be it a script or whatnot. all your scripts need to go through the middleman to access the file, and that gives the opportunity for locking. there might be better ways to do this kind of work though
makes me wonder how useful those lock files are that ive seen some programs like libreoffice create when one program accesses something. feels like that would be suspect to race conditions imo
thinking more about it, its funny, in a way we dont even care whether its a file being read or not, all we need is a single lock that can coordinate between all instances. when the lock is acquired, then and only then the program does whatever, finishes with the file, and then releases the lock
12:30
I'm sure there must be a way to use the file server as the middle man
Currently I essentially do this:
for file_path in Path(r'\\file-server\shared').glob('*.pdf'):
    new_path = file_path.with_stem('.pdf.locked')

    try:
        file_path.rename(new_path)
        if new_path.exists():  # Just to be safe
            break
    except IOError:  # Maybe it's not a FileNotFoundError, but IOError ought to work
        continue
(The if new_path.exists(): wasn't there until 5 minutes ago, when I figured I might as well play it safe)
"On Unix, if target exists and is a file, it will be replaced silently if the user has permission. On Windows, if target exists, FileExistsError will be raised." source
i have very little faith this is atomic
but im not very confident at all. it seems to me like the unix version has a better shot at being atomic than the windows one
There really shouldn't be a file with that name, but I guess there's a nonzero chance that error could be raised because of race conditions
Why don't you simply have a mutex file?
12:41
ooh, new terms to google
Same. Are you sure that works on a network file system?
I suppose you're not in control of the code accessing the file, and therefore can't guarantee that the user would check for the existence of the mutex before accessing the real file?
No, I'm writing the code
my understanding is that it's aran's script. but they're like, all running from different machines. that's the assumption so far.*
I keep telling you: users are the worst
12:42
so it's different instances of some script that essentially have probably no real knowledge of each other
laurel
In which case, the empty file should be enough? You'll want a back-off mechanism though, I guess
So that a failed read doesn't actually just fail, it starts a loop of retries on some schedule
just so we're on the same page, you're saying that if aran's codes essentially make a new lock file (akin to what libreoffice does) on this shared network drive to indicate that they're holding it, there's no race conditions introduced?
why is that? is file creation atomic?
I can't conceive of any race condition in that case..... you just preempted me
12:46
usual disclaimer, i dont know the answers here, just here to be a party pooper :P
(though really, would love to know one way or another)
Provided that the code consistently starts by writing the lock file, I'm confident that's a blocking OS call and the reader can't proceed without first writing that file
So if I want to open foo.pdf, I try running open('foo.pdf.lock', 'x') and if that fails I know someone else currently has that file open?
or someone's script crashed while the file was open :'(
My unsubstantiated claim is that yes, this is correct
Aran is writing the reader code so he should be able to ensure in almost every case that the code cleans up the lock
@AndrasDeak--СлаваУкраїні that might be a solvable problem with a finally clause or something
12:49
not for the kind of crash I have in mind
i suppose it could happen if the script lost access to the network drive somehow
Exactly. Barring a power outage or something, it should clean up
which would be amusing, much to Aran's frustration
I usually have more than a dozen bash tabs open, some of them having stuff open in vim. When I bork my session I usually end up with stale vim lock files.
I'm not saying this is a common scenario :P
@ParitoshSingh That is actually quite likely. The company network/internet has been acting up lately...
12:50
Hmm
I could think of a broker mechanism that could be internally consistent - it is hosted and running the server and it obeys its own lock file, but this is somewhat more complicated
Alternatively, can the file be put into SQLite?
Is it tabular like a CSV?
Nope, they're pretty large PDFs
You just have to be difficult, don't you? :P
what's the sqlite approach though? cause i think again, that should be good enough if its the middleman
the file doesnt need to "actually" be in sqlite, you just need a reliable middleman
@ParitoshSingh SQLite will automatically lock a DB with a writer for 5 seconds (the default timeout). If you try to write while there is another writer, you'll go into a holding pattern
Readers still have access, and you could activate the WAL to help with concurrent readers while there is a single writer
I didn't realize you could (safely) open an sqlite file (for writing) in multiple processes (much less on multiple PCs)
12:56
I don't think you could store a PDF in SQLite. Maybe as some kind of blob type? In any case, it feels gross before I even start investigating so I'll put that to one side
@Aran-Fey it also drops a lock file
yeah, sqlite also uses a lock file for stuffs
If you can run something on the server then you could have a simple broker in Flask that controls concurrent requests for the file. I'm not entirely convinced that it's more stable than just creating a lock file every time you want to read, though, if there are network issues
I don't know if this will help, but I know that on Unix, you can easily get the list of process that have opened X files (as in, the list of processes and the files they are accessing). From there, while not really efficient, you could run that as a while loop and check if any process other than your python script is trying to do something to your file.
@ParitoshSingh I think technically that a mutex exists in code form. I perhaps incorrectly refer to the lock files as a mutex. Still, hopefully the new google-able term was useful
I don't think I can set up a HTTP server on the file server. I could maybe do it one of the other servers (the domain controller or something), but if I botch something there I'm gonna be crucified
Googling whether x mode works on a network drive wasn't very successful, but I did find this question, so knowing that files can be locked, it's probably safe to assume that x mode works as expected
@NordineLotfi I'm on Windows, unfortunately
13:15
@Aran-Fey Locking semantics depend on both the OS and FS, so you would have to search for that combination. From my own experience, these things generally are not documented well enough if at all, though.
This does not bode well
Distributed/remote locking is deep into headache territory. If you want to use something, have a look at distributed lock managers. If you want to get an idea of the gotchas, there's a good article from redis.
In practice, you should be good if you're dealing with filesystems that are made for consistency first and availability second.
Genuine question; what's the argument for availability over consistency? Is there a popular example?
Ideally though, use an algorithm that can stomach minor races. "eventual consistency" should be a good search term for these.
any non-journaling file system sounds like "consistency second"
not that I know the first thing about FSes
13:29
What if I beef up the rename solution by adding a unique identifier to the file name? I rename foo.pdf to foo.pdf.Arans-PC, then check if a file named foo.pdf.Arans-PC really exists?
On the surface it sounds like a horrible prospect. "I can rely on my system to maybe tell me a lie"
@roganjosh IIRC the CEPH object store (RADOS?) is eventually consistent. It's more fault tolerant that way – e.g. if your network is unstable – and still just as observably reliable when written data isn't read immediately by multiple clients.
When you run a database with one master and several slaves that's similar; the slaves might be out of sync for a short time but the overall system is more available.
OK, that's a very blatant example that just didn't enter my radar, thanks
Although that should at least be internally consistent? Replication would be atomic vs. just throwing back some mixed state?
Anyway, I don't want to distract from Aran's question since that's an actual problem vs. my pontificating
13:46
I should mention, it's not really that big of a deal if two people end up opening the same file. One of them will spend 5 minutes working for nothing, and then probably have their program crash. Could be worse
 
4 hours later…
17:18
@Aran-Fey right :/ While I did say this was for Unix, it's probably possible for Windows too, except I don't know how to do it unless it's Unix
that remind me, I recall talking with Kevin and someone else here about file locking a couple months back, although I can't seem to find my conversation using the search function in this chat
17:43
"although I can't seem to find my conversation using the search function in this chat" chat search is super-advanced. It's clearly user error
It's a case of "remember obscure word you used or bust" basically :/
18:05
looking on it again though, this wouldn't fit as a solution for Aran-Fey :/
18:52
FWIW, on windows you need msvcrt instead of fnctl.
 
1 hour later…
20:19
I'm kinda scared of these low-level solutions because 1) low-level stuff is inherently spooky and 2) I have no clue how or if they work on a network drive. I have more faith in high-level stuff (like renaming the file) because there's less room for unexpected interactions with the network
 
1 hour later…
21:22
sounds like something I'd expect to be wrapped up in a library somewhere already
oh, I think I missed a message earlier where someone was asking if I had wiki access on sopython. It was indeed given to me but I've been shy to edit anything thus far
21:36
yeah, I think I recall when it was given to you a couple times back (I was online I think). I thought you already documented the multiple canonical questions there already :P
@KarlKnechtel not saying you should, actually: did you ever thought of documenting the work you did in managing (not the right term I know) SO questions/answers inside say, a git repo? Would probably be unique since I never seen that kind of stuff done there/in that format yet.
@MisterMiyagi ah, so that's the equivalent. I actually already used both separately but never noticed they were equivalent of each other between Windows/Unix. Thanks
21:59
@NordineLotfi as it happens, I have a repo for workshopping stuff I think is missing: github.com/zahlman/so-workshop
since you mention it, documenting stuff I took action on, would also work in that format probably.
perhaps once I am actually blogging, I could integrate the repo content somehow.
 
1 hour later…
23:06
I just spent five minutes trying to figure out why a kwarg named 'foo' was getting a value of 'foo' somewhere, which obviously made no sense. Turned out I typoed **kwargs as *kwargs in a function call :|
4
23:22
@halt9k Definitely very curious as to how this failure happened, but yeah the git cli utility is a bit obtuse for the use case of setting every blobs in this commit to be parent to this other commit, though glad you seem to have figured it out, and yeah empty buffer commit will prevent git from checking previous blobs for comparison

« first day (4313 days earlier)      last day (860 days later) »