« first day (3368 days earlier)      last day (1597 days later) » 

12:41 AM
Can I ask on SO why my code is slow?
It works technically, but not in the way I have in mind or in the way that it should
 
That should probably be posted on Code Review
Although, there's a massive contradiction in what you're suggesting. If it it's slow and doesn't do what you expect, how is that "working technically"?
 
If there's any chance it's buggy it can't go on Code Review. If it's incomplete like a small function then it can't go on Code Review.
 
The output is good
The functions does it's job, just way to slow
my only concern here is speed
 
Then it might be on-topic on CR. Read their [on-topic] help page and codereview.meta.stackexchange.com/questions/5777/…
 
I posted it to CR
 
12:56 AM
you're a quick reader
 
Reading the whole thing is overkill
The table up top great
 
I can't find your post there
nm, I had the wrong sort on posts :/
@Vader That code is so confusing. if value_of_important_2_for_this_row is np.NaN:is almost certainly going to fail, and you shouldn't be iterating the df anyway
 
I know I shouldn't be iterating over the df
but .apply() doesn't work here
 
apply is also going to iterate over the df
 
is np.NaN is definitely not nice
use isnan or isnull or something friendly
anyway, they'll hopefully tell you this on CR where you asked
 
1:09 AM
but isn't is None acceptable? np.NaN is just numpy's None
 
I don't want to get into a big debate about the question itself, but I think you might need to take a step back and think about the component parts
 
remove_rows_that_dont_have_important_1_in_important_2 somebody is going to comment on the name.
 
I want it to be descriptive
It just for this example. My actual code is named in a better fashion
 
@Vader no, NaN, by definition isn't equal to itself
 
it is in numpy
 
1:23 AM
Well, you'll need to prove that to me
 
np.NaN is np.NaN
Out[4]: True
 
np.NaN is np.NaN, but is np.NaN == np.NaN?
 
huh, that is False and weird
I thought if is is True then == would also be True
 
The dark side of the force is a pathway to many abilities, some considered to be unnatural.
 
Star Trek?
 
1:29 AM
Oh my
 
Just kidding :D
 
:P
 
1:48 AM
rbrb
 
2:04 AM
Hi, can anyone help me with a OpenCV question please?
 
AMC
2:40 AM
Hello Python people. Do any of you know of a canonical post/answer to the many (I found 12 in a matter of minutes) web scraping questions which all stem from people using tools like Requests + BeautifulSoup on pages which are dynamically generated? I was thinking of asking and answering a question of my own, which would hopefully cover all these various posts.
There is a similar question which has 6 upvotes, although I would obviously rather not edit the existing content to death.
Also, if this isn't the right place to discuss this, where should I go?
 
 
4 hours later…
7:10 AM
[tg:cv-pls] duplicate (see comment) stackoverflow.com/questions/59594385/…
@AMC the one I usually use as duplicate has a title like "scraping ajax" or similar, hang on ...
though while looking for it I came across this which looks pretty exhaustive too:
156
Q: Web-scraping JavaScript page with Python

mocoperaI'm trying to develop a simple web scraper. I want to extract text without the HTML code. In fact, I achieve this goal, but I have seen that in some pages where JavaScript is loaded I didn't obtain good results. For example, if some JavaScript code adds some text, I can't see it, because when I ...

please do nominate the ones you found as duplicates; if someone has a better canonical still, we can then simply mark this one as a duplicate of that
and thanks for bringing it up; there are many topics like this where a common FAQ has an ever-growing morass of low-quality duplicates
this would be good to add to sopython.com too but I guess you will need to recuest privileges separately
 
AMC
@tripleee I hadn't heard of it before, thanks!
@tripleee See that one seems like it fits, but I don't know how I feel about it. That post shows how to solve the issue, yet it doesn't explicitly answer most of the questions. I think it would feel slightly odd to flag questions as a duplicate of that post since the question is the answer to potential duplicate. Am I making sense?
Essentially, question A (highly upvoted one) itself is the answer to question B (why isn't my web scraping working), which is why I find it weird to use the duplicate function, designed for identical questions, because they're not, really.
Now if presented with that canonical question, most people can figure things out, I'm more concerned with explicitness and discoverability, amongst other things.
@tripleee Update: The "Add Common Question" page says: You are not an editor. The page will be created in draft mode and must be approved by an editor., so it looks like that shouldn't be too big of an issue.
This common question on sopython does cover the issue.
 
8:00 AM
@AdarshPunj always best to just ask your question, and someone may chime in later with an answer or suggestion (like me :D)
 
8:56 AM
@AMC hmm, it's just one aspect of the answer there though
 
9:18 AM
cbg
 
9:31 AM
@MisterMiyagi Sure, but if you can't repro it, it's unclear what's going on... suggesting tripleee should add the repro instructions to their answer.
 
AMC
9:45 AM
@tripleee Yeah, it doesn’t feature very prominently
 
Is anyone comfortable in the responsibility-driven design concept?
 
10:46 AM
Does a set always store values in sorted order in Python3?
 
@AMC Well, the people asking these types of questions they usually don't know how website work (and that some content might be dynamically generated). The easiest way is to point them to use something like selenium, but I don't know if that's right (usually they return with the same type of question later). If I can hobble some quick answer and point them that the page is making other requests via JavaScript and do an example, I put that in answer. Hope that helps.
 
I tried the following code :
>>> lst = [4,7,2,4,1,2]
>>> lst
[4, 7, 2, 4, 1, 2]
>>> set(lst)
{1, 2, 4, 7}

Why does the set function yield the result in sorted order, what is the difference between unordered and sorted
 
I'm pretty sure there's half a dozen easily googleable SO posts about this
 
11:26 AM
thanks, i found them, i was simply searching for sets, and not unordered collections
 
11:37 AM
any good news about life? why talking about code like the entire time
 
@Plain_Dude_Sleeping_Alone whom are you referring to? :P
 
@Plain_Dude_Sleeping_Alone What if Code is Life...?
 
@Plain_Dude_Sleeping_Alone clearly you're new here
 
@Kevin You could view it as a Constraint-Satisfaction Problem, or a permutation problem, since individual moves are not unrelated. When an object only has one viable move option, you get a (chain of) forced implications.
@Kevin Don't do coloring, sounds like a total red herring. This sounds like A* search, since you only want the hypotheses with most moves, then compute the (unexpanded) candidates and score them according to the no. of moves. You'd need a heuristic for unexpanded candidates. e.g. "Total #moves >= #legally moveable objects for which we haven't computed move yet". Although that'll be an overestimate. I'm sure you can refine if further, considering when two subgraphs overlap on a node/subgraph
@Aran-Fey Agreed. (@tripleee) My non-expert take: The overhypeing of design patterns as a language-agnostic thing is somewhat a discarded 1990s fad, and goes hand-in-hand with Java evangelism. Really many of them should be viewed as a language-specific attempt to solve a problem given the limitations(/design choices) of that specific language. I'd love to see an essay covering which specific subsets of design patterns are obsoleted(/necessitated) by which language choices...
...and how things have evolved 1990s-current.
Some might disagree.
 
12:01 PM
uhhh, can somebody explain to me why this inheritance doesn't work?
class Window(Toplevel, SingleContainer):

TypeError: Cannot create a consistent method resolution order (MRO) for bases Container, Toplevel

Toplevel.__mro__: [<class 'widgets.toplevel.Toplevel'>, <class 'widgets.widget.Widget'>, <class 'abc.ABC'>, <class 'object'>]
SingleContainer.__mro__: [<class 'widgets.single_container.SingleContainer'>, <class 'widgets.container.Container'>, <class 'widgets.widget.Widget'>, <class 'abc.ABC'>, <class 'object'>]
What's wrong with the MRO Window -> Toplevel -> SingleContainer -> Container -> Widget -> ABC -> object?
 
@Aran-Fey If Toplevel, SingleContainer are from an open-source package, can you give a link? Neither inherits from the other? What is their highest-level common ancestor?
 
It's from my own code. I'm in the middle of refactoring, but I can push it if you really need to see it. They don't inherit from each other; their first common ancestor is Widget as you can see
that was an accident though - Toplevel should actually inherit from Container, which also fixes the problem. I just don't understand why
@smci Sorry, still not getting it. Could you tell me where exactly the problem is?
 
12:30 PM
@Aran-Fey I don't know much about multiple inheritance, but supposedly when class B inherits from A, and C inherits from both A and B, your definition should just be class C(B) not class C(A,B), which causes the confusion about multiple inheritance MRO; it sees multiple definitions of the methods from A. Per stackoverflow.com/questions/29214888/… etc.
 
@Aran-Fey I don't think I can MCVE it, what did I miss? gist.github.com/adeak/b7b126ef3085514f9af9e87cace3bd50
Are there some more diamonds going on?
I tried that on 3.7
 
12:55 PM
...although apparently for chained attributes access, not any arbitary chain of objects
 
1:15 PM
Found a better dupe target
unclear, no MCVE, probably a dupe if clarified stackoverflow.com/questions/59600245/…
 
@AndrasDeak That is a very good question. I double-checked, but there's no multiple inheritance anywhere except for Window. I can't figure out how to repro it outside of my project either
@smci I understand that class Foo(Child, Parent) can't work, but I don't think that's the problem I have in my code
argh, the problem is in the other Window class
 
1:33 PM
Heh
Did your parents not teach you to use unique class names?
 
I'm not gonna try to make a MCVE because there's at least a dozen classes involved, so suffice to say that I had two copies of all those classes, and one Toplevel inherited from Container but the other didn't
@AndrasDeak They didn't :(
 
Let's hope you didn't inherit their bad practices ;)
 
Auch, get burned. Haha
 
neither parent isinstance(Programmer), so all bad practices are my own fault
 
1:54 PM
@Aran-Fey Arrgh! I swore you were trolling us... do your penance at the shrine of no-MCVE...
 
sorry :(
 
@Aran-Fey if this is some kind of backend system would it make sense to have them all qualified names such as GTKWindow/QtWindow etc., and have a front-end module that aliases the necessary set to unqualified Window etc.?
 
and what do we do with this one:
Jan 1 at 10:13, by smci
What to do with this highly-upvoted 2010 question How does polymorphism work in Python? It has many views but the question body is not general and contains a wrong premise: OP thinks myDog.__class__ is animal does what isinstance(myDog, animal). Also, OP isn't asking "How does polymorphism work in Python?" but "How to test if an object is an instance of specified class or its subclasses?"
 
@AndrasDeak Hmm. Interesting idea. I'll give that a shot if this refactor ends up with a decent code base
 
2:22 PM
happy new year cbg!
 
^ @Aran-Fey? ghosts of ABC past for you...
 
2:38 PM
Fun challenge CodeGolf.SE: Generate a (random) Nine-Ball Pool rack given the 1 ball is always at the front of the diamond, the 9 ball always in the middle, all other balls unconstrained.
 
3:19 PM
@ParitoshSingh cbg :)
 
Sam
3:45 PM
Hi all, does anyone know if there is a package to align two lists of tokens? Consider the example: a=["30", "-", "foot"] b=["30-", "foot"]... If I want to get an alignment from a -> b the resultant list could look something like [[0, 1], [2]]
 
4:11 PM
Haven't played this in a while :)
 
@smci You're right, thanks for getting the accepted answer updated!
@smci The title you suggested in your comment seems fine to me.
 
4:38 PM
@Sam No idea but it seems trivial to walk down each of the two lists/iterators, and consider the cases (join left/ right/ neither/ (both?)). Is it guaranteed to only ever involve merging two tokens, or would you have to consider 'free-for-all' vs ['free', '-', 'for-', 'all']? Also, (esp. if whitespace is preserved, or removed), you could simply use len() to eliminate candidates...
 
Sam
It’s not guaranteed to only merge two tokens. Those examples you gave are possibilities too.
 
@Sam Well, do you want an entire recursive-descent parser? (Better to fix the two tokenizers, in the first place). Please show us a better example. Also, my comment that len() wil help you eliminate candidates, just see which side has a shorter cumulative-length so far, that should tell you what's likely to have merged.
 
@Sam what result do you expect for b -> a?
 
Sam
a=['a', '30', '-', 'foot', 'und', 'ete', 'cted']
b=['a' '30-foot', 'undetected']
Maybe this is a better example?
@MisterMiyagi We can always assume b is a fragmented version of a
 
@Sam Like I said, walk down each of the two lists/iterators, and consider which join case you're looking at; basically len() tells you, it's always the one with the shorter cumulative length. You should be able to code it yourself. Else post a question on SO and I'll answer it.
 
4:50 PM
@Sam then checking the length of the accumulated elements of a against the length of the b should be fine
 
@Sam Your case is even simpler than the straddle-case that I gave. Pseudocode: at each point, continue to get a token from the iterator with shorter cumulative length. When both lengths are equal, you should see the concatenated tokens are equal too. (else parse-error).
 
@Sam Maybe this is what you search for? dpaste.org/c4dS
 
@AndrejKesely you can just enumerate on a, no need for that while loop
 
@MisterMiyagi Yeah, not perfect. I tried one liner with itertools, but failed
 
Sam
@AndrejKesely That looks perfect thanks. I'll try remove the while loop now
 
5:02 PM
@AndrejKesely I don't think itertools has anything useful -- groupby and accumulate can get you half way there, but they don't play well with auxiliary information.
 
@MisterMiyagi Exactly...accumulate was my first idea, something with 3.8 walrus operator. Maybe I'll look at more.
 
5:22 PM
@AndrejKesely it's possible to do with enumerate and accumulate -- that saves you the current_a += a.pop(0) and i += 1.
with some error handling, it looks like this: gist.github.com/maxfischer2781/f9040c6d1fe33afeaa5d4274854dd1c9
 
What's the correct procedure when I'm in the middle of refactoring, and I want to try a different way of refactoring? What do I do with the current unfinished state of the project? Commit? Stash?
 
I'd go with "stash", or maybe commit as a branch
 
Let's assume I'm currently working on a refactor branch and I've already made a couple of commits (with code that's still broken, but where individual parts have already been successfully refactored)
 
From git-scm: "Often, when you’ve been working on part of your project, things are in a messy state and you want to switch branches for a bit to work on something else. The problem is, you don’t want to do a commit of half-done work just so you can get back to this point later. The answer to this issue is the git stash command."
 
You could branch the branch
 
5:32 PM
But not without committing, right?
 
well your commit would be on a branch.
 
Ah, you mean create a new branch just for the half-finished thing
 
@MisterMiyagi Looks good :) Maybe OP will find it useful.
 
Stashing sounds like the better option, but I wish there was a way to "bind" the stash to that particular branch
...I'm more and more convinced that I'll write my own VCS at some point
 
Iirc git stash stashes on your current branch. And you can also have multiple stashes and list them with git stash list
See the link for git-scm I just posted for more accurate details.
 
5:36 PM
> saves it on a stack of unfinished changes that you can reapply at any time (even on a different branch)
 
Ok. I'm slightly off. Still should be good enough tho.
 
it'll do, thanks
 
np gl.
 
Sam
Follow up from earlier, is there a reason why "hello world." would get encoded with "``", "hello", "world", "\'\"`? Is this character escaping on the speech marks?
 
@Aran-Fey branches are just labels so it's not terribly exhausting to do that
 
5:47 PM
Yeah, but I don't want to have another branch
I just want to temporarily save the current state of my branch, dangit
 
when you no longer temporarily need it, delete the temporary branch?
the current commit you have is the current state of your branch
 
But why should I save the state of my branch in another branch? That's just silly
And if I make a commit then I have to remember to delete it, or it'll show up in my history forever
 
if you put that commit on a new branch named TMP_DELETE_ME_LATER you might remember, and deleting that branch will hide that commit forever
Either 1. I misunderstood your problem, 2. you're misunderstanding branching, or 3. you're lazier than I'd imagine
 
I don't think it's laziness, more frustration at having to find silly ways of accomplishing what I want to do
 
Perhaps you're looking at it the wrong way? :) Which is not to say git is perfect, I just don't see your problem
 
5:55 PM
Sure I can use naming conventions to remind myself that refactor_TMP stores the current state of my refactor branch and that I need to delete it later, but what the heck kind of UX is that
 
The current state of your refactor branch should be the refactor branch. If you wnat to take its current state as-is, that's exactly sitting on a given commit, so you either take a post-it note and write its hash there, or you create a branch, or you create a tag (would be even weirder)
 
When I said "current state of the branch" I meant "current state of my working directory while that branch is checked out"
 
if you want to go in a different direction then that direction could be another branch
@Aran-Fey including unstaged changes and whatnot?
 
yeah
 
@MisterMiyagi Ok, here is the shorter version that I came up with dpaste.org/jOYs But it needs a separator character ( '|' in this case )
 
5:59 PM
Hi guys i work with BeautifulSoup. I want to get only user1, user2, user3. With this code user = soup.find_all('span', {"class" : 'users'}) I get this pastebin.com/MYPsD61C Can someone to help me?
 
I guess then you can indeed use git stash. I think it remembers where you stashed from, even though you can reapply the changes anywhere
 
stashing certainly seems like the best option... for now
 
although you have to stage your changes for that I think
 
Couldn't you just stash
 
I guess you can always cp -r your repo to a tmp directory, problem solved :PP
 
6:02 PM
I was gonna say "IntelliJ does that for me", but then I noticed that one of my files wasn't stashed... sigh
 
This is my code pastebin.com/G4qD4rJA
 
things like stackoverflow.com/questions/20028507/… seem very hacky, might as well just commit
although that might not 100% match your problem, I didn't check in detail
@Pijes it's not an error, just a minor style thing: I'd use the same kind of string literals (either all ' or all ")
@Pijes also: seems like for each match you have to further match to find the span you need
so you might have to loop over your matches and do another round of finding
 
@AndrasDeak Ok can you give me some example?
 
no, I don't usually use beautifulsoup but it's quite straightforward
I bet there's a user.find_all or similar method and friends
 
@AndrasDeak OK Thank you
 
6:08 PM
there might be a way to get the right span in one line but I'd have to know BS for that
 
@AndrasDeak Can I do this without BS?
 
You can but I wouldn't.
 
OK. Thank you
 
(when I said user.find_all I meant user[0].find_all using your naming, of course)
 
@AndrejKesely after staring at that code for 2 minutes, I still have literally no idea what it is doing.
@Aran-Fey I recommend using git stash only for short-term storage, such as stash-checkoutbranch-stashpop. Long-term, you should name things. If both of your variants are non-trivial and divergent, that's exactly what branching is useful for.
 
6:18 PM
Wait, stashes don't have names?
 
Future you will thank you for the pain and suffering of doing an additional git checkout -b.
 
@AndrasDeak I get only this [] when I add this user = soup.find_all ('users[0]') to code pastebin.com/KYXQPnWB
 
@Aran-Fey you can optionally name then, but then you can just as well create a proper branch.
 
oh, well, "WIP on <branch_name>" is good enough a name for that kind of stash I think
 
@MisterMiyagi The idea is to create regex patterns from list b (eg. 3\|?0\|?\-\|?f\|?o\|?o\|?t), then re.match string from a (eg. 30|-|foot|und|ete|cted) - count number of separators (|) in found substring and we have [start, end] of the group. Ok, I'm going to have dinner now
 
7:07 PM
@Pijes because that's not what I said
you need to loop over soup.find_all(<what you have now>) and find things in each of those tags
 
@AndrasDeak OK. Thank you
 
(or find out how to find nested tags without a loop)
 
stashes are named stash@{0} stash@{1} etc
 
@user10662977 when they said name they meant as in message
 
7:30 PM
What is the easy way to open online cloud Doc "Text" with python? like on google drive or one drive?
 
8:11 PM
Not sure how to answer that other than "find modules that let you access google drive/one drive files"
 
wim
@Aran-Fey that would be my choice
the stash is annoying, has no advantage over branching
branches are free and easy, create as many as you want, your reasons for not wanting to branch don't make any sense to me
 
9:04 PM
for saving an instance of the artist line object in matplotlib that is unpacked from the list the idiom is often to do this:
>>> a, = ax.plot(range(10))
>>> a
<matplotlib.lines.Line2D object at 0x73967eb0>
But I naturally do this:
>>> a = ax.plot(range(10))[0]
>>> a
<matplotlib.lines.Line2D object at 0x73967e90>
are these two approaches the same?
 
wim
in this case they are the same. in the general case, the first one will raise exception if there are more than one artists returned.
 
ah, did not consider that
I'm working off of a raspberry pi computer at the moment. these things really aren't bad for 40 bucks. anyone in the world can have an OK computer with these things on the market
 
9:33 PM
@Permian Both of those are web frameworks. You don't need that if you just want an API
There was no way to answer that question reasonably. You can just have sockets open in your microservice, but it's still really broad
 
10:12 PM
@roganjosh I can't find the question anywhere :D
 
It was deleted as I was mid-way through typing why I downvoted. Apologies for the disembodied reply here
 
aaah...OK
 
I recognised the name and felt that I should at least try explain myself :)
 
yeah, sure, I just got confused
 
10:29 PM
Just keeping you on your toes :P The way you navigate SO/Meta, I'm imagining a full chart on the wall with pins and string to work out what I was replying to :D
Someone on FB keeps going on about The Whitcher. It's been god-awful getting to episode 5; I'm wondering whether I can finish the series while trying to get back into work mode for tomorrow or whether it's just rotting my brain
 
Watch itttt :P
only 3 more hours
 
10:45 PM
Lol. I'll do an hour tonight to clear another episode, then I'll re-assess
 
if you think it's hard to stop between episodes you should probably only start the penultimate one if you're ready to watch the last one as well
 
AMC
@AndrejKesely That sounds good to me! I think the transition HTML parser + request --> Selenium is a rather natural one, since both follow the same "select the elements using CSS or xpath etc." style.
I think I'm going to start writing a draft for a canonical/reference answer
 
@AndrasDeak It's taken effort to get through 5. I certainly won't be staying up to get through the rest, but I have ep. 6 ready to go :) 1 a night is more than enough
rbrb
 
I thought you were struggling with the binge
rbrb
 
11:06 PM
@smci perfplot?
 

« first day (3368 days earlier)      last day (1597 days later) »