« first day (1918 days earlier)      last day (3045 days later) » 

12:45 AM
I have a question involving python, beautiful soup, and some web extraction. Can anyone help?
 
possibly? It's been a while since I've played with beautiful soup. In general, just ask, don't ask if you can ask (from da rules: Asking a question: #1)
 
oops. You know when you do a web search on a webpage and it displays your results. If your search is very broad it will have a ton of results, some sites have a "load the results" Button on the page. If you want to see the rest of the results you have to click the button. How do I go about extracting all of the results?
 
beautifulsoup can only get what the server sends you as a response. If you need to interact with the page to see the data you're looking for, then you need a framework that allows you to interact with the page, not just read it.
you might check out mechanize
but I don't really know what the go-to tool for that is nowadays
I've never had to play with that.
Maybe selenium?
 
phantomjs is what I have had the easiest time with for that
 
APZ
Hi everyone
 
12:57 AM
There may be an easier way, though. Check to see if the page you're interacting with has an API you can utilize. That will DEFINITELY be easier than using a webdriver and scraping the results
cabbage apz
 
I have done a ton of reading and searching but haven't been successful
 
APZ
I am new to flask and I am writing a REST api for a simple video upload app. My app developer told me that sessions wont work for android app so my storing of user role in sessions wont work. What would be the best way to store user role for a user till he logs out?
App does send a user id with every endpoint but I dont want to go to the DB everytime to find user role.
 
@APZ not sure, but this question came up on a quick google search. Is that useful?
 
@AdamSmith @JoranBeasley : I will check it out
 
Good luck @Masterminder
 
APZ
12:59 AM
yeah I already saw that, dont know enough android to comment on the relevance but have defenitely sent it to the app dev
 
If you don't have a way to persist session data then you'll have to look up from db every time
sounds like your client needs to persist session data :P
anyhow, weekend rhubarb for me -- ciao!
 
@APZ why doesnt he think sessions will work with android?
 
APZ
@JoranBeasley exactly, its bothering me too , I think he needs to do some reading but he hasnt given a exact reason
 
alternatively ... OAUTH2 should be easy enough for you to implement ... and should work regardless of anything
 
APZ
alright
 
1:02 AM
if you are using a web browser sessions should work
if you are using something like phonegap ... then im not sure
 
APZ
yeah web browser is all good, its just when we use the app, we are using whataspp type otp authentication and I am setting session[account_id] there, but subsequent request fail as the session[account_id] is not set and none of the endpoints which work on browser work on the phone app
 
hmm well I regularly use request.session and my endpoitns work in android chrome at least
 
APZ
hmm alright, let me see where he is missing out, thanks
 
In non-programming related news, Kraken and alcoholic ginger beer is delicious.
 
 
2 hours later…
2:57 AM
Cabbage
 
@MitchPomery :P
 
I never liked the taste of Kraken. But Alcoholic Ginger Beer is delicious
 
3:12 AM
The ginger beer mellows it out a lot, the kraken just adds a light bite to it and a hint of spice.
 
DSM
3:31 AM
@MorganThrapp: try d=-((int(x)*8-sum(('vexuobcl'+2*'clornielatrisuke'+4*'Ratahoesuhihie').count(q[‌​-3:-1])*(1-12*(q[0]=="N"))for q in l))//8)
 
3:42 AM
sweet I just discovered metaclass also is inherited
 
@DSM I get SyntaxError: invalid character in identifier.
 
DSM
Ah, weird. Try d=-((int(x)*8-sum(('vexuobcl'+2*'clornielatrisuke'+4*'Ratahoesuhihie').count(q[‌​-3:-1])*(1-12*(q[0]=="N"))for q in l))//8)
 
Same thing.
It looks like q isn't being assigned a value before it's being indexed.
 
DSM
That wouldn't cause a SyntaxError. Weird characters are being introduced in the copy-paste for some reason.
You can see the extra bits: .count(q[\u200c\u200b-3:-1])
 
Huh, yeah I just deleted q[-3 and retyped it and it works now.
Yup, it certainly seems to work.
Why that slice? From the brief glance I did, it looks like the last two are unique enough.
Also, beating Perl now!
 
DSM
3:53 AM
Stalker vs. Carrier, and I was too lazy to experiment with hacks to see if I could fix it easily.
 
Oh, yup, I missed that one.
 
DSM
4:08 AM
I looked for all two-combinations of characters where there was no ambiguity about the resulting score. Then I manually shuffled it to take advantage of overlaps (which I wanted, and why I had to move away from using the first letter), and verified that it gave the right scores for all entries (I was worried about accidental overlaps).
 
DSM
4:58 AM
@MorganThrapp: managed to squeeze out a few more characters because so many of them were *2 I could just make that the default. Try sum((('vexuobcl'+2*'clsuie'+4*'Ratahoiesuhihi').count(q[-3:-1])*(1-12*(q[0]=="N‌​"))or 2)for q in l).
 
 
1 hour later…
user559633
6:21 AM
hello night owls
 
Night? But it's only 2.30 in the afternoon
 
user559633
:) For the people that were active when I posted that, it was the middle of the night
 
8:08 AM
cbg
 
 
5 hours later…
1:08 PM
cabbage, this room is sleeping, not even on the first page on chat pages (until my messae)
 
1:23 PM
@DSM I get 3 for 0 Nexus MothershipCore Mothership Carrier, but I should just get one. I think there might be some overlapping.
 
DSM
@MorganThrapp: I get 2.
 
@DSM That's really weird.
 
Ruh roh.
 
But either way, it should be one.
 
cabbage. I've been lurking, but it's rather quiet here tonight.
 
DSM
1:33 PM
Did I just screw the insertion up? The formula seems to work: proof
 
This question about dunder methods got me thinking: is there a reason to implement non-standard dunder methods, or is that considered evil because they may clash with future standard dunder methods?
 
2:01 PM
cbg
 
DSM
@PM2Ring: I wouldn't do it not so much for possible clashes but because then they're not really doing what the syntax suggests they do.
 
Greetings, George. Welcome to SO Python Chat! I assume you've read the rules :) You may also find Salad Language helpful.
@DSM Ok. My hypothetical use case was for methods that get called in a way analogous to what happens with __len__ and __iter__: so if you define __foo__ in a class it gets used if you pass an instance of the class to foo(). Sure, that doesn't require the dunders, and it's probably much more sensible to have some other convention to denote such custom magic methods.
 
DSM
Some projects I work on use single underscores, _foo_, for similar cases.
 
That sounds sensible.
 
2:24 PM
Yeah, I would go with single underscores personally. dunscore makes me think it's overwriting something from stdlib.
 
typo stackoverflow.com/questions/34827842/… ... Ok, bad indentation isn't necessarily a typo, but it's close enough. :)
 
Cbg all.
 
cbg
@PM2Ring unlikely to help future readers. ... so yep :)
 
Bhargav happy pongal mate :)
 
Ty, same to you :)
 
2:31 PM
@BhargavRao Exactly. A recent Meta question discussing that close reason came to the consensus that the "this one was resolved in a manner unlikely to help future readers" is the key clause, and it should perhaps be moved closer to the start of the close reason to make it more blatant.
 
Yep, I had read that too.
 
DSM
@MorganThrapp: try this. I haven't changed the formula,
 
2:57 PM
@DSM Yeah, that seems to work.
The base score is almost at 200 now!
 
DSM
I can get one more: q[0]=="N" is the same as q[2]>'v', because "x" is the only third character of any name greater than v.
Well, "w" I guess would be cleaner. (So I forgot about a letter of the alphabet. How often do they come up anyhow? :-P )
 
That whole greater than instead of equals thing always feels like black magic.
 
3:17 PM
rhubarb
 
rhubarb PM
 
3:28 PM
rhubarb
 
4:11 PM
Cabbage, all
 
4:43 PM
cbg Steve
 
Jim
Hey guys, quick question. Does anyone know if you can use any hackish trick to actually "bypass" a data descriptor used on an instance of a class and actually set a new value?
 
DSM
Just-arrived-at-work-on-a-Saturday cabbage for all!
 
Meh, nvm
cbg, DSM
 
DSM
TLA FTW!
@Jim: so if you have A.x being a property and A._x being the underlying variable it sets, you want to modify _x directly but not by changing A._x?
 
Jim
@DSM yes, that's what I'm after. Tried doing something like instance.__class__.propertyName.fset = None but I'm not allowed to do that either.
 
DSM
4:59 PM
But the setter could be arbitrarily complicated-- it could even construct the variable name that it's changing randomly, or use exec to bypass parsing, or whatever. So there's really no way to get at what it's modifying except in very restricted circumstances which you haven't said exist. Is there a real problem you're trying to solve? Maybe there's another way to address the issue.
 
Jim
It really came up from trying to answer a question on here which needs to modify a numpy array protected by a @property. I saw it and then got curious if one could actually bypass them. (Question in hand, if anyone is interested: stackoverflow.com/questions/34827838/… )
p.s A simple "no you can't do that!" is obviously accepted. It just piqued my interest :-)
 
DSM
5:17 PM
In that particular case there's no underlying coef to change in the first place: it's basically passing along a computed property safe_sparse_dot(self._dual_coef_, self.support_vectors_). So the best we could do would be to modify/shadow coef_ (which might be tough) or _get_coef() (which would be easy, but not quite what you were after, I think, even thought it would have the same effect.)
 
Jim
Yes, that case is a bit different from my original Q. Either way, to get a bit of closure on this, it is actually possible, albeit really hard, to modify an attribute protected by a data descriptor?
 
DSM
After thinking about it for a bit.. I'm not sure. :-)
 
cbg all
 
@Jim yes... I'm trying to remember how :p
 
Jim
5:32 PM
I see, it seems it might be something that is possible and interesting. Do you guys think it would be solid as a question or is it flirting with the too broad cv?
 
DSM
I'd actually be a little surprised if there weren't something already on the subject. Where's the (other) ninja when you need him? ;-)
 
I ended up working late and haven't yet ingested the copious amount of tea required to stimulate my small puppy brain
 
@jacob21 welcome to sopython!
 
Hah, hello (cabbage?)!
 
So, to the problem (for anyone else who's interested)
I'm assuming that upgrading Ubuntu is out of the question, right? :)
 
5:36 PM
At the moment, yes :)
 
Jim
I tried searching but unfortunately couldn't find relative content, so I came here :-) Anyway, I'll try and tackle this a bit more and if I end up hating pythons I'll ask the Q.
 
IIRC - you just have to be a bit nasty and go raw - something like object.__getattribute__(the_real_object, 'property_name') - that'll around the getattr and property stuff
 
It's scheduled for December
 
OK. Probably your best bet at the moment would be to compile 2.7.11 from source on your own and see if you can figure out all the dependencies. Are you able to install packages on your own?
 
Yeah
 
5:41 PM
grrr... this is going to bug me now - wake up brain!
 
Alright, let's give this a go
 
Jim
Thanks Jon (and previously, DSM), refresh your memory and if interested in answering, keep your eyes open if I do finally decide on ask it. (or do a Q&A and ping me, that'd be great too!).
 
@jacob21 sounds good. Just ping me when you're done, or if you have any questions.
 
@Jim all the info. is here
 
5:45 PM
yo yo Bobby G-oh
 
Sup :)
Have you been putting together lucrative offers for us all?
@tristan check these salaries; probably ties in with what you were saying about UK tech salaries agencycentral.co.uk/articles/2015-04/…
 
6:00 PM
Hmm
Compiling /usr/local/lib/python2.7/zipfile.py ...
make: *** [libinstall] Error 1
 
Looks iike a nice little problem with yak-shave potential
 
Removing old lib directory appears to have fixed it, all compiled and installed. Still getting an undefined symbol exception @MattDMo
 
@holdenweb congrats on the 6k :)
 
Thanks - all came in a rush over the last two days, haven't had much time to spend on SO what with the new new job and all
 
10k will come in no time - then you can see all the hilarious deleted stuff :p
 
6:10 PM
@JonClements that sounds like a no
:D
 
Not even sure I'd have time to look at the moment!
We're trying to build an MVP by the end of February, and prior to my arrival they were using "scrum" except the were no specific goals for each sprint, so things were pretty chaotic with no visibility into what was going on and when certain things might happen/be ready
 
Which doesn't defeat the purpose for a scrum or building a MVP at all sighs
 
Sounds like a scrummy way of doing things.
 
That's a difficult one to call, because scrum partly entails not necessarily knowing what comes when in 3 months' time
 
So changing the process we use, trying to learn the coding base, etc.
@RobertGrant I don't mind not knowing what's coming in three months, but it would be nice to know what we were trying to deliver next Friday :)
 
6:15 PM
E.g. I've worked around 50% extra hours in the last month because there was a deadline, which made the whole thing not scrum (or agile)
@holdenweb oh, right yeah. Agreed :)
 
@Kevin This was put out for us yesterday:
 
As to overtime, trying not to do that, though right now people seem to be working about a nine hour day. Oh, and did I mention that no issues had any size estimations on them, so last Monday we just had our first round of planning poker
@MartijnPieters they certainly spoil you guys at Facebook
Thanks for the visit, by the way, even if work did not ensue
 
My company does fresh fruit, but not for people onsite
 
coughs might have misclicked a button there out of sheer jealousy looks innocent :p
 
Then because the Director of Engineering is going to do something else now within FB, there was also this:
 
6:19 PM
@RobertGrant so I'm hoping after another couple of scrum cycles we will have some kind of a handle on team velocity - anything would be better than nothing
 
More doughnuts!
 
@MartijnPieters didn't know you worked there!
 
@RobertGrant 3 months already now!
 
@holdenweb our points system is rubbish
Wow, congratualations
Smart coworkers?
 
Couple this all with a founder who wants us to be Bloomberg by the end of the year, is hopeless at explaining his ideas and cannot prioritize and it all adds up to an interesting life :)
 
6:21 PM
Huh okay. I hope they pay well :)
 
@holdenweb but they'll have a CTO that's way more than competent bringing things into check and organised and efficient very shortly :p
 
But it's interesting work, and it's the first time I've filled a CTO role (in fact it's the first time in ages I've been employed by anyone but myself)
And while I don't know what the devs are getting, they are paying me pretty well
 
Just as someone with a vague eye on CTO one day, what do you do as CTO at your co?
 
TBH I think it currently taxes my interpersonal skills more than my technical skills at the moment
The role is to take hands-on charge of all development work, but I interpret it more broadly to include (when I get time) overseeing all applications of technology within the company - not in terms of operations, but of technology choices
So it's a sort of VP Engineering+
 
That's cool
How big's the company, if you don't mind me asking?
 
6:25 PM
If we get funded this year we'll be hiring - in fact we are now, but I'm not sure I would recommend it until the next funding round. I think we're up to about 30 people now
 
Oh, nice. That's not small.
 
Just big enough that I have time to decide on things like career structures, salary scales and performance evaluations before they become important
Last place I worked they had about 35 devs, and some hadn't had an evaluation or a pay rise in 12 months+ - want to make sure we don't end up there ...
 
Wow yeah
Must be very hard to do with devs though; can be hard to gauge value
 
So long-term I think the difficult thing is going to be cutting out time to stay a coder
 
Maybe you can practice (in a way) by critiquing code
 
6:30 PM
Scrum can help with the evaluations, because if you have records of the sized tickets somebody has closed, that gives you a fairly direct measure of the value they have contributed to the company (at least in relative terms). Now we are ficusing on delivering value, that is.
 
I remember one job I had, I was very happy being the Senior Dev. guy, and for some reason the company decided to "promote" me to the "Manager" of that department - I hated it at first
 
DSM
At some point I may have to consider moving to a more programming-side job. Having to deal with the frustrations of working with real data is staring to wear on me.
 
Dunno if you are familiar with the agile team maturity journey (that may not be its formal name)? It's five steps, and I assess we are currently at step 1 :)
i.e. starting to focus on delivering business value
@DSM what, having to do a lot of clean-up and data munging?
 
@DSM Join FizzyCorp! I could refer you, get you a nice cushy job, then you could fight for me to be promoted.
 
@Ffisegydd no no no - PuppyCo FTW! :p
 
6:35 PM
No fighting for the referral bonuses, guys ...
So what's your current deal, Jon?
 
@JonClements "at first"? What happened?
 
PUPPYCO?
 
DSM
@holdenweb: the munging I don't mind so much-- I do it for free anyhow answering lots of pandas questions. :-) At the moment it's dealing with disease utilization data which is consistently inconsistent, if you follow me.. It's great to have phone conversations with heads of thoracic societies, but it doesn't help when they give me numbers which are forbidden by the laws of arithmetic to work with.
It'd be nice to spend more time working on purely algorithmic and mathematical problems, and less so estimating incompleteness fractions and miscoding frequencies. :-/
 
@RobertGrant it's a mind set shift to go from dev to management... I'd done team leading etc...
 
So mostly people problems at least as the underlying cause?
 
6:37 PM
@JonClements yeah I've only done dev, design and team lead
 
DSM
"To summarize the summary of the summary: people are a problem."
 
@JonClements you got that right. I finish a day's work where I've spent six hours in meetings and wonder why I'm not learning the code base fast enough
 
@DSM I need to reread Douglas Adams at some point.
 
@RobertGrant turns out though, I'm "reasonably okay" at management...
 
DSM
Everybody does love puppies.
 
6:42 PM
Gross and inaccurate generalisation.
Cats 4 life.
 
Cats are the one true pet.
 
... I say "reasonably okay at management" - I've failed obviously to promote someone like @Ffisegydd to RO status and have them talk cats about cats - when obviously it's always puppies for the win :p
 
Better to serve a cat than own a dog.
3
 
Yeah I wonder if I'd like management; don't really know what it involves actually
 
6:46 PM
Unfortunately the Paradise Lost quote didn't back my argument, or I'd have served that right at your face.
I could really do with winning the lottery, got so much I want to buy ;___;
 
Blame Camelot - they always pick the wrong numbers
 
Yeah I know, bar stewards.
 
by "wrong" I mean it's not the numbers I picked - it's clearly a fault on their side
btw, did we end up having a room meeting and I just missed it or something?
 
I could comfortably go drop £3k on camera equipment tomorrow without even buying anything too extravagent.
No we never had it in the end, need to re-schedule it.
 
good - didn't want to think I missed it - I'll leave it in your safe hands to organise then?
 
6:59 PM
We'll see what we can do. I am utterly, completely, totally, emotionally, physically, spiritually, and sexually beyond busy at work at the moment.
So even if we get it sorted, I won't be attending.
 
Well, got to rhubarb now - off to dinner with the neighbours. Good to see you all
 
@holdenweb take care - speak to you later
 
See ya Steve.
 
@Ffisegydd let's just shove it to Feb for now
 
 
1 hour later…
8:04 PM
I've got a friend trying to write some tests for his Flask app, any suggestions? I may be asking a few questions on his behalf...
@Ffisegydd I love my little kitty cat, but I swear it's a one-sided relationship and she's actually cold-hearted psychopath.
 
hi
 
8:53 PM
@AaronHall It depends on what exactly he wants to test. Does he want to test the frontend functionality? Then maybe some selenium tests. Otherwise I like py.test for testing actual python code.
 
9:14 PM
Personally I like to test the alcohol. For that I use my mouth.
 
DSM
9:28 PM
d=-((int(x)*8-sum(((('vexuobcl'+2*'clsuie'+4*'Ratahoiesuhihi').count(q[-3:-1])-‌​12*(q[2]>'w'))or 2)for q in l))//8)
 
prays this question doesn't once again receive more down than upvotes
I took the big risk of actually asking something on SO...
 
DSM
@paul23: it's harder to ask a good question than we like to admit. One of the reasons I've never asked one on SO, ever.
 
10:05 PM
Being able to edit messages in here has me spoiled. I keep wanting to be able to edit text messages after I've sent them.
 
10:33 PM
cbg! How would someone change the default python path?
I don't want to pollute my system python as main interpreter so I want to change it to the one from homebrew, but I can't - when I say homebrew install python, it says I already have python 2.7 but it's not linked. (I'm using python2.7 because of opencv)
I have these following $PATH variables. I think this enthought canopy installation is messing my paths up:
`# Added by Canopy installer on 2015-08-25
# VIRTUAL_ENV_DISABLE_PROMPT can be set to '' to make bashprompt show that Cano\
py is active, otherwise 1
VIRTUAL_ENV_DISABLE_PROMPT=1 source '/Users/hongshuhong/Library/Enthought/Canop\
y_64bit/User/bin/activate'`
This is the one that I added: # Homebrew export PATH =/usr/local/bin:$PATH
 
11:15 PM
does anyone know answer to this question
-4
Q: Python script for wordfrequencies in a url

Reazur RahmanI am new to python. I have a problem that i would like to solve but i am not sure how to go about it doing it. I can get it done in java but don't know what's available in native python to get this done. Here is the exercise: Preferably with 20 lines of code or fewer (not counting 'import' line...

 
DSM
11:30 PM
It's against room policy to ask recently-asked questions here (see the room rules linked on the right). But if you break your problem down into a few parts, like "python command line arguments" and "python read from url" and "python word count", all the parts are already answered on SO.
 

« first day (1918 days earlier)      last day (3045 days later) »