« first day (3195 days earlier)      last day (1978 days later) » 
00:00 - 17:0017:00 - 00:00

17:05
cbg...
17:20
does pip -e /my/directory do anything to /my/directory?
As in, I suspect it doesn't and I am looking for confirmation
user10984358
I have managed to download the images but they are not getting saved, why is that so?
user10984358
this is what I tried
user10984358
for idx,url in enumerate(imgList):
	localURL=urllib.request.urlopen(url).read()
	imgPage=bs.BeautifulSoup(localURL,'lxml')
	finalImgURL=list(map(lambda x:toAdd+x,filter(lambda x:x.endswith('.jpg'),(url.get('href') for url in imgPage.find_all('a')))))[1]
	fileName=path+finalImgURL.split('/')[-1]
	print(fileName)
	urllib.request.urlretrieve(finalImgURL,fileName)
@piRSquared I've got a modulename.egg-info directory in mine, not sure if this is specific to how my setup.py is written. But it's quite minimalistic.
thx ad
17:33
@TheNamesAlc Looks fine to me. Are you getting an error message?
user10984358
I am getting some images saved in the directory I want
user10984358
it shows up as jpg but when I double click it fails to open
Maybe the page content is being served in an unusual way. What does it look like if you open the jpg in Notepad or some other text editor?
@TheNamesAlc if you've managed to install BeautifulSoup you should probably look at requests or for scraping stuff, possibly even requests_html which throws BeautifulSoup and lxml on top of being able to query the document as well...
If you're doing some hardcore scraping, then you might want to look at scrapy
user10984358
@Kevin I tried to open it in sublime and it just shows Loading Image...
17:36
I guess sublime text has the ability to display images? Try using a text editor that does not have the ability to display images.
user10984358
I got a page O_0
@Kevin it does, yeah :)
has for a little while, previously, it use to open it in a hex edit view though...
user10984358
so the problem was with the indexing then I assume, but how did it get saved without an error?
user10984358
@JonClements requests as in urllib.requests or BS has some requests as well?
user10984358
also is there anything easier than this?
list(map(lambda x:toAdd+x,filter(lambda x:x.endswith('.jpg'),(url.get('href') for url in imgPage.find_all('a'))))
user10984358
17:39
I am pretty sure it isn't meant to be done this way
Possibilities:
- when you navigate to thesite.com/the_image.jpg, it actually serves html that has the _real_ jpg embedded inside it somewhere.
- you messed up the url somehow, and whenever anyone tries to navigate to a page that doesn't exist, the site returns some html saying "page not found"
user10984358
well thanks for that!
user10984358
I just got into BS, I only need it for downloading dataset off of GitHub, I dont want to download the entire code base
@TheNamesAlc probably sounds like you'd get a lot from using html.python-requests.org though...
user10984358
17:41
but glad I learnt something along the way
user10984358
I will try this one out for sure, the spider in the icon got me intrigued :)
@TheNamesAlc I'm thinking hrefs = [url.get('href') for url in imgPage.find_all('a')]; urls = [toAdd + x for x in hrefs if href.endswith(".jpg")]
user10984358
replace map with list comp., yeah I forgot that
user10984358
its like two for loops now, is this the way to actually see what's in a document provided I get the links from a previous page?
user10984358
even what I tried has two for loops
17:44
Which using requests_html could become... hrefs = page.html.xpath('//a[@href]/@href') and something like page.html.xpath('//img[ends-with(@src, ".jpg")]/@src') or similar...
user10984358
anything less than the O(n^2) approach I am taking here?
Two for loops does not automatically make something O(N^2)
user10984358
@JonClements I have seen syntax like such somewhere Idk where but that looks more convenient I guess
user10984358
O(n*m) at least?
Both your code and my suggestion are O(N) with respect to the number of <a>s on the page
17:46
@TheNamesAlc no idea... it seems you've only given snippets of what you're trying to achieve... if you're also trying to do page navigation and following things, it sounds like you'd be better off using scrapy as I have said
user10984358
I feel like I can avoid doing an 'x in href' scenario there
I guess you could do it in one loop with urls = [toAdd + url.get('href') for url in imgPage.find_all('a') if url.get("href").endswith(".jpg")]. But I don't like that approach because it calls .get twice as much as it needs to. It's arguably less efficient than what you have now because of that.
asspressions to the rescue!
user10984358
@JonClements if you can check the chat like 2 hours back I said what I wanted to achieve, sorry idk how to link old messages
meet you in the middle: urls = [toAdd + urlgethref for url in imgPage.find_all('a') for urlgethref in [url.get("href")] if urlgethref.endswith(".jpg")].
(no, don't do that)
17:48
Or, just narrow it down to you're guaranteed to only match suitable hrefs... [toAdd + url['href'] for url in imgPage.find_all('a', href=lambda L: L and L.endswith(('.jpg', '.gif', '.png'))] for instance
In any case. I recommend you add print(finalImgURL) to your code, and verify that you can visit that url using an ordinary web browser. This will help distinguish between the two possibilities I listed earlier. If you can see the image, it's a content type trick. If you can only see a "page not found" page, something's wrong with the url.
user10984358
after going through this I guess I will stick what you suggested initially, use list comps, I have to look into scrappy or requests_html as well
user10984358
thanks a bunch yall
user10984358
there are actually 4 jpg links and only one of them is the image
Actually, I have no idea what scrapy et al are capable of, so let me not make sweeping generalizations
user10984358
17:52
I just need to find which
user10984358
I always assumed if it ended in jpg its an image, didn't know it could be a page
it probably shouldn't but as Kevin has said, if it's 404, then what you're attempting to retrieve might well be a custom 404 error page :)
user10984358
I read somewhere on SO that you had to add headers or such if you get into errors, I hope this aint one
user10984358
mimic user activity something
imgur will return an mp4(?) when you visit a url ending with ".gifv" so this is not without precedent
17:55
You can check the responses content-type header to see what it's actually returning you... if it's not a image/* like type, then it's not an image :)
(so basically, if the content type returned isn't the content type you expected, probably just ignore it)
user10984358
the images are getting downloaded!
@TheNamesAlc If a site is rejecting users that don't look like a human, they usually do so by returning an HTTP error response. The fact that your jpg file has html in it implies that you've gotten past that point.
user10984358
well I had to specify the URL, there were three other .jpg urls turns out the last but one was the actual image URL
So I don't think you have to worry about headers in general or the user agent string in particular
user10984358
thank you so much
user10984358
17:58
the site I was trying to download was from github
Oh yeah, github definitely does the content type trick. Consider the page github.com/kms70847/Animation/tree/master/src/3D_Dragon and the link to github.com/kms70847/Animation/blob/master/src/3D_Dragon/…. Despite being named "output.gif", it actually takes you to a page that has the real image embedded in it.
Hmm, I wonder if you can always get from the fake image page to the real image page by changing "github.com" to "raw.githubusercontent.com"
probably yes
umm... if it's github... wouldn't it just be easiest to just clone the thing locally?
Is there a command to get the most recent version of exactly one file from a remote repository?
they only want to download parts of the things
@Kevin I've only seen branch-level filters for cloning, though I haven't really looked
18:13
git has approximately one billion commands so I wouldn't be surprised if there was a way to do it.
@AndrasDeak oh okay, but if you're spending a day or so on just trying to get a "few things" vs just cloning the whole lot and just taking the bits you want... meh... I'd go for the later
But I would also not be surprised if the way to do it was a command five hundred characters long
git archive locally into a tar that contains only one file
ah, no, you're still downloading all, and filtering is done by tar afterward
@JonClements hehe, yeah
For a typical github project I bet that image data accounts for most of the size of the repository anyway, so if you want every image file, fetch-everything-then-filter is probably like 10% less memory efficient than fetch-exactly-what-is-required
ahhh... well, from my pov, I'd not be overly impressed by someone wasting a day :)
heck, by what I've seen so far, what'd be way more convenient is doing a wget/axel command with a max depth and a filter and just run that on the site and have it follow links and get images...
then get on with learning either something else, or as you're going to have the results very shortly, then learn how to get bits as required which you can compare against as you build it...
user10984358
@AndrasDeak lol how did you catch that one? well still on the track, will be joining as an intern this august, again :/, I guess
user10984358
I have the day off so I didn't mind spending time on this, though after the hassle I went through, I'd probs not recommend but on the longer run maybe I can change that to google images though a plethora of codes already do that
Room owners read much and forget little :>
user10984358
I should be vary of what I type here, my academic project now sends me back to PyQt :p
user10984358
18:27
remember helping me with that a while back?
Yeah.
user10984358
I have to add in a webcam window to a PyQt application
user10984358
I have the faintest of idea, gotta start with that now
user10984358
its not exactly a webcam window, its just the output of a video detection code my friend and I did
The last time I did webcam work I needed to use COM objects, which I did not enjoy.
18:35
opencv can handle webcams I think
@Kevin because object is a built-in type, and built-in type instances have no __dict__?
@holdenweb That's the conclusion I was eventually led to, yeah
wim
wim
Imagine how hard programming with timezones is gonna get once people colonize mars
Can we please not have DST on Mars?
I vaguely recall there was a subplot in The Martian about how sols don't line up neatly with days
Ha, there's been dispute about whether mission times should start with sol 0 or sol 1. Where's dijkstra when you need him
> In the science fiction series Mars trilogy by Kim Stanley Robinson, the Mars settlers use traditional Earth watches that stop ticking at midnight for 39 minutes and 40 seconds before resuming their timekeeping.
Worst possible solution? Worst possible solution.
19:30
Yeah, because if they were have a rover race and two finished within that 39'40" gap, it would be a technical tie.
Mildly interesting math problem: A is an infinite series of numbers. A[0] == 1. For any integer N > 1, A[N] == one plus the average of all previous numbers in the series.
I'm pretty sure races use the time elapsed from the starting point, not something like UTC when crossing the line :)
so for example A[2] == 1 + (A[0] + A[1]) / 2, and A[10] == 1 +(A[0] + A[1] ... + A[9])/10
At first it seems as if calculating A[N] takes O(N^2) time, but there is an O(N) approach that states A[N] only in terms of N and A[N-1]. If you find this relation, you may discover the name of this series.
So what is the "problem/question"
This problem emerged from an actual problem I'm trying to solve, so I haven't completely polished it to have a traditional Q&A form. "What's the name of the series?", perhaps
19:40
ok
If my algebra checks out, it's view spoiler
The actual problem I was trying to solve is roughly equivalent to this code:
N = int(input("Enter N:"))
while N > 1:
    N = random.randint(1, N-1)
    kevin.eat(CandyBar())
Given N, how many candy bars does Kevin eat on average?
The answer being, A[N]... Or maybe A[N]-1. Dang fencepost errors
A[1] is 2 which already violates that
Ah, definitely A[N] - 1 then
When N = 1, I eat zero candy bars. When N = 2, I eat one candy bar. So at least the first two terms match.
Or, uh, I guess A[N-1] - 1? The rare double fencepost error
20:13
ironically, given the talk today, scraping a website today myself
@roganjosh but is it ethical for the workplace to tickle you? Did you consent?
No, I found it totally humiliating and I shall be writing up a formal complaint to the site as soon as I regain my dignity
@roganjosh what's going on if I may ask?
@JonClements My last comment? PiR was making a joke :P
his fancy was being inappropriately tickled
20:38
oh right, just gotten up for a night's work on server stuff, so wasn't sure if I was missing something I could possibly help with or not... blah blah
I said the site tickled me; I was responding in kind, nothing is actually happening :)
@JonClements much appreciated that you asked, though!
not that I could do much anyway if another site's mods' want to do anything
you could join the stiffening ritual of the upper lip ;)
One already does so? Excuse me while one fetches one's china glass of tea :p
I upvoted the question and one of the answers.... I didn't gather what country that was in
20:43
@piRSquared I think part of it is a cultural difference even between the US and the UK
I don't doubt it but I expect disparate opinions in the US alone
In my mind, clearly unethical.
Quick Flask question regarding stream_with_context used in the server. I'm writing a test client for a server using stream_with_context() to generate the results. I have googled up, down, and sideways, but all I find are examples on how this is used in the server. Does this mean the client does nothing different?
think youtube music has borked... no search results even for the track I'm listening to... interesting
@PaulMcG unless someone else knows, the guy to ask is @davidism :p
My dad grew up in the US and, even though that was a while ago, he did make it sound like it can be more cutthroat over there. The questions on The Workplace often seem more combative than situations I can easily imagine in the UK
To be sure. But I was sternly reprimanded the last time I pinged an individual by name, and there is a remote possibility that other flaskers might be in attendance anyway. ("Sternly" may be a bit string, but I was just trying not to ping needlessly.)
20:49
In this context, I put myself in the shoes of someone who doesn't want to "Volunteer" even though I'd likely volunteer because it sounds like fun. Get a workout, hangout with coworkers I probably like, eat pizza. But for those who can't, it puts them in a terrible spot. They might truly be looked at differently by the powers that be (otherwise, why the complaint that there weren't enough volunteers) and this can cause stress.
We are trained (if your company is big enough) as to what constitutes inappropriate behavior. This qualifies.
Asking them to volunteer once might be OK. The second step with "enough people didn't volunteer, you have to volunteer more" is not OK.
Yes, I agree with exactly that
defeats the purpose of volunteering
"you have to volunteer more" was never said (that I can see)
I only skimmed the subject, I thought there were assertions that the company was unhappy about the number of volunteers
20:52
Mentioning that not enough volunteered is near the same thing
the exact same thing
No, a comment was apparently made, but that kind of assertion wasn't
let me read the question
> The person who organized the event complained that not enough people volunteered.
well, the question is whether the person represents the company or not
Not really. "Unfortunately, there were not enough people on Saturday so we will have to now continue the move during Monday's hours" would also fit that bill
I can see why people wouldn't be inclined to do the move, totally, I just think it's a bit drastic to call this "unethical"
if the company (who has power over you, because they pay for your food) complains in any official manner that there weren't enough volunteers, it's wrong
20:55
barely. I put that in the fuzzy quantum context with probability 1e-10 @roganjosh
I'm not sure I'd call it an ethical issue (but maybe I would)
is there a contract that states out terms of things?
It doesn't say
Just that they get free pizza
"free"
yeah
The main practical issue here is office-dwelling coworkers getting injured at work on the weekend. How does that work out with insurances?
20:59
reasonably contracts will have clauses that you can't as an employer restrict the "benefits" from that the employee anything unless that explicitly put it in writing
This isn't the first time I've been on opposing sides of such "arguments". I suspect in some way I may be naive tbh. I kinda get invested in companies and, thankfully, I usually get treated ok (even with the car crash of this latest venture). If it's likely to help keep doors open, I wouldn't see such a request as an insult (certainly not unethical) even if I couldn't comply
@roganjosh as I've said: I believe the request itself is fine. The potential pressure to make more people volunteer or make non-volunteers feel bad for not volunteering is not fine.
@AndrasDeak the insurances have to come to the employer - since they're the one's doing it
especially if Kyle from HR will tell you three months from now that you weren't even willing to volunteer for a few hours of pizza
But this is imaginary
21:01
@JonClements but weekend is one's free time, right? And everyone is volunteering. It's basically the same as going to a pizza place.
If Kyle raises that point, then it's an issue
@roganjosh you're all just brains in a jar to me; what's your point? :P
@roganjosh I get invested in the companies I work for as well. But I look at it from a leadership point of view. How do you lead a company of varying personalities when you present an opportunity to unfairly value some over others. Some people won't have the capacity to "volunteer" and have a justified right to be nervous about unfair treatment. I'd never do a think intentionally. And if it was pointed out to me, I'd fix it post haste.
@Andras not quite... if you volunteer to do that, your employee still has to cover you in a way
@JonClements I'm happy to hear if that's so
I'm not even sure how it would be handled here
21:03
in the UK - if a member of your staff works under the "company orders", you cover them and you're responsible for it
@AndrasDeak Ha, and that's why I don't work for you (plus, like, a million reasons rooted in practicalities like location and subject matter) :P
@JonClements same here and if a company said "But they volunteered" a judge would laugh in their face
Well of course, the Flask docs will be all about the server side. Now chasing down how to receive such a response if using requests in the client.
Maybe it's the case that I've worked in start-ups. If you created a start-up on a shoestring budget with 10 employees and you needed to move offices, would you immediately go for a removals company? Pizza + the chance of a bonus or still drawing regular income vs. an even-more-stressed financial situation that might collapse; what would you go for? The OP doesn't specify.
In a 10-employee start-up you don't start screaming unethical policy when something like that happens, right? The existence of the question alone makes me assume it's a large company.
21:11
Um I'm confused about a bitwise thing, I'm trying stuff out in scratch files and googling but I don't geddit
@PaulMcG There are good articles by miguel about streaming but I assume you've considered the options. I'm curious why you need to keep the context alive (though I'm unlikely to be able to help here)
elif n == 3  or ((n + 1) & n) > ((n - 1) & (n - 2)):
    n -= 1
what does that or condition imply?
right puppy snooze time, rbrb for a bit
rbrb Jon :)
I am merely the tail of the dog here. Our product has introduced an API GET call implemented using stream_with_context. I need to write client-side test code to pull that data.
So far, an unadorned get is adequate when just getting a few records. But I'm pretty sure the dev who chose a stream_with_context option is thinking this may run into the largish number.
21:14
@aadibajpai if I were you I'd start investigating the smaller bits (no pun intended), (n+1) & n and (n-1) & (n-2) and their relationships for various possible numbers
I think it refers to when should odd numbers be decremented to get closer to a power of two
But I'm not sure how they arrived at that expression and there's not even a single comment
So I guess I'm really looking for requests help, not Flask help.
requests seems to be rejecting my stream=True arg to requests.get. I think its time to formulate an actual posted question...
21:29
Is there any authentication involved?
Eh, I'll pipe down. I think an actual question will invite infinitely more insight than I can muster here, so I'll just watch that instead and learn from it :)
hey, there's a partial lunar eclipse going on right now
I'll try walking the dog to take a peek
21:58
@Kevin Useful to know, but also frightening it now has no owner and isn't a mgmt priority.
it was never a priority
status update: lunar eclipse looks like a gibbous moon, except it's oriented the wrong way and by all rights it should be a full moon tonight
@AndrasDeak Are you saying you have a partial eclipse in your part of the world?
yup, 66% coverage or something like that
@Code-Apprentice everywhere on Earth, actually :)
it's just that the moon is above the horizon here
oh yah...I guess lunar eclipses can typically be seen in more parts of the world than solar eclipses.
22:13
@Code-Apprentice that's pretty cool though.
In Django, would it be appropriate to call methods in views.py, actions?
I think we get another blood moon next year. timeanddate.com/eclipse/map/2021-may-26 .errr two years >.<
I learned to filter out coloured and supernatural moons due to how they are hyped and confused all the time
@earlyriser01 no, they are views
at least if you register them in urls.py, they are views
and if you have functions in views.py that aren't registered in urls.py, they probably should be moved somewhere else
dupe (2nd call) this of this question, incl. piRSquared's excellent, if overlong, answer Please just close this already, folks
22:25
sorry for the extra pings...my clipboard didn't have what I thought it had
@Code-Apprentice hehe, right
and no worries, I've got chat muted
@smci always retag with the generic python tag, before asking for close votes the first time
gold badgers can't retag themselves otherwise they void their hammer on that post
I'm surprised there's no xkcd about blood moons, though...
there sort of is one
22:27
yah...that's a new one, right?
"two week window in which astrology works"
"Dire Moon"?
Is that Jon Snow's Moon?
If only @smci would've retagged it with
rbrb all
22:33
@piRSquared (I stopped retagging, people here and in R gave me grief for it, it was becoming thankless. Didn't know you only follow selected pandas-specific tags.)
I'm certain that nobody has ever "given you grief" for retagging dupes before closure
@AndrasDeak Oh ok, didn't know either of those, thanks. Can you add those to room rules ("Before posting a request...")
no, because <2k people might start suggesting crappy retags due to that
@piRSquared Because when we tag we now have at most 3 tags left, usually fewer... people on Meta were quibbling... another Meta headache with lots of opinions but without a resolution
it's just common knowledge of how dupe hammers work
@smci he was joking that he would've had a hammer to close it with
22:36
@AndrasDeak Are you so sure? Go look at the R room GMTs for numerous counterexamples. There were also some comments in the R room itself, but it got closed by recent (ahem) management change.
I can't vouch for the R room, never been there, but this sounds very weird. I'll try to check it out later.
@Code-Apprentice Okay, so each individual method is considered a method of each views.py? Sorry, I'm just trying to figure out the proper terminology for when I go over my code in CRs.
and last time I checked GMTs was not an R room
@AndrasDeak What on earth are you talking about? GMTs is the main R room, now that R got closed after a years-overdue management change and R-Public is avoided by experienced users. (Why do you question my basic grasp of facts? It's somewhat rude. Better to phrase as "I wasn't aware that...")
your basic grasp of facts just made you claim that "I stopped retagging, people here [...] gave me grief for it" which is not true or not the complete picture, so I apologize if this makes me consider you to be an unreliable narrator
22:51
@AndrasDeak Please don't call me a liar in a backhand way. That's unacceptable room behavior and we've discussed "unwelcoming" many times before. I'm busy next 12 hours, so I don't have time to dig up citations for you. Meanwhile, do you at least accept that GMTs is the main R room? you just claimed it wasn't even an R room.
I never said you should dig up citations. I said I'll look into the GMT transcript myself.
@smci I've seen bits of the transcript from GMTs before and I looked at the room description before I wrote that, no sign of R there. After your complaint I checked the linked rules and that's indeed low-key R-related
and I don't think you're a liar, I just think you might be wrong
@earlyriser01 I don't understand your question. What do you mean by "each views.py"? Do you have multiple files with this name?
@AndrasDeak Do you accept that GMTs is the main R room, yes or no? It's that simple. You just claimed it wasn't even an R room. I don't accuse you of being an "unreliable narrator" and I don't use snarky links to insinuate things. I simply said you were incorrect, on that one. I didn't cast aspersions on your character or why you might have made the error. Can you likewise simply agree to knock off doing that to me in future? It would be a good step towards welcoming.
or you could pick up snark and insinuations, I welcome you to do so
I can accept that GMTs is the main R room, although if I were them I'd make this more obvious, because it doesn't look like that at all
the description on chat.SO says nothing about any languages
> This room was initially created in order to moderate SO R tag during the GMT time zones due to inactivity of most of the users during this time of the day. Hence, making the moderation harder. Though, with time it evolved into a room where we just make fun of @CathG most of the time.
this is on the rules page linked in the description
@AndrasDeak Knock off the gaslighting and take responsibility for your unwelcoming behavior today. I'm simply asking you to stop doing that in future. We've had this conversation too many times before, and this room is cited as 'unwelcoming' by other users (and avoided by still others). Ok?
22:58
and room owners are expected to be R-related, I guess
wim
wim
the room is a heck of a lot better than it used to be
@smci I don't gaslight.
Ignoring your complaints because I don't really feel like arguing (again) is not gaslighting. It might be unwelcoming (love that expression!).
gaslighting implies a conscious manipulative action aimed at undermining a person's perception of reality
fortunately I'm just a meh guy and not a sociopath, so I don't care how you percieve reality
@AndrasDeak Seriously? You just wrote "I apologize if this makes me consider you to be an unreliable narrator". Untruthful, unwelcoming, also, the snarky tone is clearly the remotest thing to an apology. Like most of us you sometimes write things that are incorrect, and I simply point that out, without the undertones. You can do that too.
I also don't care if I'm wrong or if I'm perceived as wrong, and I don't need justification from others to sustain my self-esteem. This removes most of the typical motivation for trying to manipulate others.
@smci snarky, yes. Unwelcoming, maybe. Untruthful? How so?
wim
wim
I care a lot if I'm wrong. Fortunately, that's never happened.
23:06
@Code-Apprentice No, just in Ruby on Rails in a controller (which is essentially views.py in Django), each method is called an action. We're migrating from RoR to Django so I'm just trying to figure out if it's appropriate to refer to each method as an action.
23:23
@earlyriser01 In Django parlance, the functions in views.py are called views.
Or one of them is "a view"
so no, we don't use the word "action"
wim
wim
did anyone here played Cicada 3301 ?
00:00 - 17:0017:00 - 00:00

« first day (3195 days earlier)      last day (1978 days later) »