« first day (3571 days earlier)      last day (1363 days later) » 

1:00 AM
Is there a way to see how many times a certain file has been accessed? For example, I have a text file and have a program that opens and reads that text file. I'd like to have some sort of standalone counter to see how many times it's been opened after I start a monitor on it?
 
1:21 AM
Does print statement debugging work for Flask?
I'm not sure what is and is not executing, and I realized the reason that nothing is printing in the console is because print is blocked while Flask is running
 
 
1 hour later…
2:49 AM
Hey guys, I need to actually do this task to do automated tasks from a website
I am looking if it is possible to do so and if ever which libraries should I look into
Can selenium do this?
The problem with the .csv file that can be retrieved from the Export feature found in the page is that some essential information such as Status tends to be blank for quite a lot of data points!! I am not sure as to why the data is available on the platform but data in the .csv file is incomplete. But it is already not in my control so one of the few information I have is the Order Number which I currently have to screen 1 by 1 to get the details regarding the Status
The data looks like this in terms of how some rows have some with no information on the columns. Had to block off some information due to confidentiality
 
3:48 AM
@biggi_ either use file system hooks (it varies a bit by OS) or create a log of file access that you maintain in the code
@JohnnyApplesauce You can flush or redirect print to a different output but I would highly recommend looking into logging at this point (print should go to the console by default in DEBUG mode so only "blocked" once in production)
@Pherdindy If it is not JS based you can just use BeautifulSoup. If it is JS based you'll need something like selenium or a combined BeautifulSoup with requests or similar setup. Just search for webscraping JS based websites with Python - there are many guides
 
@LinkBerest there were attempts to web scrape previously using beautifulsoup. It was JS based I think so it wasn't doing things correctly
            req = requests.get(link, headers=headers, proxies={"https": super_proxy_url})

            json_data = json.loads(req.text)

            if 'mods' not in json_data:
                print(f'retrying page {page}')
                continue

            try:
                for item in json_data['mods']['listItems']:
                    name = item['name']
                    original_price = item.get('originalPrice', '')
                    price = item['price']
                    rating = item['ratingScore']
Not exactly web scrapping but i'm not sure how the developer got these
The endpoints rather
But the URL is completely different from the one I want to work on now. Surely, there are some user restrictions to prevent just anyone from getting my data. There is an API for us to use probably best to look there first right? Won't the API also give me the blank data though you think?
 
4:48 AM
Apparently a lot of folks appreciate my advice on how (not) to iterate over dataFrames
@LinkBerest remember this?
This is the only answer that focuses on the idiomatic techniques one should use with pandas, making it the best answer for this question. Learning to get the right answer with the right code (instead of the right answer with the wrong code - i.e. inefficient, doesn't scale, too fit to specific data) is a big part of learning pandas (and data in general). — LinkBerest May 30 '19 at 14:26
 
 
1 hour later…
6:05 AM
@cs95 It is indeed a well written/explained answer (as is your other answers) :)
 
 
2 hours later…
7:51 AM
Hello everyone, does someone know how I can filter out timestamp values( Note: I want to exclude the timestamp values) from a field containing predominantly float values in pandas?
 
8:50 AM
Is there something one can do if a Q clearly matches a close condition, but is protected by a bounty?
@user3680510 There isn't really "that one list" – there is no science to developing software (let alone software related fields, e.g. data analysis, devops, ...) that provides a set of well-defined approaches. A good starting point these days is using the tools of github and its ecosystem. That is sufficient for most tasks, and provides a foundation to understand the things going beyond.
@biggi_ In general not unless the file itself does the counting. For active monitoring, inotify on Linux allows to track many file accesses, but not all (e.g. not on remote file systems). The available tools will vary strongly by OS, FS and privileges.
 
9:06 AM
@MisterMiyagi flag for mod explaining why it should be closed, wait a week with unhandled flag for the bounty to expire, close question, retract flag
I've got a pending custom flag from the 24th—of June
 
Right, found an MSE post saying the same. Seems like a strange workflow to protect bounties but expect mod flags, still.
 
9:28 AM
They are protected, hence only mods can fix it. They are exception handlers. A crap question being bountied used to be an exceptional case back in the day.
Mod flags are not part of the workflow, just a workaround that works very bad these days
You can vote on meta.stackoverflow.com/questions/399013/… to make more nothing to happen
 
10:13 AM
@RaphX Can you give an example?
 
5.5
6.7
8.8
25/6/2012 15:23:00
25/6/2012 15:23:00
25/6/2012 15:23:00
25/6/2012 15:23:00
5.5
6.7
8.8
10.3
11.95
13.6
15.25
16.9
18.55
20.2
21.85
23.5
25.15
26.8
28.45
30.1
31.75
33.4
35.05
36.7
38.35
40
41.65
43.3
44.95
@roganjosh its like above
its a revenue field which shouldn't contain any timestamp values
 
Ok. So we know the column is of object type. How have you gone about trying to filter it?
 
I couldn't, I googled but couldn't find any situations similar to above, most were like extracting a particular datetime rather than all datetime values
 
I'm not asking about google. I'm asking for your thoughts on what you might do
You and I can clearly see the difference. How are you going to communicate that to pandas?
 
10:29 AM
checking if '.' is present ?
 
Sounds promising :) One way would be to check that it could be converted to a float, yup
I suspect that you might be able to filter on the length of the entries rather than checking the contents of the string. It depends how big the floats get, I guess.
 
ok
 
I exploded my little finger last night trying to close the bathroom window and my typing style has become properly borked now :/ I wonder if any programmers insure their fingers
 
10:46 AM
this little piggy went "Owwww!" all the way home
 
I'm round at my sister's. Bonus - I have actual internet. Con - I lost my battle with the window. Can't have everything :P I think I'll go get some WD40 to prevent further mishaps.
 
that will only make your fingers more slippery
 
But that will mean that I can't get a proper grip on the handle to give it some real force, thus preventing me from slamming the window on my fingers
 
good thinking!
 
11:16 AM
@AndrasDeak Nice find, good read too. TY, voted.
 
12:05 PM
@Pherdindy if there's an API: always try it first
@cs95 nope (I tend to try and forget the pandas tag though). I stand by the comment though :)
 
I'm half of the mind to go digging and see if I can create an abhorrent mess, but is there a better suggestion for this OP?
Actually, I'm utterly confused now. You're trying to piggyback on Flask to grab an IP but you don't want to run an actual application? I guess it's an ingenious concept, but I can't think of any way to make that work. — roganjosh 7 mins ago
I don't like the question but the moxie of the thought process that I think they have is tempting to me to give it a shot. But that's not really helpful
 
12:30 PM
It's foundational stuff: they don't have an app defined so they cannot add routes (that's not the problem with what their trying to do just the first problem)
 
That was what I answered. I've just deleted that. On further consideration, it's just completely misguided. I can answer to the question title but what they're trying to do is fundamentally broken
 
oh, wait. I missed that they responded to that - yeah they want to make a server (to which I say "good luck" your going to need to get real friendly with ServerFault and SuperUser people)
 
Even if I could get Flask running in such a borked manner, it's of zero help because they need to be using the server itself for this kind of thing
 
yep, if their doing that they'd need python3 -m http.server or something (note: I have never setup a server in Python nor would I ever try for more than testing so I don't want to drop that as a comment or answer - it would likely cause more harm than good)
 
Ugh, is my designated bastion to try and preserve but I'm at a total loss on how to handle this one. Time to put the kettle on
 
12:38 PM
ah...this question of their's with the server tag is telling and brings me to my original statement:
7 mins ago, by LinkBerest
oh, wait. I missed that they responded to that - yeah they want to make a server (to which I say "good luck" your going to need to get real friendly with ServerFault and SuperUser people)
 
I undeleted my answer
 
I added my comment
I still feel the OP is confused about how frameworks work (that they have to run on a server and what its role is) but what can you do?
 
Eh, I tried to catch both sides in my answer. My suspicion is that the question title might be catchy enough to grab people, but the impetus behind the whole question is broken understanding about things unrelated to Flask
 
12:53 PM
hmm...might upvote the question (its researched and shows effort - helpfulness is debatable but 2/3 is rare for a question lately) - I'll give it a bit cause it will probably get a random one anyway
 
1:10 PM
@LinkBerest Pherdindy started out here trying to write a scraper for an e-commerce site that offered no API. Allegedly they gave up on trying to write fake reviews for themself.
 
1:21 PM
huh? either way my advice remains - learn the API or search for tutorials
(the fact that looking at that user's chat message and seeing a lot of "I'll learn the basic later" type comments make me feel even better about that answer)
 
@LinkBerest part of the message was "there's no API if this is the same service"
 
weird cause their last statement was "There is an API for us to use" - hence my reply :)
granted, that is my default reply to "I need to webscrape something" followed by (directly after the "I don't want to use the API" answer I will invariably get) the reply "prepare to learn what rate limiting and IP blocking is like"
 
1:44 PM
@LinkBerest ah, I just missed that, sorry
 
lol, s'alright :)
 
Yeah, definitely use the API if there's one. We spent quite some time trying to bring home the messages that "use the API rather than being a burden for the server especially since you're complaining about being rate limited" and "using a program to create fake positive reviews for yourself as a merchant ranges from unethical to illegal"
for the former the reply was "there's no API" and to the latter "I stopped doing that because it wasn't efficient enough, I just bought ads on facebook" or something along those lines
 
^ second part is untrue, should be: it straight up is unethical, it ranges from breaking a user agreement to illegal but I digress ;)
 
@metatoaster I totally see where you're coming from with your comments but I think it's easier to stick with my assertion. The assertion is not incorrect, and I feel like further detail just muddies the waters
 
I just attended an interview for a job that had a requirement of knowing python, the interviewer asks me a question then proceeds to say dont use collections or itertools, is this common for interviews?
 
1:52 PM
for interviews? yes
are they lying? possibly, kinda...in a way
 
@LinkBerest How is artificially constraining the solution lying?
 
they may use a different library for those tasks but tell you they don't use them at all to test your knowledge of Python
 
"I want to see how you solve this from scratch" seems clear enough in terms of intent
 
If hiring someone for a python dev shouldnt they expect them to solve things pythonically?
 
I didn't say it was a good way of doing it....just that its done (its actually not the worse way, honestly, not my preferred one but for other reasons)
 
1:54 PM
I understand if they were testing me for code solving abilities
 
@python_learner There are probably several strategies of conducting interviews
 
@metatoaster similar to how we keep saying "don't use the dev server in actual applications". Well, I've broken that plenty of times. It's easier to have a consistent narrative
 
Some might want to you to be able to write good code, some might want you to show that you can reinvent the wheel if necessary, some might want to throw tricky "trivial once you know the answer" logic problems at you, some might give you constraints that are impossible to keep to see how you handle it
(Disclaimer: I haven't been on a single interview, but there are plenty of stories floating around and human nature is fairly constant)
 
wow for the no interview part, but what you say makes sense
 
<senses my balloon-in-back-of-van anecdote bubbling up> :P
 
1:56 PM
@roganjosh yeah, fair enough, I think it might be better if we just delete the comments in which case
 
@AndrasDeak I misread it as the interviewer saying "we don't use collections or itertools" (and I've seen that constraint phrased that way before)
 
@LinkBerest ah, that's weird
 
yep, but then it was apparently my turn to misread something :)
 
@python_learner it's not that wow if I tell you that I'm a physicist working in academia
 
@metatoaster Sure. I don't wanna feel like I've dismissed your comments offhand. You made really valid points, I just don't feel able to make an unambiguous point if I have a go at editing. Thanks for the feedback, though!
 
1:58 PM
I dont mean to sound rude or anything but how can a physicist know so much python? I mean I have almost 4 years of college education and I barely get through lol
 
I've actually said "we don't use collections or itertools" for a bag of words interview task once (wasn't a lie - we used Spark - it has a unigram function). On top of that - I wanted to see how the person used for loops to solve it
 
I have seen your suggestions in this room to me and to others, that was my basis
 
@python_learner joke's on you, I don't know so much python :D
 
@AndrasDeak it's the math that tricks people - you just know too much math ;) :P :)
 
laurel, but yeah math-y people tend to code better
 
2:00 PM
@roganjosh no worries! I saw that you got my points quickly from the bit of back and forth, and your clarification there will certainly be useful should I deal with similar kind of thing in the future because I don't like muddying the waters unnecessarily too.
 
I've been programming for ~19 years
 
good lord I am thereabouts also with length of time since I started programming
 
O_o started with python? I assume python 1 must have been a thing back then
 
@python_learner no, programming. But after a while the specific language is marginally relevant.*
Python for ~4 years, maybe
 
technically 1989 - usable version '94 or so (but most of us were still using Perl or Matlab or shell or C or .... at that point)
 
2:04 PM
are all devs here the same range wrt to python? sometimes I dont even understand what happens here :D
 
(*within the same paradigm. I'd be an abysmal functional programmer)
 
@AndrasDeak Better or worse than I would be as a Python programmer?
 
I met a fellow at a small hackathon/conference some years back who was a colleague of Guido, they used to sneakernet Python pre-releases around at CWI
 
@python_learner no. Once people start talking about the full python internals, I have no idea what's going on. I've had to take a breadth-first dive to cover lots of tech - you're not alone
 
My math joke was in relation to the fact that I can follow Andras's code but sometimes him (or others like PM2) start going into really specific math and I end up reading a bunch of papers just so I can figure out what they were talking about
 
2:07 PM
I just wonder in 1 or 2 years I can sound as cool (python) as you guys 0:)
 
@python_learner All of us are still learning, some aren't even employed (yet).
@LinkBerest You are not alone, although I don't usually deep dive too far on the math.
 
eh, I like math
 
@CodyGray worse.
 
Shudder
 
the journey never really ends - if I had it my way I would be looking more into how rust can work with python
right now I am stuck maintaining a nodejs app that fell into my lap
 
2:09 PM
Sometimes I do weird things with probability or eigenvectors and get back at everyone though :)
 
@metatoaster I read a headline or an article that referred to some efforts in that direction. Also mentioned Rust potentially going into Linux.
 
yes, and that's precisely why I want to investigate deeper into it - the first project that caught my eye with regards to this was python-ext-wasm which is a pip installable package, but note the complete lack of setup.py yet it can produce a wheel
lots of exciting changes with all that, I might write servers fully in rust, that will produce wasm for the more interactive frontend, and somehow still be extensible via Python for the cases where it is useful
 
It certainly sounds good. Even better if I don't need to end up picking up js to produce passable demo frontends.
 
ah, to be young and not stuck in a corporate (or gov) mandated OS, language, tools, etc... :)
anyway - rbrb all
 
I expended very significant effort in vain trying to improve the integration of Python and JS from the development perspective, but it's unlikely my integration solution will get any traction (partly because I expended zero effort in promoting it, as I don't know how well my solution actually fits with how all the other devs out there manage this)...
the "in vain" bit comes from how stupidly quickly (I would argue pointless) the node.js ecosystem moves, it's practically impossible to catch up to anything because things never stablise and I wanted to provide a stable solution - I mean, my solution does work, but it may mean using a slightly older webpack or whatever
one okay outcome may have some longer lasting impact, as I did fork and fix slimit and that Javascript parsing library is actually used in many many packages. Plone now uses my fork and it's a lot less broken when it comes to how it deals with minifying JavaScript
 
2:49 PM
@metatoaster That's cool :) I'm delaying and procrastinating learning js on an ongoing basis.
 
Well, you still have all those SQL dialects to work through :P
 
3:01 PM
In my (not so humble) opinion, Node.js/Javascript is more of a language that is on a "good to know" out of necessity basis, rather than one that one might want to write programs in
 
3:19 PM
which is preferred, '/'+str_obj or f'/{str_obj}' ? all I want is to add a / to the string
 
3:29 PM
really doesn't matter, assuming you don't need to support older python versions that don't have f-strings
 
I am the end user :), on 3.8
 
4:05 PM
@roganjosh Hush.
@metatoaster No argument there.
 
@toonarmycaptain it'd be boring if I didn't stir the pot a little though, right? :)
 
4:31 PM
@python_learner If you're using user-input, there are occasions where you don't want to use f-strings. That said...*clippit voice* "It looks like you're using strings to make paths, would you like to use a Path(string1, string2) or Path(string1).joinpath(string2)"
@roganjosh It would be concerning, certainly.
 
 
1 hour later…
5:48 PM
@toonarmycaptain I would go for Path(string1) / string 2
I think overloading the '/' operator is a huge win in the Path class, along with the read_text, write_text, etc. short-cut methods in place of with blah.open() as file: blah_str = file.read() all over the place.
Though those methods do conflate the notion of "path" with "file that the path points to".
 
@PaulMcG I did not know this, thanks! I was using with's :/
 
6:18 PM
@python_learner that's not a bad thing to do, mind you.
 
 
2 hours later…
7:54 PM
is there any "database" channel here on stackoverflow?
 
@VisheshMangla no. Try chat.stackexchange.com
 
ok, thanks for the info though
 
8:47 PM
@PaulMcG That works too. I just don't like how it looks.
@PaulMcG What are you talking about in place of with blah.open() as file: blah_str = file.read() ?
o.0 Are you talking about Path.read_text(), Path.write_text() - How did I not come across these before?!
 
 
2 hours later…
10:27 PM
Not sure how I managed to call for closing a closed question. I guess I missed that the page hadn't fully refreshed or something. Oopsie!
 

« first day (3571 days earlier)      last day (1363 days later) »