« first day (4342 days earlier)      last day (830 days later) » 

00:56
@MattDMo You need to use either selenium or Playwright. Scrapy-splash is also another option. The idea is to use a hidden browser to pull up the information.
01:21
Hi Guys
02:13
@Neamah Please cut your code down to a minimal example. Are you sure load_data, Add, Save, Clear etc are essential for your question?
 
2 hours later…
03:54
@Sweetcheeks12354 Yes, I'm well aware of that. What I'm asking for is a good dupe target explaining that, so I can hammer all the questions that keep getting asked.
@Neamah Please read the room rules, specifically the first two points under Asking a Question. Questions must be 48+ hours old before they can be asked about here.
 
2 hours later…
05:30
@roganjosh Yes, in the example shown the user_loader part was not included. I do have it in my code though. As expected, since the id for the user is None, no one gets loaded
@Aran-Fey :-)
05:43
@roganjosh I am also working on Flask-principal but I really wanted to get joined table inheritance out of the way knowing that it can or cannot be sufficiently utilized. As you have noted, the example shown is primarily from the docs, and I was implementing a simple test using what the docs is recommending.
06:11
@roganjosh I wasn't sure if it was VRP either, but based on the library's name they mentioned (optapy) it seems to fit. Although I know it's used for other things too, but yeah
 
2 hours later…
09:01
anyone ever played around with p-adic numbers? or I guess work could be a better word? Just discovered that and thinking this could be a new rabbit hole I could get into
I think Sage also has support for it, so that's nice
def print_nested_keys(dic,path=''):
    for k,v in dic.items():
        if isinstance(v,dict):
            path+=k+"."
            yield from print_nested_keys(v,path)
        else:
            path+=k
            yield path
>>> [*print_nested_keys(d)] # Here, d is your nested dictionary
['Name1.NNum',
'Name1.NNumNode1.SubNodeA',
'Name1.NNumNode1.SubNodeASubNodeB',
'Name1.NNumNode1.Node2.SubNodeA',
'Name1.NNumNode1.Node2.SubNodeASubNodeD',
'Name1.NNumNode1.Node2.SubNodeASubNodeDSubNodeE',
'Name1.NNumNode1.Node2.Node3.child1',
'Name1.NNumNode1.Node2.Node3.child1child2.grandchild.greatgrandchild1',
'Name1.NNumNode1.Node2.Node3.child1child2.grandchild.greatgrandchild1greatgrandchild2']
Basically it is printing all the keys from dict
In this line- [*print_nested_keys(d)] , what does the * indicates?
* unpacks the iterator in the []-list.
Compare [*range(4), *range(2)]
I think some other languages call that the "splat operator"
@MisterMiyagi Thank you
 
2 hours later…
10:51
stackoverflow.com/questions/11813387 I re-opened to try to close under the new system
@KarlKnechtel I've hammered it to the first dupe; I assume you want the second (which links to the first) not to apply?
you understood correctly but somehow gave the wrong link :) I fixed it, as well as cleaning up the question generally; thanks
11:08
Ooops, yeah, that was a brainfart on my part. :)
It's nice to see you cleaning up things every now and then when I check the main page, by the way. Wish there would be more people doing that still.
11:38
not only do I clean things up, but I try to figure out more organized approaches to the cleanup as I go
you saw my other room, yes?
12:35
Hi guys, is making a "canonical post" on how to make a web scraper in python a good idea?
@DialFrost by that, you mean asking a question with the goal of answering it yourself and making that a canonical? or did you mean that for the answer?
I think scraping is too broad of a subject to make a good canonical for
@Aran-Fey Mkay
Like just on extracting stuff from the html code, nothing special @Aran-Fey
And the answer is just a requests.get followed by a BeautifulSoup and then a soup.find? Is that really useful?
I mean, there many other ways to extract values from HTML pages...bs4 sometimes doesn't work if js is involved, etc
but as you said, it's too broad
12:44
@KarlKnechtel Yeah. Don't have much time for SO right now but want to drop by once the after-summer-rush has died down here.
we are lacking a good canonical for just "q. why didn't beautifulsoup work? a. the content is produced dynamically; you have to either use something that can run the javascript, or figure out how the javascript works and do something smarter; or look for an explicit API"
13:19
@Aran-Fey funny that you just discussed this, I just had to do exactly that 5min ago
Welp. Then I have no more objections to this kind of canonical
this will be a pretty huge canonical to be honest. You'll have to pack bs4, selenium, requests, playwright, etc in a single answer
unless you go for the "it work most of the time" way and be done with it
How would you approach a situation with a coworker if you suspect he has: en.wikipedia.org/wiki/… ?
probably a better place to ask would be workplace.stackexchange.com but if others have had situations like this, they can also answer here.
(I never had so that's why I'm recommending this)
good point
ugh, just missed the window to delete
13:39
@MattDMo Sorry for that I am new here
Does somebody know of a good bash cgi generator from python? I'm not exactly sure what I'm looking for, but I have a busybox with a cgi which runs bash scripts to return the html. And looking at that code, it's horrendous. Now I was wondering, if there is a nice python program which outputs bash cgi, if you get what I mean :D
why not just directly use python to output/return the HTML? I mean I know there is bash CGI and python CGI, but I don't think I ever seen one that generates the other (especially given they both already generate the HTML anyway)
It's a small busybox. It doesn't have enough RAM to run python :D
Oh I just remembered micropython, I gotta try that
13:56
I was about to mention that yeah...there is also other fork of python for embedded I think, like ironpython etc
 
4 hours later…
17:58
@Hakaishin Another option is to write your CGI script in awk, which is nicer & generally more efficient than processing text in bash. For that matter, simple text processing in awk is often faster than Python.
I guess it's not so relevant for CGI scripting, but U&L has an excellent canonical on the pitfalls of using a Bash read loop to process text files. unix.stackexchange.com/q/169716/88378
TL; DR: Bash read is expensive because it uses a syscall for each character that it reads.
18:36
yep, and even if you try and do it in huge chunk, I recall that read has a limit of chars it can read at once
I wanted to ask you this since I mentioned it earlier: tried to do things with p-adic numbers? I noticed it's also supported by Sage, so thought you might already know about it @PM2Ring
@Hakaishin you would approach them like any other human being as we each have our own idiosyncrasies. There is no answer in the general case, only for particular scenarios where you think you might be able to support some outcome by working with them differently
19:11
@NordineLotfi Well, read is a line reading command, so it's constrained by the line length limit of the OS. But even if the file you're reading has big lines that doesn't really help much because read makes a separate kernel call for every single character.
@NordineLotfi Sorry, I've tried to study p-adics a few times, but it just doesn't "click" for me, and I've forgotten most of what I learned about them. IIRC, there are a couple of regulars in the Math chat who know about them, but there's not a lot of number theory discussion in that room.
@PM2Ring that's true, but it does help quite a bit depending on what you're doing. I remember abusing read a lot back when I didn't know python and wanted to process files in chunks. I timed it and the more chars you process at once, the faster it goes, until you hit the limit of the OS, as you mentioned :)
you also have to take into account the "remainder" of the file, but that's not really an issue
@PM2Ring I see :o I also find it very rare to be discussed on SO/SE post, and only found 10 or so post about them :/ I think I only found about 3 repos on github with python, so that's a bit sad
what made me look into it was this youtube video I found once.
19:44
@NordineLotfi Greg Egan wrote a novella involving 3-adic numbers. He wrote about it here: fromearthtothestars.com/2018/09/05/inside-3-adica
@PM2Ring nice, didn't know about this. Thanks!
If you like stories with a mathematical flavour, check out kasmana.people.cofc.edu/MATHFICT
that's pretty good too, thanks :D
20:40
@Neamah No worries. Welcome!
@MattDMo not related to python, but I saw you worked on woods too (based on the last couple entries on the chat from woodworking.se) :) although I think your post was deleted so I couldn't see it in time
Yes, I'm an amateur woodworker of middling quality. I love doing it, though, and am always looking to improve. Unfortunately, my tiny shop doesn't afford me enough space for new power tools, so I have to make do with my SawStop (literally takes up half the shop) and small/hand tools.
Didn't you post on WW a little while back? I thought I saw your name...
@MattDMo I know what you mean. I can't afford professional tools but even if I did, it wouldn't be possible for me to keep them because of space. I was curious to see what your post was though :P
@MattDMo I did yeah. It's in the recent post history, but if you're curious: woodworking.stackexchange.com/questions/14089/…
(I'm also an amateur woodworker btw)
Oh, I was still in the market for a table saw at the time. I knew I was going to get a SawStop, but I wondered if I should just get the "cheap" fence that was included in the price and retrofit it with a Delta Unifence. I used to have a Delta long ago, and really liked the fence. However, shopping questions are off-topic there, so it got closed and eventually deleted.
ah, that explain it then
20:53
Not to worry, though, because the "cheap" fence has actually turned out to be pretty awesome.
that's good news
How is your case project going? I once built a case out of plywood :)
FYI, everyone, this conversation is on-topic because it's about building a case for a desktop computer that will be used to run Python.
@MattDMo it's getting along, at least in my head :D I'm making simulations in my head and measurement on paper. I think it should work since I don't care about it being fancy, just looking good enough and a few holes for the ports and airflow
@MattDMo indeed! I wanted this for running some stuff using multiprocessing, especially since I figured out how to use it (I think)
I also planned to make the simulation using python but for now it's just in my head
Well, good luck with the design and build!
Thanks :D
21:01
I haven't played around much with multiprocessing, or even threading, for that matter. A lot of the stuff I run is pretty low volume processing, and the (potential) speed gains of using multiple threads or processes would be completely outweighed by the extra time needed to implement it. Not that that's ever stopped anyone before :)
yeah, that's how I think too. It's already complicated enough for me to make functions and the internal logic, then have to also take into account inheritance and classes...if I add another layer of complexity, it's even harder
I just notice it could work for smaller test case I do on some input/output, so that was kind of nice
one thing that I did once was to see if a number was divisible by any number between 1-100000000. Using multiprocessing over a range is easy, so that was fun.
the author of that answer also made many other nice ones with multiprocessing
What are you thinking of using to do the simulation? I assume you're modeling heat and air flow...
hmm, I didn't think too much about heat and airflow with the simulation. I think I only thought of "which parts will be inside the case? which way it will fit more nicely..." that kind of simulation. I think taking into account airflow and heat would be nice too but I feel like it's even more complicated
Yeah, that's why I was asking. I know a very small amount about CAD, but I'm also starting to get into 3D printing, so I'm trying to learn more. It's so crazy powerful that doing simple things like designing a screwdriver holder for my pegboard takes forever.
If you're not considering airflow and heat then what are you simulating, sorry?
21:11
I think I'm saying simulation to say "I'm imagining in my mind" basically
I'm thinking back to my old CFD modules. I forget the name of the software we used but I never really appreciated it at the time :'(
Navier-Stokes FTW
also yeah, CAD is powerful but mighty complicated. It's basically just like how I feel about Blender.
My one venture with Blender: I set up a turbulent ocean scene years ago and it took my laptop all night to render it in all its glory. roganjosh wakes the next day, completely stoked to see the result. --> Press play, and within 2 seconds a giant wave went over the POV and the remainder of the scene was just sitting under water with no rendering effort at all
I mean, the first 2 seconds were glorious, but talk about a deflating experience
Fail.
Naturally, I blame Blender
21:20
As it should be
you know, I once put in my never-ending todo list to play around with blender-as-a-python-module once. There also a "game engine" with blender (or at least you can use it as one) using python.
it's not as slow as one time rendering since you're moving each object inside of blender, although...if you ever want to "export" your games, good luck
I've wanted to play with game engines for quite a while but I think I've firmly come down on the side of "I wouldn't even bother with trying Python to do it" (unrelated to Blender). PyGame is valiant but it's just never going to be
I once started learning C# just so I could play with Unity, but I lost interest after a few weeks. You wouldn't believe how much space Visual Studio, the Windows APIs, and the Unity API take up.
More than node_modules?
The more I learn other programming languages, the more I love Python.
Never did C# but I tried Unity on free trial once. I can confirm it's horrible (in terms of space loss at least)
Pygame is fine as a game engine if you take into account that: 1. you're limited by the C API from Pygame, but you can use opengl and other third party opengl/mesa modules to fix rough edges. 2. doing 3D is possible, but only high performance if you use tricks or doing voxel type of graphics, otherwise 2D is really performant
21:29
This has prompted another ambition (one that I'm slightly more held to now). I have two half sisters; they want to build a robot. One is 3 and one is 5. I was thinking of getting an Arduino and then trying to come up with some ideas from where. I know of DIY robots to shoot mini canons etc. but that's probably not going to be their desired outcome. I don't suppose anyone has seen a decent DIY robot that might fit the bill?
guys am un super terrible situation... has anyone used github gist?
need urgent help ... I was doing deployment and also maintaining steps in online github gist in browser. While scrolling on my mouse pad vertically I scrolled horizontally, and browser went back, now am not able to see my uncommitted edits in Edit mode...
Do I understand this correctly, you were coding in your browser? And now your new code is gone?
@roganjosh you might already know this already, but there is a nice list of existing projects with arduino and other board on create.arduino.cc/projecthub
there even seems to be some tagged "for kids"? might give you some ideas
21:40
It was not a code, but deployment steps... kinda documentation stuff. I accidentlay scrolled horizontally. And it went back from edit mode to the earlier version of gist
It sounds like you're on a Mac? You should just be able to go forwards in the browser I think (I disabled this nonsense right away)
In any case, you'll probably spend more time trying to figure it out than just re-writing the code. You're not doing serious development in gist for deployment?
yeah I just did it, I did forward and all typed uncommitted text was there... damn!!!
thanks!!! I got super-panicked for a while!! on tight schedule and loosing those steps was strict NO...
@roganjosh didnt get ...
@roganjosh some people put 100 files on a single gist. Wouldn't be surprised if some are actually doing that
you can see those kind of gist if you search using any query, then going to the last 3 pages
@NordineLotfi Just code it up locally and dump into a gist once you're done? It's not exactly a great coding environment
yeah, indeed
21:44
In any case, I apparently saved the day with a guess. I had no idea that would work, so happy days :P
@Mahesha999 In any case, turn this off because it's just abominable
@Aran-Fey I think I did that kind of answers once or so on unix.se when I didn't know better.
I've managed to use "in any case" 3 times in the last 5 minutes. I don't know what's happened; my language ability is either dulled or I've become some ultimate guru of objectivism. In any case, I'll make a conscious effort to stop that
yep, I noticed it too...
21:50
I can only offer my apologies. I'm better than this...
Must've been influenced by the earlier talk about Nordine's case
this might be my fault yeah. My bad :P
I mean, I'm happy to blame Nordine if everyone else is? Don't do that again, Nordine!
Yeah, josh is case-sensitive
ROgAnjOSh
I did it. I confronted my fears, and I'd get away with it without regex too if I really needed! Yey!
@NordineLotfi I'd forgotten about this; I was more in the musing phase. The spewing-smoke-robot on the landing page is a good start. In the back of my mind, it'll eventually end up in a show-and-tell class down the line and, the more disruptive and obnoxious, the better
"My brother helped me build this"
Though, there's another side that would quite like them to be able to tinker with it too as they get a bit older and maybe make it do some stuff. It almost certainly has to be mobile. They've been bugging me about it for a week, though, which is a long time in the memory of a 3 yr old so maybe it's not an entirely-passing phase
22:06
@roganjosh :P
@roganjosh Yeah, there was even one that seemed to be remote controlled with wheels somewhere, think it's this one: create.arduino.cc/projecthub/jithinsanal1610/…
@Aran-Fey it took me longer than I thought to get this one
@NordineLotfi This looks great, thanks. I totally missed the idea of looking at the official site <facepalm>
22:35
@roganjosh ohhh daamn yes... need to check what Windows equivalent ...
@roganjosh Never used arduino, but I had a feeling there would be some decent project on the official site, especially since raspberry pi's site has some similar stuff on theirs: projects.raspberrypi.org/en
@Mahesha999 Windows equivalent would be to disable trackpad gestures. It depends on your driver, but I bet it's actually using Synaptic drivers, which I always disable and replace by default mouse driver. YMMV
it's good to do this with an external mouse as backup, just in case you can't use the trackpad anymore and want to save yourself
I moved from Windows before it really had this to Mac and then just assumed it was the default here. If Windows were to pick anything up from Mac design - this was the worst feature. I don't get how anyone uses this. Anything embedded (picture or table) and you scroll around enough, you just start hitting "back" on the browser. It's awful.
I agree, that's why I always disable it
It really feels like the default design is just to pander to people that scroll endlessly through social media. It makes me sad to think how this became the default behaviour shipped
I guess it's because of the "touch screen" craze. I don't know who started it, but you start seeing it on Mac, windows 8 etc and then it's everywhere
then they emulate it on trackpad

« first day (4342 days earlier)      last day (830 days later) »