« first day (3670 days earlier)      last day (1271 days later) » 
06:00 - 21:0022:00 - 00:00

user13428955
6:54 AM
Dear mentor, thanks for You kind assistance. I have updated my code. But I am still facing the errors. Please may have a look.
Thank You
 
7:16 AM
@roganjosh hey man just so you know that actually worked for the submitting and getting past the first barrier, could I ask i seem to recall you telling me I should serialize the forms to make it simpler, when you serialize the forms do you need to do anything within the serialized data before validating? Seem to be recieving "invalid option" from a select field now not sure if it is because of serializing the data and not doing anything else once sent just doing standard validation
 
8:12 AM
Hi, this is not related to this chat room. I want to know whether there is a chat room for statistics questions ? Thank you
 
8:43 AM
@YatShan try chat.stackexchange.com/… but not familiar with their chat culture
doesn't look terribly active I'm afraid
 
@tripleee Yes, it seems that last conversation took several days ago
 
hello! i'm trying to write each json object to a new line, but its not working - i'm not sure why as this code snippet has worked successfully in the past. would really appreciate it if anyone could take a look at my code! codeshare.io/5zrX8r
 
in how far is it "not working"? Do you get several objects per line, or one object over several lines?
 
one object over several lines
what's really wack is that sometimes it'll write just like i want it to (i.e. one object over several lines), i'll scroll through the outputted json file to manually check that its alright but when i scroll back to the start of each line, its not starting with {"data": {"all_awardings": [],
when i open past json files in vscode, they are totally fine. i thought it might've had something to do with my vscode environment, but its just this code that's like this
 
are you sure that the content of these objects does not include newlines?
 
8:50 AM
entirely sure. i've scrapped reddit like this before
 
@cookiestarninja that sounds suspiciously like formatting characters in your data
but it's hard to tell without knowing the actual data
does it look weird in other viewers as well?
 
what else can i use to check the formatting of my json?
i've tried chrome but even my other json files aren't showing correctly
wdym by formatting characters in my data?
 
things like \r, \b and such.
 
i have outfile.write('\n') - which worked the last time i scrapped reddit, but my json output looks the same with or without this line
i just ran my script on a different subreddit and it's formatting correctly.. i'm not sure if there's anything special about the r/politics subreddit??
 
9:18 AM
I can only guess that some posts include characters that your viewer cannot handle correctly.
 
so more of a vscode environment thing?
 
@cookiestarninja opening the file as a binary stream might make debugging easier. it feels to me like something in the decoding going wrong, and having the raw data of the offending line side by side with its (probably incorrectly) decoded counterpart will show you what you need to do
 
@Arne
thanks i'll have a look!
 
9:56 AM
@Asif 1. You don't have to call us "dear mentor"; we're all equals here. 2. Your code shows the exact same problem that I told you earlier. The signature and function call don't match.
 
@Asif This would be much simpler if you were to construct an MRE. The underlying problem is that you passed function parameters are ambiguous with respect to the function arguments; there is no need for dataframes, meshes and train/test sets.
# Consider:
def foo(a, b): ...
foo(1, a=2)
What do you think is the value of a and b in this case?
 
10:22 AM
You know it's going great when the top starred comments here are all cynical...
11
 
oops, there goes another
long time no cbg
 
cbg
always good to see people stumble back in here
 
"stumble" :)
 
"shamble" has negative connotations :P
 
10:36 AM
@MisterMiyagi especially after they've had a couple of pegs
cuz that's when the stumbling really begins
 
hey cs, cbg!
 
csbg?
 
saving a few letters on the internet, that always gets a vote of approval :P
 
hiya
when converting coffee to typing sounds efficiency is of the utmost importance
 
Not sure who's doing the voting and who's keeping count, and definitely not sure who's reviewing whether the final result makes any ounce of sense, but hey, the complaints department didn't receive any negative feedback so it must be fine! Although then there's the problem of not having anyone actually in the complaints department in the first place...
 
10:56 AM
You might want to file a complaint about the empty complaints department to the complaints department.
 
Can't get negative feedback if I don't have a feedback box insert dude tapping his temple smugly meme here
 
Finding multiple bugs including a hard to trigger but still existent race condition bug right after writing unittests is the greatest feeling ever :)
Also man are integration tests important. The unittests all pass, but only because the race condition only appears when the different components play together :P
 
 
1 hour later…
12:29 PM
cbg
 
1:05 PM
Hello everyone I have a question: is it a good idea to have a variable name that happens to be the same as a function argument? or is it bad practice?
 
I only do that in case the arguments needs some preprocessing done, eg : arg_1=arg_1.strip() or arg_2=sorted(arg_2)
 
That works if you don't then need your originally passed variable... it's generally best to avoid it though...
 
Will it make sense to store data from 5 different sources (same structure) in 5 tables or in one table with a new extra column to tell where the data is from? will having more rows in a table affect the querying? will use sqlite3 for this
 
Cool thanks @JonClements
 
@python_learner generally makes more sense for one table with an indicator... else if you suddenly have 3 more locations - you don't particularly want to then have 3 more tables... it just makes querying much more awkward
 
1:15 PM
thanks @JonClements, so querying a table with 10k rows and querying a table with 100 rows will not be a difference? (time to get results)
 
well... there will be a difference depending on the query and how the table is defined... if a query ends up having to sequentially scan a table of 10k rows... it's obviously going to take longer than sequentially scanning 100
 
but your point of suddenly having 3 more locations makes more sense to me now, I will end up in that situation, so I guess I will just add an indicator, thanks
 
If location_id is something you're going to be using a lot in queries, then make sure to create an index on it
 
@Arne fixed - you were spot on! it was literally because i specified encoding='utf-8' when writing json to file 🙃
thank you!!
 
happy you could solve it =)
 
1:28 PM
@Arne oh hey... I think I got a notification the other day from mailman for the Python job board stuff saying it'd removed your email as it'd recently bounced too many times or something?
 
1:47 PM
Geometry puzzle I discovered the other day: start with two points a and b, with a.x!=b.x and a.y!=b.y. Using only the line and circle tools found in MS Paint, draw a circle whose center is a, and whose circumference passes through b.
 
From PyData:
Check out PyData Global 2020 (November 11th-15th), the very first fully-online PyData Conference! The Tickets are now live, go and get yours today by going
[through the link](https://global.pydata.org/pages/tickets).
 
More formally, a brief description of the actions available to you:
- you can draw a line passing through any two points.
- you can draw a horizontal or vertical or 45 degree diagonal line through any one point.
- you can draw a circle by drawing a square, and inscribing a circle inside it.
- given two points A and B, you can draw a circle such that A lies on a corner of the circle's bounding box, and B lies on the circle's circumference.
- you can draw a point at the intersection of any two lines and/or circles.
 
@JonClements We should mention, though, that tables with millions of rows aren't unusual. I think our current largest table was about 600 million rows until we downsized it.
 
So far I've been looking for a way to rotate A about B so they lie on the same X or Y axis. Then it's easy to draw the square that describes the goal circle.
 
@JonClements oh, now that I'm checking it, I am not getting mails since a week ago. I wonder what went wrong there
Is the slack group still active? I wanted to check something about that anyway
 
1:55 PM
I am struggling with my bachelor thesis :( It's about aspect-based sentiment analysis, anyone knows how can I get some help on the topic?
 
@Arne not that I'm aware - I mean it might be but even if it is I doubt anyone's there... wassup?
@holdenweb sure... transactional data can grow quite large... quite handy to be table partitioning in some cases like that though... definitely wouldn't want to be using sqlite3 for anywhere near that number of rows though :p
@dsuzer from you lecturer/teacher/professor maybe?
 
last time I tried to resolve something I got stuck, I just wanted to make sure I can do everything I'm supposed to. And I am getting a little more air in my dayjob for volunteering, so now might be a good time to get that fixed
 
stuck in any particular way?
 
Nope. Though sqlite is surprisingly efficient, I wouldn't like to push it to PostgreSQL levels. About ten years ago I saw a talk at OSCON by a bloke who was managing a 15TB PostgreSQL database full of medical records.
 
I'll try to repeat the steps, it's been a while
 
2:04 PM
cool... feel free to drop me a message if you need anything
 
will do, thanks.
 
2:20 PM
Hello again everyone, if you had a choice of replacing values in the column of a pandas dataframe using either 'loc' or 'replace' , which one would you use and why?
 
What are you trying to do and how do you imagine using loc to replace values?
 
df.loc[df[inputColumn] == 'skidos.axeman', inputColumn] = 'Axeman'
So this is one of the many manipulations I need to do
I was wondering which would be better for the replacements. For now I only know that replace uses a re.sub under the hood.
 
nope... it doesn't need to use a regex... it's definitely more flexible as you can replace multiple values at once or even apply different replacements per column...
it's definitely worth having a read through what DataFrame.replace can do and its examples
 
just at a glance, with no context about the conversation that has happened before, your code snippet looks like you're only changing one column at a time.
Honestly at that point, why not directly work on the column series itself? and perhaps use a map or something similar?
take a look at that
 
2:37 PM
Yes its one column at a time. So map will be more efficient than loc right since its similar to vectorization? @ParitoshSingh
 
Mapping would be slower because it will run in a loop
 
mapping doesnt use vectorization afaik, but you'd have to use loc multiple times if you're doing this task more than once, and so it should easily win out.
 
@Kwsswart Sorry, I don't really follow what you're asking. I don't see why you would get an error from the serialization. This is on submission of the final form, right?
 
However it' s the readability that you should favour first and foremost, and map is a lot easier to understand there imo.
 
Cool, thanks a lot!@ParitoshSingh@roganjosh
 
2:42 PM
@roganjosh essentially I got around this and have the ajax working correctly with dpaste.com/88XF9WWJ8 and dpaste.com/656P5CVEZ
I would love to say its fully functional but I have an issue at the moment where the ajax submissions work perfectly fine and going through but trying to figure out a way to undo the e.preventdefault() I had to do at the start to delay the form submission to allow the ajax and if the ajax returns the form.errors and displays it to leave the preventdefault() the way it is so they can fix it before resubmitting
 
@Kwsswart ok, I'm confused. Why do you have that loop running on form submission? You aren't sending the form body as far as I can tell
 
Ok #update_form is the form that updates progress, I also have a form generated for each student in the class that doesn't have their own submit buttons but rather what I am trying to do is that on clicking submit on the update_form form it the ajax will send the forms for each student individually to this route dpaste.com/96QJ4DNK2
so this loop is to send the forms of each students attendance via ajax before finally allowing or not allowing the update_form to finish the submit process
this route seems to be working well adding the marks and attendance well, and sending the errors to be displayed as needed in the ajax
 
Hey guys, if you were to create a class such as Temperature, Density, Score, etc. would you make them a subclass of a float type so they can be used directly in the program such as Temperature() + 10 as opposed to Temperature.calculate() + 10? Or is it a bad idea subclassing builtin types?
 
however what I am trying to do is on returning if it has had to display an error message from the forms.error then the update_form submit will be prevented however if all was fine and no error was displayed then allow the submit to go through
 
2:50 PM
@Kwsswart but it doesn't send the form, it sends JSON payloads of form data. This is what I'm not getting. Why have the forms at all if you're going to manually grab data from the page and dump it into JSON manually? You can have subforms in WTForms
I just think the setup is... wonky. But I don't have time to help with it right now until this evening sorry
It might also just be me taking a cursory look in between running some queries :)
 
well this is the thing when sending it with the original serialize i was receiving the form.error comming back saying 'invalid option' when running the validation which wouldn't make sense as it was one of the only two options available in grabbing it manually i went around and allowed the form validation to take place correctly... I am sure I may have been missing something with serialising it
Thanks anyway will continue here to see if i can find a way around this
 
@multigoodverse What is the purpose of these being classes? So far they just look like variables.
 
Ok, we'll look at it later. In the meantime; an analogy would be that you want to send me a multipack of crisps that you already have. You could just give me the bag, or... you could open the multipack (the outer form), open the inner bags (the inner forms), re-wrap each bag in cling film (the JSON body) and then pass them to me. Both end up sending all the crisps to me, but only one approach really makes sense :P
 
@multigoodverse Consider that if you define both a Celsius and Fahrenheit float-subclass, you accept that Celsius value is compatible with Fahrenheit value.
 
@MisterMiyagi how so?
 
3:00 PM
So, be careful to balance convenience with correctness.
 
They are as compatible as Dog and Cat
 
@AndrasDeak By treating both as numbers, they become implicitly compatible. Because number + number is valid, for example.
"Though shalt not mix Dogs and Cats" is the moral of that cautionary tale.
 
@PaulMcG Such classes are responsible to scrape a value. E.g. the Temperature class scrapes a Temperature value from a specific URL. Other parts of the program need that value.
So, the question is whether it makes sense for Temperature to inherit from float.
 
numbers can't scrape websites, so no
 
@Kevin as per the Single-Responsibility principle?
 
3:06 PM
Essentially
 
Don't confuse "has-a" or "is-implemented-using-a" with "is-a"
 
3:34 PM
sometimes, life's toughest questions are answered by: just use a dict.
(and sidenote, in general you wouldn't have had to subclass float for allowing addition, it would have been enough to define specific dunders as necessary.)
 
Dipping my toe into group theory... Is there a name for "a finite set and an operation that can be used to generate an infinite set"? For example, the set {1} and the operation "+" is the 'generator' of the positive integers.
{2} and "/" is the generator of all powers of two, because e.g. 8 == (2/((2/2)/2))/((2/2)/2)
It's not a "closure" because {1} is not closed under addition. The positive integers are closed under addition, but that doesn't tell us anything about the relationship between the positive integers and {1}-and-"+"
 
hey guys how can i import a module using a variable?:
for example i have a variable
a = "random"
import a
i also tried
>>> d = __import__(a)
>>> d
 
3:51 PM
__import__ works on my machine.
>>> a = "random"
>>> d = __import__(a)
>>> d.randint(0,10)
4
 
ohh ok i got it
 
@Praveen I hope you're aware of the obligatory "but don't" warning
99% of people don't have a good use case for dynamically importing things
 
i thought random.randint(0,10)
 
Perhaps you're thinking "yes, but then I have to refer to the module as "d" rather than its real name. I want to be able to do random.randint(0,10), not d.randint(0,10)." In which case my question is: if you know ahead of time the name of the module you want to import, why not just import the module normally?
 
no there are a bunch of modules so the user chooses which one to import
 
3:54 PM
Basically the only use case I can imagine for this is if you're creating a custom REPL, and you don't know which modules your users will want to use, and user A is in charge of deciding which modules are available, and user B will be using those modules
Despite how oddly specific this use case is, this problem seems to come up all the time
 
no i am making my own modules
 
Great, so just import them normally and you're good
 
morning cabbages, folks
 
Life is tough in the footgun mines :)
 
no the user chooses them
there are alot of modules dude
 
3:57 PM
If it's less than ten thousand, I think Python can handle it.
 
yeah but i dont want my python to run slower lol
why shld i type all of them just let the user do it
 
Importing lots of modules does not make Python run slower after it has completed the importing process.
 
Your user will probably not be a user for long
 
@Praveen Good point. Simply make your user write the import statement themselves.
 
ok so its not like tht
 
3:59 PM
Don't let those lazyboneses try to do random.randint(0,10) unless they do import random first
 
well i said u only the 5 percent of it, we have took care of all the posibilities and the logical part of it
 
If your REPL has a few dozen imports that are common to your particular application domain, importing them ahead of time will be a plus in the eyes of your users.
 
i am not doing an REPL
 
Oh no, I dropped my footgun~
>>> s = "random"
>>> globals()[s] = __import__(s)
>>> random.randint(0,10)
7
Ah, now everyone can see it :-(
 
@JonClements I must have given up too early last time, as far as I can see everything works fine now. I'll look into my mail issue and get back to you if I think I got it to stop being a mean bouncer.
 
4:01 PM
@Kevin thank you
 
Feeling like the old man from that one comic right now
globals() is a burden that brings death
 
@Kevin this one moves i.imgur.com/Sj0NURP.gifv :)
 
i watched the 'joy of painting' today and i am now addicted to it
 
@Praveen is that Bob Ross?
 
@AndrasDeak Thanks
 
4:05 PM
now i want to become a painter lmao
@AndrasDeak yep
and its rated 9.4 in imdb ....what!?
 
probably similar rating than Mr. Rogers, if the internet is any indication
 
Yep they're basically internet divinity. The remaining slot in the trinity being filled out by Bill Nye or LeVar Burton or similar
 
or Keanu
 
Steve Irwin, Jim Henson,
 
@Praveen will the user also decide what to do with these modules?
Otherwise, you can just import them in the functions that need them.
 
4:11 PM
@multigoodverse I'm not even sure why you want different classes for your different values unless the type of value is significant (e.g. if you did use both Centigrade and Farenheit temperatures and wanted interoperability).
 
no users decide
everything is upto users
 
If it's not a REPL, it's unclear to me how the users can even access the module object
"an interactive calculator where the user can type math.sqrt(4) and see 2" counts as a REPL if that's not obvious. It doesn't have to be a full-featured Python interpreter.
 
ok dont worry i will share the project once its over then u may know it
 
@Praveen then why not just eval whatever they want?
 
its basically what begginers do after learning python
but some added spice to it
 
4:13 PM
Building unmaintainable, huge ML models that break randomly?
 
@holdenweb tq i got the answer already
@MisterMiyagi what do u mean?
no not a ML model i am not interested in ML
ML is overrated in noobs
or overhyped
 
lol
 
@Praveen Was just pointing out that unless you have to have Py2 compatibility importlib.import_module is a better bet.
 
ohh ok
 
4:23 PM
Any sqlalchemy gurus online? I'm using it to make some adhoc queries in a one-off python script. I am trying to figure out how to use where(), but all the examples I find require something like select([mytable]).where(mytable.c.somecolumn == 5). I don't have a mytable object, though. Can I get one by introspecting the database? Or can I just build the where clause from text instead of python constructs?
 
If you don't have classes representing your tables, then sqlalchemy might not be the right tool
What's that library people use to do string queries? it has cursor.execute, and we always yell at people not to do interpolation...
sqlite3, I think?
 
Each db engine has its own API in Python...or ODBC
one reason I'm using sqlalchemy is I'm playing with an MS SQL Server and a MySQL at the same time.
 
Or maybe I'm working backwards here, since the database you're using will dictate the module you import, not vice versa
(most of the time)
 
I can always build the SQL statement as a string and use sqlalchemy to execute it. I'm trying to learn more about the sqlalchemy API and so far found some nice stuff to avoid building the query manually.
 
This looks useful: stackoverflow.com/questions/17972020/… ... oh, you already know how to execute strings. Never mind.
 
4:29 PM
I definitely need to learn more about the table classes.
 
AFAIK your two choices are "forego all of sqlalchemy's nice features and execute the string query raw" and "define classes that represent the tables you care about"
 
yah, I'm comfortable with executing strings. I am using this task to learn more about the higher-level apis available in sqlalchemy
 
It'd be neat if sqlalchemy could poke around the db and autogenerate the table classes for you, but I don't know if that's a thing
 
Maybe I should look if there's an API to create table classes automatically from an existing db schema
I'd be surprised if it's not a thing
 
Google points me to docs.sqlalchemy.org/en/13/orm/extensions/automap.html, so maybe it is a thing
 
4:35 PM
@Code-Apprentice You cannot use the ORM if you don't have a class that inherits from Base (unless I'm missing a trick)
You can still use SQLA raw queries, as you suggested
 
Going by the pages and pages of documentation detailing how to fix it if it screws up, I'm guessing this isn't foolproof enough to be the de-facto standard way of generating table classes
If you want an ironclad schema, write one yourself
 
@Kevin Any of the RBDMS wrappers, which will implement the DB API spec
 
👍
Three cheers for standardization, and only one ugh for the unfortunate drawback that it makes it impossible to tell what library a querent is using when they helpfully omit all their imports
 
Actually, it seems that I was missing a trick and automap is the thing to use. Huh TIL
 
Not that the inclusion of imports would result in an MCVE anyway, since querents invariably omit their table definitions too
 
4:41 PM
It generally shouldn't matter other than things like param style where different dialects have different params that will work (SQLite is ? for example, while Postgres is %s)
 
@roganjosh ok, thanks. I'll just build the query strings myself then. I already have a Django ORM and don't really feel like duplicating the whole thing in sqlalchemy just for a one-off script.
 
I mean, I think I may be giving too-rosy-a-picture in reality of how far standardisation actually goes, but I prefer to leave you feeling like everything is addressed :P
 
My other option is to forgo the where clause and do what I need in Python instead...hmm
 
<header class="masthead"
style="background-image: url(' {{ url_for('static', filename='img/home-bg.jpg') }} ')">
visual studio code shows two errors on **static** and **bg.jpg** respectively.
I'm not an HTML guy actually I was trying to make a flask app.
I copied the above code from a youtuber and he gets no error probably that's because the video I saw was 2 years old . How can I fix this?

The original code waithout Jinja tempalting was :-
<header class="masthead" style="background-image: url('img/home-bg.jpg')">
 
@Kevin sqlalchemy can introspect existing tables, I believe. The issue is that it gives you two way to access your data: tables and declarative bases, which were different enough to confuse me (not that that's difficult).
 
4:54 PM
@Code-Apprentice it does look like Automap might be worth a quick punt before ditching your initial plan
 
@Code-Apprentice You could also try using littletable. You would still need classes, or at least SimpleNamespaces or namedtuples for the table elements. But if you are loading the database from a CSV or JSON file, you might be able to skip the db (and the schema parts) altogether.
 
@TanishSarmah What file type have you told VSCode it actually is?
It will infer HTML from .html but you actually have Jinja in there. You can download the Jinja plugin and set the file type to "jinja HTML"
If you don't do that, but have format-on-save enabled, it will repeatedly bork JS every time you save the file. I got bitten by that when I first moved over
 
@roganjosh Basically a file from bootstrap's clean blog folder
 
@holdenweb @Code-Apprentice yup... there's also sqlacodegen which is a useful tool
 
@TanishSarmah That wasn't what I was asking you, though
 
4:57 PM
OK, time to rhubarb. My barber only works Mondays, and I scored his last appointment before lockdown!
 
@roganjosh now that I'm digging into this more, looks like I can do this easily without generating sql strings anyway.
@JonClements thanks! I'll keep that one in mind.
 
(you might have to tweak a few bits manually but it's generally quite good)
@holdenweb lucky you... enjoy :)
 
Later, all.
 
@roganjosh It is an index.html file
 
I mostly work with Django's ORM inside the apps I build. Sometimes this is more than I need to write a script that just has db interactions and no web stuff.
 
4:59 PM
@TanishSarmah very bottom right of the editor, there is a banner. Does it say "HTML" there?
 
@roganjosh Yup
 
Then as I said, you need to download the Jinja2 plugin. Then click on the "HTML" to "set the file type" and set it to Jinja HMTL
Sorry, "Select language mode" is the actual name of the option
 
@roganjosh That works! Thank you
 
Just remember to do that if you start mixing unquoted Jinja2 into JS files (sometimes you need to do this for plots). It will break your file repeatedly if you don't, by trying to expand the Jinja out into dictionaries
 
@roganjosh Will keep that in my mind :)
 
5:05 PM
It's less nefarious in HTML at least because it will just highlight the syntax and not explode everything in order to "fix" it :P
@Kwsswart where did you get up to?
 
@roganjosh so far i figured my problem with regards to sending the ajax first and then checking submitting the total form after should be solved by using the jquery when() method currently researching how to assign allthe ajax requests to this when method as to the problem i originally had with serializing I am still trying to figure out what is the reason why i got that error with serialising but not by doing what I have done there
 
This is the function I sent you towards the end of last week. Is that how you were serializing the outer form?
 
dpaste.com/428TPJ4SR essentially that is what i had this morning
and that when using the wtforms validators were returning invalid option on the selectfield until i changed it to the manual way
will do
 
5:20 PM
NM, I missed an important line in that :)
Yeah, ok, so less derailed but maybe mixing two approaches. Still attach the listener I gave to the form. In the latest code you sent, you haven't actually attached the function to the form submission and prevented the default. In the code you sent earlier today, you had prevented the default on the form, but then manually unpacked all the params into JSON
 
@roganjosh as to attaching that with the line context: form, it is returning a js error uncaught ReferenceError: form is not defined as to this I have this in this part dpaste.com/4UEQREXW9
 
var form = $(this); is in my example
 
Umm... curious meta post
 
yeah just realised and implemented and now working well ok serializing working now i was missing the context line which i think made the difference
now to figure out how to assign multiple ajax called to a $.when() method to ensure it submits the second point
once I am done with this feature I am going to go through all those other ajax calls and change them around to match the serialize() method
 
5:56 PM
Alright, so since my python-related question was closed for needing, and I quote "details or clarity", I have decided to ask my question here instead since the solution is almost definitely trivial in nature.
I have a 'setup.py' file that is not including an automatically-generated python file, and I don't know why.
I will provide any details as necessary.
 
My condolences. stackoverflow.com/questions/64638474/… looks pretty detailed to me. Clarity I can't comment on because I haven't a clue how to use SWIG.
 
Indeed. I'm somewhat tempted to vote to reopen it myself
 
It's basically a preprocessor that takes an input file and creates a wrapper 'C' and 'python' file, which are then used as a python module.
 
Alas, I don't know enough either. It strikes me as one of those questions where it would have been nice to have been left a comment or 2 on why it apparently needs more detail, though
 
That was my thought exactly.
 
6:08 PM
I think some users simply close any post that doesn't contain a question mark :-P
 
Meta Idea: Perhaps it should be required to give a reason upon closure of a question?
 
@Mr.MintyFresh That would be broken in so many ways. I close plenty of questions and the vast majority are self-evidently because of the close reason. If I was required to give a reason on every single vote a) I'd just stop altogether and b) I'd probably get more revenge downvotes than I already do.
 
I think that's been proposed before, and sentiment among Meta StackOverflow users was "nah, we like it the way it is now". Never mind that question answerers are overrepresented among the MetaSO userbase compared to question askers...
 
I don't want the system to compel me to have to give a comment, but generally I will if it's clear that the OP put time/effort in but missed the mark somewhere (and someone else hasn't already given pointers). I can't speak for everyone on that front, though
 
What if they had a system to highlight the regions with issues within the question?
 
6:11 PM
Then half the questions would just be a ctrl + A job :P
 
I just can't win.
I don't like to consider this, but I might have to manually create a build system for my python package (shudders).
I have created build systems for some other personal projects before, and it is simply a headache involving makefiles.
 
Have you checked the return code of os.system("swig -python ./swig_example/example.swig") to verify that it succeeded?
 
Interestingly, I don't need to since I can see that it succeeded by creating the files.
It's a 'setuptools' problem or something, I think.
 
hmm
On one hand, 99% of the time when one thinks "maybe the third-party library is at fault", it turns out not to be. On the other hand, python packaging software tends to be a hundred times jankier than the average third party library.
 
Given that, I will consider filing a bug report, after having had some lunch. Thanks for the idea.
Aside: Don't you pine for the day that men were men and wrote their own makefiles?
 
6:27 PM
I got into Python so I would never have to compile anything ¯\_(ツ)_/¯
 
\o
good day all
 
yo
 
oh hey cap'n... how you been?
 
whats the usp of python
 
@JonClements
HEY!! I'm not too bad all things considered! :) how about you?
 
6:28 PM
@sachinverma it can swallow alligators whole...
 
I have a questions about multiprocessing/asyncio. I have some tasks that need to be done. They are independent but there are a lot of them. They involve using a tensorflow model to make a prediction.
Each process has its own data and a NN will process said data. Keras/TF is not multi-process safe and doesnt really work well for this. I am curious, could I just start a process with my neural network initialized in the process, then pass the data from each process to this neural network prediction process? what tool makes sense for this?
 
wow thats what you really need to do with computers
 
@idjaw the same all things considered... my appreciation of 2020 just keeps on growing! :p
 
@JonClements We definitely are going to have stories to tell around the campfire
 
@idjaw "The Last Of Us" style? :p
 
6:33 PM
@JonClements I'll bring the guitar. You bring the horses
 
Oh fine... you take the easy job why don't ya :p
 
I have decided that the year is ending in two days, to be followed by 2021 v0.1 alpha
Looking forward to January -57th, we're really going to turn things around
 
@JonClements You're a resourceful pup. No one will suspect the pupper stealing two horses.
 
oh... you only want two horses? I was going to go for an entire herd but okay...
 
I was gonna say "I'm pretty sure that's not the correct collective noun" but apparently it is (if they're wild horses)
Also apparently, you can call them a "mob"... I think I prefer this one :)
 
6:41 PM
@Kevin I don't know anything about TF or Keras. But, when you say it is not multiprocess safe. What do you mean exactly? Thinking of it naively, if each process worked with a single thread where would the danger be? Or does each process fork multiple threads?
@JonClements You make a good point. We're going to need more people for this job
 
@roganjosh sadly... I did look up collective nouns for horses before posting... :p
 
@Kevin I do not know much about your situation, but would this help?: stackoverflow.com/a/42506478/6692650
We should call a large group of ponies a 'Rainbow'.
 
@JonClements Well, we're in it together :P I can't believe it's taken this long for me to wonder what it's actually called!
 
The collective noun for crows has to be one of the best ones though :p
 
"A pandemic of corvids" rejected for PR reasons
 
6:51 PM
that's pretty funny
 
@idjaw It just means that the model has to be loaded for every process and can't be passed in memory
@Mr.MintyFresh and @idjaw: Here is my question: stackoverflow.com/questions/64651506/…
 
Using 'asyncio' is, TMK, best suited for performing long-running operations in the background that are dependent upon something else, so I would think you should choose something else. If the NN is the bottleneck, then if you don't make more NN's, then everything will basically have the same running time as if you'd just used a Queue (or slightly worse due to overhead).
 
Right. I'm not sure what the entire architectural layout is for that solution. But, if data between those nodes is transferred over network, then you definitely would want to leverage something that will be able to manage the processes for you as requests come in (like gunicorn for example). Depending on the scale of this would dictate what your vertical and horizontal scaling would need to be like.
 
I think this sounds like a standard producer/consumer model, but am not 100% sure.
 
7:06 PM
@JonClements I was just reading this earlier today. Wild!
 
Not quite sure what it says about the quality of the engineering of the track and barrier itself though... but wow... certainly a story to tell your mates :)
 
If the neural network instances don't affect each other, then you might be able to spawn a few processes, each with its own NN, and increase your throughput that way (though there will definitely be overhead associated with IPC).
 
@Mr.MintyFresh yeah possibly. I've never used anything close to TF/Keras. So, outside of generalized solutions for what seems like that problem, I wouldn't be able to go lower to suggest a confident solution. But, if the problem really is a matter of "simply" dealing with parallelism then along with your answer there are a plethora of solutions out there to look for that will provide a solution for this.
@JonClements Glad it had a "happy" ending. But, also reminds me of how some engineering practices can sometimes be quite scary without the right laws and regulations
With software ruling more and more, it really gets concerning what areas of the industry don't have appropriate accountability and regulation for the automation that is built in to some of those large machines.
 
@idjaw Yes, agreed. Even the standard library has the 'multiprocessing' module for doing some of this.
 
++
 
7:16 PM
@Mr.MintyFresh yeah, this is one way to do this, load a model in each process. The thing is the model may be bound to the GPU and may be huge (GB of memory)
If I could just have each process just call the model (or I guess call a function in the main process) that will get the data, and return the result to the process that called it
Just like an API call
 
@Kevin I think you should try making a minimal example using a Queue for now, and see how long it takes to process ~10,000 of these strings. Otherwise, if that takes too long, you may want to look into something like JSON RPC (JSON Remote Procedure Call), and the tools that may be available for that. Beware, though, if you need to support any data type other than a string or simple object over JSON, JSON RPC may not be the way to go.
 
@Mr.MintyFresh Thank you. I was thinking this, just using a queue. do you know if I can call a function in my main process from my sub process?
 
Not directly in python I don't think; that's still under the category of RPC, which, AFAIK, only Erlang supports well out-of-the-box.
Frankly, I'm making custom RPC for one of my personal projects, and, though it's easy, it's also somewhat tedious.
If you need to support bi-directional RPC, then you should use 2 communication channels as well, since otherwise you might end up with interwoven, different messages between the processes.
This happened to me, but I solved it my simply using 2 pipes instead of 1 - 1 pipe for 'upload' RPC, the other for 'download' RPC (though both pipes were also bi-directional).
@Mr.MintyFresh Okay, I have found out what the problem that I am experiencing is: when I run './setup.py build', 'setuptools' is calling 'build_py' before 'build_ext', so the python files are copied before all of them have been generated, thus the wrapper file isn't properly included within the wheel package.
 
7:37 PM
When I've been stuck on something for a while, and then realize the problem was something along those lines, I always look at the silver lining that the problem wasn't something fundamental that would need me to put together hours of work.
 
Unfortunately, the most elegant way to fix this that I currently know of is by changing the order in which the 'build' command calls these two, which is something I'd have to modify 'setuptools' directly to do in order to do correctly. I will see if I can find a more elegant solution.
@idjaw I have spent at least 1 day on this problem, and these ones are the worst for me since I know that the solution is extremely simple, though I can't seem to get to it easily at all.
 
@JonClements Ok, got the mail client mollified I hope. Should I re-subscribe manually for the list, or will you fix it?
 
@Arne ahh okies... two ticks... lemme see... I'm not sure if the system just marked you as inactive or completely removed you from it...
yeah... it's just a "nomail flag" - I've removed that so it should be back to normal now...
 
@Mr.MintyFresh the worst kind of problem! I always question life with those problems
 
@JonClements thanks!
 
7:52 PM
(so hopefully you'll start seeing new pending review stuff coming through or something...)
 
If I don't see something by tomorrow I'll get back here to complain =)
 
sounds good to me :)
It's (not unsurprisingly) quieter than normal at the moment... there's one up for grabs though on python.org/jobs/review :p
 
hmm...a bit surprised there's no MFA for python.org accounts
am I being too picky?
 
what accounts can do on python.org are pretty much remarkably limited anyway...
 
very true.
 
8:08 PM
and the vast majority of things don't need a login anyway... so it's rare to bother creating an account
 
^ Make an account to brag to people who don't understand technology
 
Ahh... see you got it then @Arne :)
 
jup, which reminds me to update my name
 
06:00 - 21:0022:00 - 00:00

« first day (3670 days earlier)      last day (1271 days later) »