« first day (3904 days earlier)      last day (1027 days later) » 

4:38 AM
not MCVE: fails to say what pandas version, doesnt show entire pd.read_csv line stackoverflow.com/questions/68109609/…
 
4:56 AM
@CoolCloud just as fyi, linux paths that start with a forward slash are full paths in linux contexts. relative paths start directly without a slash.
 
 
1 hour later…
6:08 AM
question: potentially overengineered - if i wanted to have a config for the column name literals i'd use in my dataframes, what is a good way to store it? One option was a bunch of variables such as country_col, value_col but i've done this in the past, and was wondering if there was something better. So i started exploring enums, turns out you have to write myenum.mycol.value to actually make it use the value. So, is there some alternative where i don't have to write the .value portion?
And i could use a sanity check if there's a better way to do all this
 
6:19 AM
@ParitoshSingh use a dict is my best bet ex: collumns["country"].value . beautiful pict by the way
 
6:32 AM
columns["country"]= value
 
7:17 AM
@ParitoshSingh What I the config for? I mean, in terms of a whole package, something that gets set at runtime by your own code etc?
 
nah, it's nothing more than just someone on my team wanting 1 place to define the strings. they're the column names for my internal dataframes, and if i was just coding for myself i'd have just written them out.
 
@CoolCloud this discussion appears to only have generated new problems by losing the file they had. The error appears to suggest that they are having issues parsing the file; they already had the file. More likely is that they didn't set the correct delimiter
 
so nothing more than the requirement to turn code that looks like df['country'] into df[variable_defined_somewhere]
 
An Enum might do it, but so would a dict, I guess
Hmm on review of the CSV question, perhaps I'm in the wrong there since it was apparently fixed. I'm not used to that type of error due to file paths but rather file contents.
 
7:37 AM
cbg, is there any library I can make use of to find the centroid based on a list of latitude longitude tuples? I have looked into some SO answers and most are custom solutions like stackoverflow.com/questions/6671183/…
 
nvm sorry, I misunderstood the problem. I would go for some dedicated gis software package
 
@python_user centroid is easy. Geo is difficult.
 
I will have to check how centroid differs from Geo first to ask anything further
 
Well, centroid is easy if you're OK with leaving the surface
@python_user two different things. I'm just compressing on mobile.
 
sure, I guess I am ok with leaving the surface, I will also have to find the area of this
 
7:45 AM
Much more challenging
 
one way I thought was to use python google maps api and see if it has some methods for this
 
Once you have Cartesian coordinates, centroid is easy. Getting coords from lat/lon is hard on a real planet.
And yes, area is a lot harder.
 
ok, at least now I know the scope of this problem
 
It's possible that Google Earth type plugins will allow you to get the area of geofences in the odd case (before you have to start paying)
 
I will look into google earth api / plugins as well, thanks
@roganjosh thanks again for pointing me towards that, earthdatascience.org/tutorials/… looks helpful I can change this to python
 
7:50 AM
I think you'll want a GIS system, ultimately. This might be with geopandas or arcGIS or something else
 
@roganjosh yeah, perhaps one reason why i am hesitant to use a dict is that even though it would solve my problem, it's still a bunch of literal strings flying around in the code, though sure this time they're just the dict keys
im almost tempted to just write a class with some variables for this. (gasp)
 
Perhaps a types.SimpleNamespace would work for you?
 
that might be the best compromise, thanks
or i should just suck it up and write the dict
 
I vote for a bunch of COL_FOO constants
 
Pro tip: COL_FOO1 = {"name": "Country"}
 
8:04 AM
haha
yeah, i was probably just overengineering this all along. i'll go for the constants instead, thanks all
 
8:47 AM
I've been messing around in Sage, experimenting with various ways of drawing circles on a globe. I'll try to post a demo in the next message. Sorry the code's a bit cryptic, I condensed it & stripped the comments so it'd fit into a chat message.
 
omg, it was a sign error. Finally, woah man this feels so good :)
 
that looks very cool
 
Thanks!
 
@PM2Ring you could collect full links in a github gist :)
 
8:53 AM
sage must be python inspired, that code looks like almost python to me
 
@ParitoshSingh it is
python with bells and whistles
Jun 9 at 19:16, by PM 2Ring
SageMath can do 2D plotting (using matplotlib) and 3D plotting (using three.js). Here are a couple of examples. https://math.stackexchange.com/a/3991940/207316 https://puzzling.stackexchange.com/a/109191/36040 https://astronomy.stackexchange.com/a/28036/16685
 
@AndrasDeak I was thinking about doing something like that. But then people'd have to click on the gist link, then click on the SageMathCell link. This way's quicker. :)
 
ah i see, looks interesting. dang i wish sage could simply be used as a python package, apparently it's different enough under the wraps to merit being separate.
 
@ParitoshSingh I'm only using a couple of Sage tricks in that script, so it's fairly close to plain Python. But all the variables are actually Sage types, not native Python types. The main Sage feature in that code is the use of a few symbolic variables, x, y, & t.
Sage is huge. It contains so much stuff that it'd be scary to import it as a regular Python package. :)
And even without importing anything, your global namespace already contains about 1000 names. :(
But if you want to do algebra or calculus, it's got all sorts of goodies.
A cute thing about Sage's symbolic variables is that you can easily build complicated functions with them that can be differentiated & integrated symbolically. Sometimes, Sage can't integrate a function symbolically, but you can then fall back to integrating it numerically.
 
9:12 AM
It's basically Wolfram Mathematica in python. Or it wants to be.
 
It can simplify multivariable polynomial expressions, but it's a bit limited at simplifying trig expressions... which is annoying when you want to simplify some horrible spherical trig equation. :)
@AndrasDeak Yep. It's the poor person's Mathematica.
 
@AndrasDeak yeah, that's the first impression I got too
 
The thing about things that do everything is that they do few things really well...
but I'm not familiar with Sage
 
Doing 3D graphics in Sage inspired me to take a look at three.js, since that's what Sage uses for its 3D rendering. But Sage only uses a tiny fraction of what three.js can do. So I've had to brush up my JavaScript skills... However, writing & running JS on a phone is very frustrating, since phone browsers generally don't provide a dev console.
@AndrasDeak Sage mainly relies on lots of 3rd party components that are pretty good, like Pari, Maxima, Numpy, Matplotlib.
Here's a basic three.js thing I did a few months ago: it creates an interactive stereo pair of a simple cubic frame. Sorry, you have to download it to view it. gist.github.com/PM2Ring/6804ce81b403053e0963959580275360
I'd like to see a stereo pair version of this: mikelortega.github.io/tesseract
 
9:38 AM
@ParitoshSingh Oh I did not know that, thanks for it
 
Does that stereo pair work for you, @Andras ? You have to view it wide-eyed. I don't think three.js has a cross-eyed option, since it's intended for use with a VR viewer.
 
@roganjosh Losing the file?
 
In terms of, I thought the parsing error was from the contents of the file that they could no longer open when they changed the path. It seems I was incorrect
 
👍
 
cbg
 
9:47 AM
cbg
 
recbg
 
10:14 AM
@PM2Ring which one and how do I view it?
your gist to be opened as html?
@PM2Ring yeah, it works. I can view it cross-eyed too but it's very uncomfortable
 
@AndrasDeak It kind of works cross-eyed, but the depth is wrong.
 
10:35 AM
guys anybody know how to prevent pandas from escaping "/" characters? I have datas as 27/01/1998 and when saving to json it forms to 27\/01\/1998 I would like to have it as normal
 
pandas. and json. o.o
sounds like a bad match
but i suppose it's not as bad as i initially assumed.
can you make an mcve?
 
@Kwsswart That's apparently valid and useful JSON behaviour.
 
link for workaround
is there a reason you don't want these slashes to show up? i assume it should be parsed back just fine
 
Essentially I dont care but my boss wants these gone so that people who are "less technical" can work with it
 
10:50 AM
@Kwsswart decouple data from display. If "less technical" people have to work with it, use a to_technical() method that pretty-prints
 
11:08 AM
"The output must look nice" is always a valid project requirement
I'm trying to imagine the kind of person that is technical enough to work with json, but not technical enough to work with json that contains "\/". Seems like a narrow demographic.
Maybe the boss thinks that making the program 0.5% more accessible is a worthwhile endeavor even if he doesn't have a specific end user in mind that it would benefit
 
@Kwsswart Well, the boss is the boss. ;) But I'm a bit concerned about these "less technical" people working directly with JSON. Hopefully, they're just reading, or maybe copying stuff from it. They probably should not be manually modifying it.
 
11:53 AM
real quick question, how do I encode a string into a byte array. codepage is cp932 (shiftJis), but the string I want to encode uses english characters. My issue is that its only encoding as single bytes instead of double wides. (Need 2 byte characters)
should I encode with utf8 instead of shiftjis to get doublewides?
 
Do you have a specific example demonstrating what you don't like in the result?
Is your issue that ASCII stays ASCII? Is this not a feature of that codepage?
In [393]: 'potato'.encode('cp932')
Out[393]: b'potato'
This would also be true for UTF8 so I don't understand your situation without an MCVE.
 
ya issue is ascii stays ascii lol
program reading file, expects doublewide characters but Im using ascii/english
 
So my question is: is the program not buggy?
 
Either that or it's not using shiftjis
 
the program is buggy as all hell lol. but its outside my control
hmm
 
12:04 PM
It's reasonable that most encodings preserve ASCII because a lot of things are ASCII. I know UTF and latin1/2 do that.
 
well it did read the ascii, but then it crashed cause it expects the bytes to be divisable by 2
 
what is "the program" here? inhouse code? some library?
 
inhouse code
doesnt matter 2 much
 
what encoding does it expect?
 
Maybe the length of the text as a whole must be divisible by 2?
 
12:06 PM
now im not sure lol, don't worry about it. will look into a few things. thought it expected shiftjis
 
@Aran-Fey that would still be weird
b'potato' is OK but b'cabbage' is not?
 
gonna confirm that it works with divisble by 2 (but it looks like it should)
 
this whole divisible by 2 thing makes no sense to me.
 
yeah, you're trying to fix the code in the wrong place
 
no source for the other code heh...
 
12:08 PM
@JasonBrown How exactly do you encode the data? Note that the entire point of UTF-8 is that it is not a fixed-with format.
 
@JasonBrown sounds like this is a gift that will keep on giving.
 
Lol
well thanks for the info, I might just have the wrong code page
is there a cp932 equivalent that is fixed-width?
 
theres either cp932 or theres "not cp932". encodings arent meant to be just randomly mix and matched. however..i suppose i'll go ahead and say it: utf-16 is probably what the program expects if i had to guess.
What troubles me is, how does inhouse code get to this stage, if you don't even have source code for it, then somewhere your company has failed
 
maintainability is probably not a KPI
 
Lol confirmed it works if its divisible by 2 so il probably just pad non even strings with a space at the end
 
12:15 PM
:|
does it "work" or does it work?
 
it 'work's
 
if you're using the wrong code page then adding a single character to make the byte count even will... be interesting
I guess you can just commit and leave it for the next poor soul in your company.
 
ha. "commit", he says.
 
lol
 
Code review will catch it anyway. If it doesn't, QA will.
 
12:18 PM
;)
 
1:07 PM
morning cabbages, folks
 
cbg
 
1:52 PM
After a full day of yak shaving, I've finally managed to write ten new lines of useful code
 
2:06 PM
As they say, productivity can't be measured in lines of code. Not even in useful lines of code
 
Let's all switch to kSoY. shaves a yak
 
Silly numpy question... I have an array of shape (m, n, k, l) and I want to add an extra entry in dimension m, so an array of shape (n, k, l). Is there a convenient way to do this? The only (silly) way I can think of is turning the original array into a list of length m, adding another entry to it, and converting it back to an array
 
Yes, that does it! Thanks :)
 
2:43 PM
hey guys i have a reactjs front end that calls a flask api, (they are completely separate apps - kid of microservice architecture) I want to add authentication using flask_login but I'm not sure if I should create a separate service to do this, ot integrate the flask login with the other flask api
 
@user129412 np.concatenate(big, small[None, ...]) might also work
 
3:04 PM
I have identified the six tables that contain the data I want, out of a possible 12,000. I can tell because they each contain a unique string. But I still can't locate the position of the first meaningful byte after each table's seemingly pointless and randomly sized header
 
Still byting the good byte, eh?
 
Do you have time to check the distribution to check random sizes for categories?
 
@Kwsswart As Paritosh says, pandas and JSON seem to be as immiscible as it gets. I have had nothing but pain on even basic operations; abort. Find any interim solution you can that doesn't rely on its JSON handling directly
 
Hmmm will take a look into another way :/
 
wait, are you still stuck with this?
 
3:19 PM
Paritosh I have sent it as a csv file for them to work with
 
nice :P that works
 
@erotavlas If it helps I looked into this quite a bit and found with react frontend its usually better to work with flask_jwt if you take a look github.com/Kwsswart/flask-react-phonebook/blob/master/backend/… you can see how I ended up working around it as with calling it from the react components in the frontend
 
3:33 PM
Ooh, I might have identified the first 0x2c bytes of the header as pointer(s) to an object all the tables have in common
My theory is that the table holds the serialized form of arbitrary objects, and the serializer has two modes. The simple one smashes the object flat and adds a couple explanatory bytes so the other end can unsmash it. The complex mode uses a maze of scary pointers, all obtuse. This is only necessary if the object has a cyclic reference.
 
You're not talking about pickle, are yee?
 
All of the objects I care about have one cyclic reference. So if I can leap over the scary pointer maze, then I should land at my desired destination at the beginning of the simply smashed data.
@MisterMiyagi No, but I'm guessing the protocols are similar, since they have very similar goals
 
I'd say implementing two different serialization formats is a strange idea, but I've seen people do stranger things, so...
 
It's similar to NRBF. A "toplevel record" (my words, not theirs) can effectively contain an unlimited number of nested records, as long as this doesn't cause an object to get serialized twice. If that would happen, it puts a placeholder "weakref" down, and starts a new top level record for the troublesome object
I think you're also allowed to make weakrefs if you just feel like it. The spec isn't demanding about that.
I really hope Unity isn't making weakrefs whenever it feels like
 
Can I interest you in some nice bridges while you're hoping for Unity?
 
3:49 PM
I hope that my hope is not hopeless, because IMO it would be harder to write a "whimsical" serializer that sticks weakrefs where they aren't needed
 
Right. That'd be, like, someone using threads on every call. Just to break the recursion depth.
 
The odds are in my favor -- such a deranged mind is rare enough that I doubt two of us would ever cross paths.
 
Depends on the chances for a rooms/6 convention. :P
Oh well, I go back to hugging servers instead of making snide comments. Carry on, Kevin.
 
@Kwsswart thanks, so the front end app can remain separate )i.e. I don't have to serve react using flask) using JWT?
 
4:50 PM
I don't think I understand that comment @erotavlas. What are you replying to?
 
@roganjosh it was regarding the example app @Kwsswart posted in the comments further up
 
That's pandas parsing of JSON?
 
1 hour ago, by Kwsswart
@erotavlas If it helps I looked into this quite a bit and found with react frontend its usually better to work with flask_jwt if you take a look https://github.com/Kwsswart/flask-react-phonebook/blob/master/backend/app/auth/routes.py you can see how I ended up working around it as with calling it from the react components in the frontend
This one I expect
 
yeah
 
At least I assume the topics are connected, as I don't understand a word of flask
 
4:53 PM
it was about authentication in the react front end if its calling a flask api backend
 
In which case, yeah, you can API-ify the Flask backend as they did there
 
I hope this wasn't supposed to be a socratic dialog demonstrating the value of direct replies over general pings, because I skipped over the fun part to get to the conclusion
 
so I can keep the react app as an independent nodejs app?
I mean I don't have to serve it using flask?
 
@Kevin it was and it wasn't. I'm a busy man, I'll take either outcome
 
A fine approach
 
4:55 PM
@erotavlas flask isn't a server. I think what you're really asking is whether Flask should be rendering the templates for you vs. React?
 
yeah probably thats what I meant
 
What kwsswart has done is just made the backend return JSON for your frontend to work with as it pleases. Flask doesn't mandate that you also have to have it render templates for you. Indeed, it'll become a horrible mess because there is crossover between jinja2 and react syntax
jinja2 is what flask invokes when you use render_template. That isn't flask itself
 
ok I don't want to go that route anyway, I want to keep flask api clean - I just want it to be an api returning json data
 
You've lost me. What is "that route"?
In terms of rendering the template for you? Then sure, just use jsonify and only return valid JSON
 
the route with jinja i believe :P
 
5:00 PM
the route (aka direction) you were discussing - using flask to render front end templates
 
5:20 PM
i heard rust is more efficient than python, apparently someone I know rewrote a python script that took 12 seconds to run, and it took only 1 second to run in rust
 
In terms of run time, it almost certainly is
So is C, C++, Java, Julia, .... What do you take from that anecdotal data?
 
All hail the borrow checker?
The real selling point is that Rust's compiler is less of a pain than MyPy. :P
 
not much, just curiosity to maybe try Rust, i never really paid much attention to it before
 
I, for one, do not hail my new overlords.
 
the Rust overloards?
 
5:33 PM
@erotavlas what about Kotlin and Swift? They do app dev too. The point I'm getting at is that I could just keep going with languages that are faster than Python at run time
 
assembly or bust
 
Heck, even PyPy (python in python) can beat python
 
FPGAs are pretty rad, or so I heard...
 
<courtroom drama> I rest my case, Your Honour <dramatic walk back to my chair>
 
I'll go get the pitchtorches...
 
5:37 PM
Oh, cruel fate that it should end like this. Hounded out of my safe space
 
hands roganjosh a blankey here, take this. it will keep you safe.
 
I just relied on my comfort Cicada but maybe a blankey is a better option
 
What's a cicada with capital C?
 
It's one that I cloned from a wikipedia article because I can't spell
Even more personalised
 
6:09 PM
i searched for cicada online..and it's an..insect? is comfort cicada a thing?
that seems like the opposite of comfort to me
 
Yes, it's a group of insects. They have membranes on their bellies with which they can make pretty loud noises. Like crickets, just higher-pitched and higher frequency.
Also, they are really common here, but all of them are really tiny species. Most people don't even realise these are cicadas.
 
I'm told that much of New Jersey is cicada territory, and this year is the peak of their 17 year population cycle. But I've only seen exactly one this year
 
hm. i've seen crickets, i suppose i don't know if we have cicadas or not
maybe what i thought were crickets were cicadas all this while
 
Here's a map of sightings during previous cycles. I'm basically exactly on the edge of the blue region that straddles PA and NJ
 
I bet that's specific to a certain kind of giant cicada...
these guys are cicadas too
 
6:22 PM
Yeah, IIRC this species only lives in the US and Japan. So it's probably not a perfect match for yours
 
They can fly and they jump a lot. Keep banging their heads at lamps.
 
@AndrasDeak that one looks oddly pretty
 
another, claimed to be a US-originating species
The ones I see and recognise usually have these non-transparent wings.
 
hm, looks like they're in some parts here, but rather rare
 
Oops, I'm wrong, the Japanese cicada and US cicada(s) are different species
I distinctly remember seeing a map of the world that highlighted only Japan and the US east coast... Maybe they have something in common with each other that the other specieses don't have
 
7:08 PM
@ParitoshSingh How long have you known me? I would have thought your sarcasm detector was better-attuned by now!
 
not sure what y'all have against cicadas
 
haha. i'll try turning it off and on again.
i think it was because i hadn't known the word, and so i went on a journey and forgot what started it :P
 
:P
@AndrasDeak An almost perpetually screaming, swarming insert. What's not to love? I take it all back
 
@toonarmycaptain Congratulations :-)
Cbg all..!!
 
cbg :)
 
7:18 PM
cicadas resembles how we pronounce keedas (c as k) which is pretty similar
 
In before argument whether it's "see-KAY-das" or "see-KUH-das"
 
There is no argument to be had. It's the former. Let's move on.
:P
 
i dont know why, but my sarcasm meter went to 100 there :P
 
Turning it off and on again really does work sometimes!
 
:D though I would want to know what Paritosh switched on :P
 
7:24 PM
16 mins ago, by roganjosh
@ParitoshSingh How long have you known me? I would have thought your sarcasm detector was better-attuned by now!
It's not that far up the transcript
 
:D okay sorry
 
 
1 hour later…
8:35 PM
today has been a total garbage day for me. Not much productivity, sadly
 
There's always gc...
 
8:51 PM
@inspectorG4dget "toyear has been a total garbage year for me. Not much productivity, sadly." FTFY... at least from my perspective.
On second reading, that sounds terrible. Maybe I meant "Fixed that for me" since I meant that it is true for me (-:
 
9:35 PM
@piRSquared too real, man! but I gotcha
oh, no sweat! I totally got what you meant
 
9:54 PM
What is/are the 4 notifications on this? I've refreshed, and tried clicking
Now it's 5 and there is no notification in chat. What a mess :/
nm, don't even bother answering. It's some ghost notification that has no meaning. I got ahead of myself and thought that something chat-related made sense
 
 
1 hour later…
11:07 PM
Brain: "You've got tons to do, get to work!" Me: "Wonder what happened in room six today?"
 
11:57 PM
@Dodge Ah me too :/
 

« first day (3904 days earlier)      last day (1027 days later) »