« first day (3635 days earlier)      last day (1296 days later) » 

12:09 AM
Good news: that file was not part of the release, so the door is still open
 
 
3 hours later…
3:25 AM
Folks, I'm using Visual Studio Code. The version of Python in the workspace is 3.6.6 .
But if I do python --version in the command line, I get 2.7.14 .
I want Python 3 so that I can run venv.
How can I point the command line to Python 3 ?
[And make it forget about Python 2 altogether, perhaps.]
 
 
2 hours later…
5:07 AM
@NickAlexeev I assume you're running on Windows? What's the terminal, cmd or PowerShell?
@NickAlexeev Basically you need to change the PATH environment variable in Terminal: Remove the path to the Python 2 installation, leaving in the path to the Python 3 installation.
@NickAlexeev Or, try invoking python3 directly
Two quick questions w.r.t. __slots__:
1. Do I need to list my dunder methods (e.g., __str__ and __hash__), and
2. Can I define __slots__ for a class that inherits from typing.NamedTuple
(I'm on Python 3.6 btw)
 
Hello People,
Not sure this is right place to ask.
Would like to learn the internals of python like compiling, keyword definitions etc
made an unsuccessful attempt reading the cython
Could you please provide a path to understand the internal working of python
Thanks
 
5:34 AM
@helloworld should've read CPython instead.
 
@pepoluan any suggestion on simple module to start with?
 
@pepoluan Yes, I'm running Windows, and the terminal is PowerShell.
@pepoluan I tried python3 --version just to see life signs. I didn't get any output. Not even an error. I'll check the PATHs.
 
6:26 AM
@pepoluan 1) No, only instance attributes need to be defined in __slots__. Dunders are class attributes. 2) Not sure, but it seems like a weird thing to do.
 
6:55 AM
Hi
What is the best way to save time into a file and read it later? I know I can store time as a specific format and read it later using string manipulation but that just doesn't seem right.
Hmm. This answer may help and save my time. stackoverflow.com/a/466376/8321664
 
Yeah, either that or just use a timestamp. It's just a float number, so no parsing required
 
Hey guys
Anybody familia with the keyword in order to skip a row within a for loop without breaking out of the loop completely. within jinja2 templating?
I have a bit of a situation where I need to run a if conditional on whether or not to display a group of data, however am using a for loop to dynamically display it and would like to stop from displaying if the value is 0 but still continue running through the other data within the loop to display that which isnt 0
 
7:33 AM
@NickAlexeev Ahh okay in Windows the Python installer does not create python3.exe. Strange, I thought it used to do that... aanyways, if you enter py -3 -V, does it indicate Python 3.x.x?
PS C:\Users\pepol> py -3 -V
Python 3.8.2
PS C:\Users\pepol> py -3.6 -V
Python 3.6.8
PS C:\Users\pepol> py -3.7 -V
Python 3.7.8
PS C:\Users\pepol>
@Aran-Fey Thanks!
 
@pepoluan yes
 
 
1 hour later…
8:46 AM
@PaulMcG <3
 
8:58 AM
@Aran-Fey and @roganjosh i edited my question and added those files you can check that out
 
 
3 hours later…
11:50 AM
hi
pk.replace('_monthly' or '_yearly','')
I want to replace _monthly and _yearly to empty value how can I do this?
 
12:09 PM
morning cabbages, folks!
 
@NIKHILCHANDRAROY duckduckgo.com/…
 
cbg. It's a bit early for morning cabbages, usually those arrive 2 hours from now. Must be daylight savings in the US
 
not yet. But also, I'm back in Canada now :)
 
12:24 PM
Hmm, does Canada start daylight savings at a different date than the US?
Looks like they use the same date of "first sunday of November"
That's like a week later than us. I could've sworn it was the other way round... I'm so confused
 
The trick is not trying to remember
 
...and work a flexible job where it doesn't matter if you show up an hour late once a year? :D
 
> The German Empire and Austria-Hungary organized the first nationwide implementation starting on April 30, 1916.
Huh!
 
12:39 PM
Oh no, it's our fault :/
 
Hehe
 
12:55 PM
IIRC Ottawa/Toronto and NYC have the same daylight savings dates. Thankfully, the only time it's ever mattered to me materially is when I was taking classes and "time" had "meaning". Now, aside from the meetings once in a while (which everyone's late to anyways)... Except my girlfriend - she's a school teacher and time means way more in her world than mine
 
user11006952
I don't see this event starred in this chat but I'm thinking “Ask a Core Dev Anything—You Can Even Ask Guido” might be of interest to the people here. discuss.python.org/t/… // app.sli.do/event/d4ifvw2o/live/questions
 
@Pax thanks, I don't remember seeing that
 
user11006952
@AndrasDeak You're welcome!
 
2:42 PM
dpaste.org/KTDB how to get bottom dataframe from top one?
I want to perform one hot encoding but data is in this form.
 
if you have more than one 1 in a row that's not "one" hot
 
From the link, a person asks "Why Python keep investing in new grammar features and flexibility instead of focusing on performance, mobile platforms or browser?" it got the most likes, so it is likely to get answered?
 
this is a k-hot encoding (which is an extension of the one-hot encoding). You'll need to pivot your dataframe to produce the second dataframe
 
ok, I get your point "one" hot encoding. It's a multiclassification task where I would be forcing the model to give me a 3x1 vector
k hot encoding, right thanks for the right terminologies
 
Time to search through the transcript for the many many times I've said "maybe the devs implemented X in seemingly unusual fashion Y for purpose Z" and compose them into a syntactic curiosities mega-question
"Dear Core Devs: please explain [gestures broadly] all of this"
 
2:53 PM
p = df.pivot(index='name', columns='category', values='category')
p.fillna(0, inplace=True)
for col in p.columns: p[col] = [int(bool(r)) for r in p[col]]
 
I'd put that loop body on a new line
And int(bool(r))?
Does that convert 0 and nonzero integers to 0 and 1?
Does p[p != 0] = 1 work by any chance?
>>> df
   a  b
0  2  0
1  0  5
2  4  0
>>> df[df != 0] = 1
>>> df
   a  b
0  1  0
1  0  1
2  1  0
unless I've misread something
 
thanks @inspectorG4dget and @AndrasDeak . Thanks for the solutions
 
Thanks @AndrasDeak. Today's coffee seems defective :)
 
Coffee, great idea!
 
3:07 PM
ya I too don't understand pivot
still trying to get what's it doing
meanwhile I got another strategy
to do a reverse explode and use map
 
I think I get pivot() well enough to understand the output here, but not well enough to explain it to my grandmother
 
I can explain pivot to you fine folks, but not to my grandma
pivot says "I'm going to create a new dataframe. This new dataframe will be indexed by whatever the index parameter is. This new dataframe will also have multiple columns. The names of these columns are the values found in the column identified by columns <to be continued>"
recap: we've set an index (the values of the column identified by the index parameter); we've created new columns (the values of the column identified by the columns parameter)
 
there are 3 parts, the index, columns and values. Index is immaterial for me
 
pivot goes on to say "In each of the new columns, the value I will put in there comes from the column identified by the values parameter"
recap: We create a new dataframe with a new index and new columns. The values in those new columns come from the column in the original dataframe identified by the parameter values
all makes sense?
 
it does
 
3:20 PM
Hmm, but we don't particularly want those values specifically, we just need something to distinguish them from the NaNs. hence the df[df != 0] = 1 afterwards?
 
its becoz of values
values range from 1, 2, etc
so make them all 1
 
@Kevin the nans are first turned to zeros with fillna
 
I'm a little surprised that pivot doesn't have a way to say "don't use a column for values, just use this one static value"
How hard would it be to tack a dummy column filled with ones to the right edge of the original df, and use that for values?
 
there's something like pandas.get_dummies that can create one-hot encoded stuff, perhaps that or something else can be used for this that knows to use just 0 and 1
 
My guess is "not difficult, but still more work than df[df != 0] = 1, so why bother"
 
3:29 PM
very true, but pivot doesn't have a way to fill NaNs. Someone with much more pandas-fu than I, can probably do this with some magic that I don't know. But I typically do df.pivot(index='name', columns='category', values='category').fillna(value=0) and then what Andras suggested (df[df!=0] = 1). That last part handles >1 values, but /might/ (please verify/confirm) catch NaNs if .fillna wasn't done
> How hard would it be to tack a dummy column filled with ones to the right edge of the original df, and use that for values?
In [17]: df['dummy'] = 1

In [18]: df
Out[18]:
     name  category  dummy
0  entry1         1      1
1  entry1         2      1
2  entry2         1      1
3  entry3         2      1
4  entry4         1      1
 
Hmm, exactly as many characters as df[df != 0] = 1. But I feel reluctant to mutate the input if I don't have to.
All other things being equal, choose the approach with no side-effects
 
Obvious Guy: adding a dummy column can be undone, but replacing values with df[df != 0] = 1 can't
 
use map
 
@inspectorG4dget The former mutates the input, the latter mutates the output
 
I have a clear mental image of a programming book containing that tip in a box, pointed to by a cartoon owl wearing glasses
 
3:35 PM
@AndrasDeak not sure I follow. df['dummy'] = 1 and df[df != 0] = 1 both mutate df. How is one the output and another the input? Or did I misunderstand and you meant that df[df != 0] = 1 could be done on the post-pivot output?
 
the latter, yes
Kevin doesn't want to mutate [the input]. So he prefers the other version [that mutates the output]
 
understood. Thanks for the clarification
 
Ostensibly df['dummy'] is a more reversible mutation, but the other path only touches what you need as an end result anyway
I'd be OK with either. I'm not a pandas user, but I would also avoid creating auxiliary columns if possible
 
correct. Understood. Agreed
 
Oops, I confused the issue somewhat by using the name df for both input and output objects
 
3:37 PM
Yes, it's Kevin's fault
 
[I am pelted with fruit from the disapproving audience]
 
But is it grapes or coconuts?
 
In other news, I've spent more time than I'd like this morning, correcting someone I thought was smart about COVID-19 being a hoax. It ended with me having to report some fake news FB posts. Someone please restore my faith in humanity
 
My 6 month old nephew recognized my face for the first time, rather than giving me the "who's this jackass?" look, the other day
 
@Kevin awww :)
 
3:43 PM
@Kevin awww :)
I think this qualifies for a "pythawwwn"
 
The child flourishes in these unprecedented times, never having known anything else, and I feel some hope for the future
@AndrasDeak Tomatoes, mostly. Eventually a fight breaks out about whether tomatoes are fruits, and the crowd turns on itself, sparing me any further pelting.
Most of the audience is aware that it's botanically a fruit, culinarily a vegetable, and legally a vegetable for tariff purposes in the US. The point of contention is whether the botanical classification trumps the others because it most closely reflects underlying objective truth, or whether culinary+legal win by simple majority
 
lawyer: legal
jury: culinary
scientist: botanical
 
A few voices ask "what about legal classification in other countries?" but they are drowned out by the louder voices of Americans, as usual
 
meanwhile, Trumpeteers: I will sue you for infringing on my liberties by putting this miniature dodgeball on the food guide and forcing me to eat it for "nutritional value", when I know you put a microchip in it
 
Fake news, no American has ever heeded nutritional advice of any sort
The only microchips we're eating are the little bits at the bottom of the bag after we've eaten the regular sized chips
 
4:03 PM
hahhaa! I might use that one for the next time
 
@inspectorG4dget the pivot gives you a multiindex dataframe. How to convert it to a simple dataframe?
 
what is a simple dataframe?
 
not multindexed
 
the code I showed you produces a single-index dataframe, so that'd be how I'd do it
 
would it be ok to share an image?
 
4:18 PM
no
if you have code or text, post it as code or text
I'm sure you can even export your dataframe to a format that can be copy-pasted into a REPL for us to obtain the same small example dataframe
 
the formatting is not preserved
 
Yes, because you're doing it wrong
use a code paste service to be safe
 
I don't know how to handle it further
 
4:21 PM
I presume you've googled the dozens of "flatten multiindex" questions
 
I checked out this post
it doesnt
 
And?
not sure that covers your use case, though
 
I actually 'm not used to pandas, especially the multindex part
 
that means you have to learn
I can see a very multiindex-specific answer on that thread, so if you really have a multiindex that needs flattening it will probably help
 
it does mean this, but I all of a suden get some work, where I try to use pandas. I can only be good if I do it more frequently
 
4:26 PM
That's how things are.
 
I learn two new pandas facts per year and I have faith I'll be good eventually
 
That's twice as often as you use Flask. I didn't know you had such a soft spot for pandas
 
pandas is essentially a library for manipulating rectangles, the best shape
(and N-dimensional rectanguloids, which are also good)
 
dpaste.org/uSSR something miraculous here. It didn't give error. The img names didn't get included in the applymap.
 
neither of your codepastes looks like a multi-index to me
 
4:38 PM
as I said I have nil experience with multindex. You must be true
 
lol
 
but I saw two headers, 2 headers := multindex?
 
please don't take this as a personal attack. You and I seem to be off-sync on terminology. So I want to ensure I understand you correctly so that I can give you the solutions you need
 
sure, I 'm assuming nothing like that
 
that's a good thought process, but false in this case. The second header shows you the index. The first one is the name of the columns
hang on. Lemme check to make sure I'm not wrong
yup, I just checked. I was right about that
 
4:43 PM
oh, and actually that's what I set as index in pivot in the first place.Hmm, the layout change is decisive.
thanks again
 
5:30 PM
@Kevin that's some progress
 
6:01 PM
why won't pathlib.Path accept int operands for /?! </rant>
but if anyone has an actual answer to this...
 
Would /2 append "2" or divide the path by two? :P
 
Try asking a Core Dev, the link is in the sidebar -->
Allow 3-4 weeks for delivery
 
I don't think path division has a semantic definition for that, does it?
@Kevin yammin yassss! thanks
 
@inspectorG4dget perhaps the devs just decided that we're not javascript ;)
 
I unironically want 90% of the Q&A to be justifications for minuscule design choices
 
6:04 PM
see also "asdf" + 3
 
I was hoping that Path('asdf') / 3 would result in asdf/3 (i.e. str-coerce all operands). I can see an argument being made especially for ints that it might conflate an i-node number and that would be problematic. OTOH 'asdf' + 3 I think is more ambiguous because int and str addition are defined, and the interpreter has no way of knowing which addition is required here
 
It's not just the interpreter. It's inherently nonsense to add a string to a number.
 
of course, a response to that may be "str-coerce the 3 like JS does", which might bring us back to don't be like JS in pathlib
 
The boring answer to "why" is because _parseargs explicitly checks that the components are strings, and throws a TypeError otherwise
 
then you end up with a slippery slope that anything with an __str__ could be used with Path's truediv
 
6:08 PM
It actually does coerce objects to strings, but only if they're a subclass of a string
 
Why integers and not floats? Why integers and not decimals? Why integers and not module objects?
@Kevin Barbara Liskov probably agrees
 
I honestly don't see why that's bad. The semantic purpose of pathlib is to treat div's operands as dir/file names. So why not call everything's .__str__? What bad place does that slippery slope lead to that I don't see?
 
Silly bugs where you used the wrong variable and got silent garbage?
 
this reminds me of how in perl you can do path = "foo1.txt"; path++; print(path) and get "foo2.txt"
 
grumble grumble... I was hoping this wouldn't lead to "explicit is better than implicit". Yeah, you're right
@Kevin wat?!
 
6:11 PM
Pretty much an unrelated post, but I recently came across great expectations that looks like it could be a neat package for Data Science etc. I'm just closing tabs so thought I'd give it a shout out because I don't think I've encountered any reference to it before
 
(disclaimer: I half remember this from twitter so it might have incorrect syntax and/or be a joke that went over my head)
 
@inspectorG4dget also refusing the temptation to guess
 
it's gotten to the point where I need neuralink to be my keyboard
 
Has someone experiences with SMTP?
 
6:24 PM
@Myzel394 Hello :) It would be better to just ask your question (in accordance with the room rules) and people will help if they feel able
 
@roganjosh Oh I'm sorry.
 
No need to be sorry :)
My reference to the rules is more along the lines of posting bigger code blocks to some external source and linking here v.s. posting a wall of code
 
Hi, I'm trying to connect to a SMTP server (I'm using https://www.smtpbucket.com/ for development), but the server always refuses.
Here's the error: smtplib.SMTPSenderRefused: (500, b'Syntax error', 'sender@mail.com')
Quoting "sender@mail.com" or changing it to "mail.com" or "" doesn't work
 
Does the address contain any special characters?
 
No it doesn't
@Kevin Would you like to see the code?
 
6:30 PM
Yeah :-)
 
I can confirm that this code gives smtplib.SMTPSenderRefused
 
Please can you also give a post of the traceback?
 
I also tried it out using a local smtp server (FakeSMTP) and it works fine there
 
(after adding import smtplib)
 
6:33 PM
Maybe theres a problem with the server?
 
Allow me...
Traceback (most recent call last):
  File "C:\Users\Kevin\Desktop\test.py", line 10, in <module>
    server.sendmail(sender_email, receiver_email, message)
  File "C:\Programming\Python 3.8\lib\smtplib.py", line 871, in sendmail
    raise SMTPSenderRefused(code, resp, from_addr)
smtplib.SMTPSenderRefused: (500, b'Syntax error', 'sender@mail.com')
I suspect smtpbucket has higher standards for the format of the message, compared to FakeSMTP
 
According to the docs, the 2nd argument to sendmail should be a list of addresses
 
@Aran-Fey Where did you found the docs?
 
I think it can be a list or a string
 
6:36 PM
hmm.. tried it with a list, still refuses
 
@Kevin oh right, I missed that part
 
I think I'll just use my local FakeSMTP, maybe smtpbucket is dead, the project seems to not be maintained anymore
 
Am I being silly? Is that pastebin an MCVE if you do import smtplib? I get ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it
 
Hmm, I think I just crashed my router.
 
@Kevin respect
 
6:43 PM
@roganjosh it's an mcve for me.
 
How strange, I wonder what's happening here?
 
Before I killed my internet I was going to say that the message probably needs at least a subject and a body, and possibly several other headers such as "to" and "from"
One observation: from, to and subject fields, for example, must be at the VERY BEGINNING of the variable "message", for example, or else, the fields will not be interpreted as it must be expected. I had the experience with just inserting "Subject" field, not at the beginning of the variable, and the message came to the receiver's mailbox with no subject. Cheers. — ivanleoncz Jan 17 '17 at 23:04
The answer here shows an example. See also the higher rated answers that generate the message string programmatically, so you don't have to futz with finicky formatting
I'm going to be annoyed if my router has some kind of spam prevention system that triggered when I sent an smtp packet with a bogus "from" header
 
7:19 PM
is there an async zip you'd recommend?
 
 
2 hours later…
9:23 PM
Like I was discussing I need a k hot encoding, which I cannot get with this function.
But if I load my dataset even with an iterator it's fill up 9 gb RAM and free plan doesn't thus give me the liberty to train on very basic models. I want to know how can I use the k hot encoding or how can I save space. What is so special about this function that it's not consuming lot of space? I barely see any increase in the RAM using this functio.
 
Do you see barely any RAM increase or do you see 9 GB RAM being consumed?
 
9:55 PM
when I use (plt.imread(i) for i in df.index) the bar goes to orange and it shows 9gb consumed.
 
How many and how large images are we talking about?
 
well (256, 1200) 7000 images
kaggle metal steel defects dataset
 
seems like about 2 GB of data, assuming uint8 images
So the rest of the 9 GB probably comes from something else
 
images are compressed when they are in png. When loaded in numpy they get decompressed. Am I right?
 
9:58 PM
yes ._.
 
I mean, png can be compressed, yes. But what matters is that once you load it you have 256x1200 bytes for each image
times 3 or 4 if it's coloured, but that's probably not the case
 
nope only 1 channel(grayscale)
 
you can load a single image with plt.imread and check .nbytes on the resulting numpy array
I expect it to be around 307200
 
1228800
 
and overhead from decompressing should be file by file
@VisheshMangla OK, so 4 bytes. Probably float32. You can check the .dtype while you're at it.
that means 8 GB of raw image data
 
10:03 PM
uint8
 
Then you're wrong somewhere. Check .shape.
>>> np.empty((256, 1200), dtype='uint8').nbytes
307200
 
(256, 1600, 3) oh God , 3!!
 
>>> np.empty((256, 1200, 3), dtype='uint8').nbytes
921600
I bet some of them are RGBA (4 rather than 3)
 
ok , so cmap="gray" can help me probably
 
you just have to give up your assumptions and start validating your inputs
 
10:05 PM
is that related to writing tests?
 
Not necessarily, but potentially
actually, no, not really
this needs to happen on your real data
before you do anything with that batch of images you'll probably want to ensure they're all the same type and have the same number of channels (whether you need the exact same size probably depends on what happens down the line)
 
I thought because it's from kaggle it w'd be clean, but as you said it's better to validate
thanks, I can use bunch of asserts to see if there is any violation in expected vs actual.
 
No, don't use assertions to validate data. Assertions are closer to tests, they ensure your code works as designed. Only assert "this should never happen" cases.
check your data and raise or fix if something's wrong
 
counter intuitive but ok I ll follow that
 
Perhaps your intuition regarding assertions is wrong
For instance, assertions can be disabled with a switch for production code, that's kind of the point for them. You wouldn't want validation code to be disabled in production.
 
10:13 PM
Assertions can't be disabled once written because they are mixed with the code?How w'd you disable them?
 
magic I guess
 
??
tests can be disabled in production ready code afaik, because they like in a separate folder
 
The easiest way to disable tests in production code is not to run them.
 
asserts , they are the old design by contract thingy
@AndrasDeak yes
 

« first day (3635 days earlier)      last day (1296 days later) »