« first day (3757 days earlier)      last day (1174 days later) » 

12:03 AM
Cbg cbg cbg!!! Just came up for air, after 3 weeks into the new job and seeing just how far behind I am in the tech stack at web scale! And finding good tutorials and books on these software tools is hit-or-miss, and mostly miss - currently reading a 4-1/2 star rated book on docker, and hoo-boy is it rough! Docker is something I wished I'd used at my last job though, going to create some little Flask containers running some simple Python web pages.
Would probably make for a good stream session on a Saturday.
 
Cbg all, I wanted to make a password storage inside of python, but I am not sure, how to un-hash the password to give it back to the user. Any ideas? All of the searches say that its not possible to unhash
but if I am the person hashing it, then wont I be able to use some key? to un-hash it as well? Not sure, just asking.
 
Is this a password for your app, or a password that will be used to connect to some other app or system?
 
12:19 AM
@PaulMcG Its where they store their social media password and all other kinds of passwords.
 
This is already taking a frightening turn, but...

Okay, you will need to encrypt these login credentials for storage, in such a way the they can be decrypted. Hashes are just a one-way thing, there is no such thing as unhash.
I would suggest looking at the cryptography package, which you can get from PyPI.
 
@PaulMcG But then wont someone else be able to decrypt the password? perhaps a database breach
 
If you are guarding against a database breach, you should pay for more informed advice than you will get here for free.
But to address your question, you would not store the encryption key in the same location as the encrypted credentials, for just that reason.
 
@PaulMcG Hmmmm that seems fair, I'll look into it, thanks mate.
 
1:08 AM
@AnnZen You should skim the The Stack Overflow Regular Expressions FAQ. Good article to bookmark and browse whenever you have a minute free at work, morning coffee...
@bashar What's your question on sentiment analysis?
 
@PaulMcG do you stream? where?
Also congrats on the new role. Web deployment is definitely way different than it was a few years ago for sure.
 
2:12 AM
@PaulMcG Docker is pretty awesome. I had to compile the DJI thermal image SDK at the beginning of this week. It required ubuntu 16.04, gcc 5.4.0, and cmake 3.x.x. My only option was to build it in a docker, and it actually worked!
 
2:27 AM
Hi, I have a column with multiple values per row separated by comma. I need to take count of each unique value ? How can I do that with data frame ?
 
2:47 AM
>>> df = pd.DataFrame({'a':['1,2,3', '2,2,3', '3,3,3']})
>>> df['b'] = df['a'].map(lambda x: len(set(x.split(','))))
>>> df
       a  b
0  1,2,3  3
1  2,2,3  2
2  3,3,3  1
 
3:09 AM
@Dodge a
0 1,2,3
1 4,5,3
2 1,2,4
In the above data frame, i want to take count of occurence for 1 is 2, 2 is 2, 3 is 2, 4 is 2 and 5 is 1
Also there won't be duplicate values in single row
 
3:31 AM
anyone know how to get numpy.zeros value only? without "array" notation in it?
[array([[0, 0, 0],
[0, 0, 0],
[0, 0, 0]]), array([[0, 0, 0],
[0, 0, 0],
[0, 0, 0]]), array([[0, 0, 0],
[0, 0, 0],
[0, 0, 0]]), array([[1, 1, 1],
[1, 1, 1],
[1, 1, 1]]), array([[1, 1, 1],
[1, 1, 1],
[1, 1, 1]]), array([[1, 1, 1],
[1, 1, 1],
[1, 1, 1]])]
looks pretty bad
nvm figure it out. with .tolist()
 
@YatShan
>>> from collections import Counter
>>> df = pd.DataFrame({'a':['1,2,3', '4,5,3', '1,2,4']})
>>> s = ''
>>> for l in df['a']:
...     s += l
...
>>> s = [l for l in s if l != ',']
>>> Counter(s)
Counter({'1': 2, '2': 2, '3': 2, '4': 2, '5': 1})
 
3:44 AM
Thanks @Dodge. But above splits character by character. Actual dataset of mine is made up of sentences, separated by commas.
 
@YatShan What are you counting?
 
@Dodge My original dataset is data["Q6"] = {['prepare data', 'analyse data'], ['prepare data'], ['model data', 'analyse data'], ['prepare data']}. Output should be prepare data -> 3 times, analyse data -> 2 times, model data - 1 times
@Dodge I tried data['Q6'] = data['Q6'].apply(str).str.split(",").   data['Q6'].value_counts()
 
Does your data look like the following?:
>>> df
                              a
0  [prepare data, analyse data]
1                [prepare data]
2    [model data, analyse data]
3                [prepare data]
 
Yes
 
4:06 AM
>>> df = pd.DataFrame({'a':[['prepare data', 'analyse data'], ['prepare data'], ['model data', 'analyse data'], ['prepare data']]})
>>> items = []
>>> for sentances in df['a']:
...     for words in sentances:
...         for word in words.split(' '):
...             items.append(word)
...
>>> Counter(items)
Counter({'data': 6, 'prepare': 3, 'analyse': 2, 'model': 1})
 
I don't want count of words, I want count of sentences. Counter({'prepare data': 3, 'analyse data': 2, 'model data': 1})
 
Can you see the part you need to take out to get what you want?
 
Yes
items = []
for sentences in df['a']:
    for word in sentences.split(','):
            items.append(word)
Counter(items)
Thanks @Dodge for helping out !
 
4:28 AM
@YatShan You could also do:
>>> from collections import Counter
>>> from itertools import chain
>>> df = pd.DataFrame({'a':[['prepare data', 'analyse data'], ['prepare data'], ['model data', 'analyse data'], ['prepare data']]})
>>> sentences = df['a'].to_list()
>>> sentences = list(chain(*sentences))
>>> Counter(sentences)
Counter({'prepare data': 3, 'analyse data': 2, 'model data': 1})
 
4:42 AM
Hi all, I have been trying to solve this problem and struggling to find a way to solve this. Can one of you please look into this .... stackoverflow.com/questions/65912008/…
I know some of you are experts in #Pandas and thought you may be able to solve this faster than i can
 
 
1 hour later…
6:00 AM
@alkasm In my imagination so far, twitch I think is the most popular. I've been wanting to do a live stream of some pyparsing bits, but reasons.
 
6:13 AM
you have one viewer in queue if you drop a link :D
 
Thanks, I surely will. :)
 
6:47 AM
@Dodge Thanks for suggesting an alternative implementation
 
 
1 hour later…
8:11 AM
hi
how can I make the following dataFrame?
I can generate the pattern but I don't understand how can I initiate each element
df = DataFrame(np.nan,
 index=list('0123'),
 columns=['a','*','c', "*", 'e'])
 
8:24 AM
@CătălinaSîrbu df = pd.DataFrame(np.arange(20).reshape(4,5),index=list('0123'),columns=['a','*','c', "*", 'e']) ? maybe better alternatives exists
 
thanks I will try to understand
 
Can you really have two columns with the same name?
 
it works :/ , I have no idea
just replaced the range part in their code and it worked
df['*'] gives a new df with two columns as opposed to a series like df['a'] would
 
For the record, '*' is exactly the same thing as "*". They don't become different values just because you use different quotes
 
8:45 AM
yes,
thanks
 
9:31 AM
Hi,is there a documentation for knxnet protocol ?(python based one or C++ based one)
 
10:02 AM
command = "source /home/usr/.bashrc; echo $PYTHONPATH"
subprocess.Popen(command, shell=True, cwd="/home/usr/ros2_foxy", executable="/usr/bin/bash")
Shouldn't this work? aka update the pythonpath?
Ok, finally. I need to not source the bashrc which sources ros, but just source ros directly :) This finally worked :)
 
10:34 AM
How can I display an int in non scientific notation?
 
How do you display an int in scientific notation? O.o
 
it's just int(x)
just print(x) ?
 
>>> print(3**100)
515377520732011331036461129765621272702107522001
How high do I need to go?
 
Ah I introduced the float with foo*1e9. Odd why would 1e9 be a float?
I guess it makes sense for consistency, because negative values would be float
 
11:09 AM
If the language devs were gung-ho about it, they could define the grammar so that 1e9 is an int and 1e-9 is a float and 1.e9 is a float. but as you say, consistency is worthwhile here
 
Huh Futures are not cancelable how odd
 
exponentfloat ::= pointfloat exponent | digitpart negative_exponent
integer ::=  (decinteger | bininteger | octinteger | hexinteger) [nonnegative_exponent]
negative_exponent ::= ("e" | "E") "-" digitpart
nonnegative_exponent ::= ("e" | "E") ["+"] digitpart
Something like this
 
that would a neat small addition. I wonder if somebody ever got bit by this in production
Wait what is the way to go if I want something like supervisord but inside of python?
I though it's futures, but since they can't be stopped that also doesn't seem it
 
I'd speculate that it's fairly rare for a surprise float to go unnoticed, either because it shows up in the output or crashes when you try to pass it to an int-only function. But even so, "rare" multiplied by "the number of Python programmers in the world" is probably more than one
 
11:35 AM
I'm not up-to-date on async, but I'm surprised to hear futures can't be canceled. What is Future.cancel for then? (not a rhetorical question)
Possible answers that I am wildly guessing:
- `Future` and `concurrent.futures.Future` are different classes and only one of them is cancelable.
- The `cancel` method just raises `NiceTryException`
- `cancel` executes without crashing, but instead of gracefully halting the Future's code, it ruins its reputation on twitter
Hmm, probably not #1, since docs.python.org/3/library/… is a thing. Maybe amend it to "futures are cancelable in principle, but in practice will return False (i.e. "sorry, I'm busy not canceling") at inconvenient times"
I think you can ameliorate that by yielding frequently in your coroutine
 
11:54 AM
"your future, is now cancelled" could be the tagline of some cheesy action movie.
 
Not if I make it into a documentary about whatever imminent apocalypse is currently trendiest
I hear water shortages are hot right now
 
Now I kinda want to see a movie filled with IT puns that regular people won't even notice
 
Relationship comedy where the couple callls chores "co-routines" in the hopes that the new name will make it less annoying
"Hopefully it will be better than our threading strategy" [flashback to a tremendous chore chart overlaid by crazy overlapping red string]
Fig 1. Trying to decide who cooks dinner tonight
 
12:10 PM
In reality, both of them would try to do it, and if they're lucky, one will finish before the other one starts. If they're unlucky... they'll probably need to order some food afterwards
 
12:33 PM
Can anyone tell me why the Bcc isn't working here?
 
if that is real login info you should change it
 
def SendEmail(Tos, Bccs, Subj, Body):
    import smtplib

    from email.message import EmailMessage

    msg = EmailMessage()
    msg['Subject'] = Subj
    msg['From'] = Email
    msg['To'] = Tos
    if Bccs[0] != 'None':
        msg['Bcc'] = Bccs
    msg.set_content(Body)

    with smtplib.SMTP('smtp.office365.com', 587) as smtp:
        smtp.starttls()
        smtp.ehlo()

        smtp.login(Email,Password)

        smtp.send_message(msg)

IPdb [24]: Bccs
['jamesmcintyre@example.com']
 
@Kevin beautiful
 
Inputs:
IPdb [23]: Tos
['email1@donmain.co.uk', 'email2@domain.co.uk']

IPdb [24]: Bccs
['email2@domain.co.uk']

SendEmail(Tos, Bccs, "File Password Protected", Body)
@python_user Thanks - good call
 
@JamesMcIntyre no mention :), better to be safe and change the password on your end :)
 
12:40 PM
Done
 
12:51 PM
@JamesMcIntyre Umm... msg['Bcc'] = ','.join(Bccs)
 
Dang, was just about to suggest that
If their current code is sending the email to all the "Tos" but not the "Bccs", I'm curious why EmailMessage can understand a list in one context but not the other
 
@JamesMcIntyre have made a slight edit to your message as well... it probably doesn't matter - but always best to not have live emails hanging around...
@Kevin I'm not convinced it is...
 
Hehe same
 
sane python libraries should absolutely handle lists...
 
email.message does not feel like much of a "batteries included" kind of module to me
 
12:55 PM
but it's smtp.send_message which should understand lists
 
I'd test this out myself but in order to get my localhost email server to run, I have to coat Windows firewall in tinfoil and kick it really hard
 
I'm feeling very xkcd.com/722 today
 
Due to Andras' and Jon's testimony, I'm promoting "the email isn't being sent to anyone at all, perhaps due to network problems" to my #1 suspect
 
Not that things are not working, but that they are just little lights on a screen
 
Today I feel like my code is a delicious lasagna, but one in every thousand dishes contains a rusty metal screw, and I have to detect it before it reaches the customer's table
 
1:07 PM
@Kevin oh I'm just describing the ideal world we should all live in. I've never touched smtp.
 
QA keeps complaining of chipped teeth, if only they had gotten tetanus instead I wouldn't have to listen to their jawing
@AndrasDeak Even that is worth a nonzero amount of testimony units
 
@JonClements What change did you make sorry?
@JonClements so will this work even when there is multiple in the Bccs list being passed in?
 
@JamesMcIntyre redacted your email address from your message (Jon is a moderator)
 
Ahh thanks
bascially is it that it doen't like a list of one... maybe if it's one it just wants a string but more than one should be a list?
 
I think you should always give it a string. Which is what Jon's suggestion does.
As a rule of thumb, it's rarely a good idea to have a variable that is sometimes a string and sometimes a list. Or any other combination of types, for that matter
 
1:15 PM
Yeah, with a sane API this would mean that you always have to pass a list even if there's a single string inside
(perhaps if there's a keyword to signal a listified input...)
 
I'm already side-eyeing if Bccs[0] != 'None': but one thing at a time
If* EmailMessage understands lists for To and not Bcc, it might be because send_message explicitly takes the to_addrs argument as a list, but Bcc has no corresponding argument. (*this is a big if)
 
So EmailMessage might just be sticking str(self["Bcc"]) straight into the message header, which is no bueno because the smtp protocol presumably does not understand a Bcc body that starts with a square bracket
 
so much for a sane API
 
@Kevin This is because we keep Bccs in a Oracle database and is often/normally Null which translates to the Python string 'None' (for some reason not None)
 
1:18 PM
Wow... are you sure... that's just daft?
 
cx_Oracle
 
If you can sanitize the list beforehand, try to filter out all the Nones so that an empty Bcc list gets represented as a list of length zero
 
NULL should come back as None... how on earth would you handle columns that genuinely contained the text "None" as opposed to being NULL?
 
Then you can just do if Bccs: instead of trying to look at its contents
 
aye it's probaby bedcause of the way I've pulled it down expecting a list. It's fine though, in this case it's consistantly comming out at 'None' so I can just do the if Bccs[0] != 'None'
I'm currently eating some food and then I'll test the join :)
 
1:22 PM
Summary: don't assign a list to msg['Bcc'], and don't assign it a string that starts with "["
 
would ";" work as well as "," beucase that's the way it sits on our databse table?
 
I haven't read the smtp specification but I'm going to guess "no"
 
Hey guys anybody know how wtforms receives the data from the frontend? Currently I have react sending the information in json format to the route successfully I am trying to figure how to get wtforms to accept the data as if the form was generated by itself and organise server side validation through that as normal
 
@Kevin recommended if you suffer from insomnia :)
 
ahh kk, win32 outlook application wants ";" and smptp wants ",". K surar surar
 
1:25 PM
Trying to figure if it is just a case of having the data laid out in a certain way or if i have to transform it somehow from json into something that could correspond with the form object
 
Now I'm worried that when there's more than one Bcc, the input will look like ['email2@domain.co.uk;email3@domain.co.uk'] , which will not work well with msg['Bcc'] = ','.join(Bccs)
Since it sounds like Oracle is doing silly things to its output
 
unfortuatnly the join resulted in the same thing: it worked for the Tos but not the Bccs
 
I think ; is invalid in email addresses, so perhaps replacing ; with , would suffice
@JamesMcIntyre did you double-check that both fields look exactly the way you think?
 
This is the change:
msg['Bcc'] = ','.join(Bccs)

This is the input:
SendEmail(Tos, Bccs, "File Password Protected", Body)
 
not a fan of PEP8? :p
 
1:36 PM
Apparently not. I find pascale case clearer and underscores are a pain to touchtype with
 
@AndrasDeak I believe foo";"bar@example.com is technically a valid email, but I'm guessing it's de-facto forbidden because nobody writes validators that allow quoted bits
 
I did the same change for the Tos (msg['To'] = ','.join(Tos)) and that still worked

Any idea why the Tos are working but not the Bccs?
 
semicolons outside of quotes are just plain disallowed by the spec
 
what is touchtype?
 
Touch typing, as in, typing without looking at the keyboard
 
1:40 PM
@Kevin I tried finding the email RFC regex but I couldn't, with three minutes of searching
 
I agree that the hypen/underscore key is hard to find without looking
@AndrasDeak I'm cheating by looking at en.wikipedia.org/wiki/Email_address instead :-P
 
@Kevin haha so like typing :D
 
@Kevin and where's the semicolon thing?
ah, it's not named, it's just ;
 
Under the local-part section
> space and special characters "(),:;<>@[] are allowed with restrictions (they are only allowed inside a quoted string, as described in the paragraph below, and in addition, a backslash or double-quote must be preceded by a backslash);
 
Yeah, thanks. I searched for "semicolon".
> John..Doe@example.com is not allowed
huh
 
1:42 PM
I guess this technically doesn't forbid it from appearing in the part after the @, but I don't think I've ever seen a URL with a semicolon
Unless... Can you send an email to foo@example.com?a=b;c=d?
Leave the msg['BCC'] off -- it reveals your hidden sender and has no impact on whether the message is sent to them or not (the arguments to sendmail do that). — Erica Kane Oct 10 '17 at 14:31
So msg['Bcc'] doesn't actually do anything ಠ_ಠ
On the other hand, send_message's doc says "to_addrs combines the values (if any) of the To, Cc, and Bcc fields from msg."... Hmm
 
@Kevin it does: revealing secrets
 
stackoverflow.com/a/56383008/953482 verifies that send_message handles CCs and BCCs properly
 
I would like to edit this question: stackoverflow.com/questions/20449427/…, but it is locked.
Specifically I would like to remove the (Note: ...) as this is now extensively covered in the answers; and I would like to remove the lines play = True, while play:, if input("Play again?")... play =no from the code, as it only distracts from the actual problem this question is about.
How can I deal with this? Is there somebody here who can handle this? Should I ask on Meta?
 
1:59 PM
@mkrieger1 maybe ask on meta. Only mods can touch it now.
 
I agree that Meta is the right place to ask, although I think they'll probably say "eh, it's clear enough already"
regardless of whether it is
Personally I wouldn't mind eliminating all references to Python 2 from the question. It's like having a grammar textbook that says "note: while modern English is typically written left-to-right, ancient Sabaean was typically written boustrophedonically"
It's true, and perhaps even interesting, but perhaps the reader is more interested in things that happened this century
 
Yep, I've just edited the top answer in that spirit.
 
In answers I'm more ambivalent about Python 2 because they usually put that information in the bottom of the post where it's easier to ignore
 
I don't know what you are up about. Python2 be perfectly cromulenth and modernth. >> sys.kevin
 
The grammar textbook can have some cool Sabaean facts in Appendix J if it likes
@MisterMiyagi printf("forsooth");
 
2:50 PM
@smci, I have a labeled dataset(Arabic Tweets) and labeled Lexicon, I want to detect the emotion by machine learning algorithms.

I did the preprocessing step and other functions. just I want to apply these steps:

compute the TF scheme in order to obtain how frequently an expression (term, word) occurs in a document.

To incorporate the affective lexical features we check the presence of lexicon terms in the sentence and we obtain a vector that represents each emotional category (anger, fear, sadness, and joy).
 
by the way it works (as a list), as long as you don't have the recip in both Tos and Bccs
 
3:16 PM
Is there a quick way to define such a class?
class Foo:
  def __init__(self, a, b, c):
    self.a = a
    self.b = b
    self.c = c
I find myself often writing this constructor which seems like it's pretty default
 
@Hakaishin unless you want to do obscure black magic... there's absolutely nothing wrong with that?
 
I have to type a 3 times? That seems pretty wrong to me
especially if it is super_descriptive_name, ofc I copy paste it, but still
 
I dont understand data classes yet but looks to be useful if leveraged well
 
@Hakaishin check out dataclasses, typing.NamedTuple, or a third-party library such as attrs.
 
type hint REQUIRED, bleh, that's how they get you
otherwise it looks neat, but I'm refusing to do types, they suck
 
3:33 PM
I dont like them either, glad to know I am not alone
I dont see myself using mypy (or others), maybe it could come in handy for auto generating doc strings
 
it's pretty convenient for these kind of things. doesn't hurt that you get much better IDE inspections for free.
 
hey guys could anybody give me advice with this... I have got this route in flask dpaste.com/8RDJ94MYB receiving the data sent from this dpaste.com/GVAJE5GY8 react component that is getting the csrf_token on a get request and storing it to send with the form on post, hoping to get it to validate with wtforms through dpaste.com/46FEPBP3Z
the form will have more validators and I have the frontend already validating separately but trying to validate it in the back also
data is being sent as expected in json form
in this form dpaste.com/EU4LVCFCV
I assumed that with the csrf_token being attached and the keys being named according to the form I would be able to get it to validate within there without having to look into validating it without the libraries..
 
3:53 PM
@MisterMiyagi that's not what for free means. If I have to invest something I am paying for it, it's not free. That's like things that are for free when you buy x. It's like no it's not for free, it's in the price included
@python_user Types will be the next big schism I tell you
 
4:08 PM
@Hakaishin call it two for one, then
 
4:26 PM
@Hakaishin they have been a big schism in python for years already
 
true, but not as big as 2.7, I guess?
 
.7?
 
@smci thx
 
4:44 PM
tbh I didnt even bother reading up on types, I only know they exists because I see them used in leetcode, which tbh has py 3.8 when most of the code sites dont
 
this is the first time I have seen Kevin move a message :O
 
cat walked over his keyboard
 
types are pretty useful, type hints not as much
 
Should we make Kevin's cat a RO?
 
4:46 PM
For example, int, list, and dict are types that i have enjoyed in the past
 
"type hints" should refer to passive agressive error messages that don't flat-out tell you that you passed the wrong type
 
"A thoughtful programmer would've given me an int to work with, but I suppose I'll make do with a string. Hmph."
 
raise TypeError(f'"{arg}", seriously? What do you expect me to do with that?')
 
 
4 hours later…
9:09 PM
@Kevin IIRC you have to put all recipients in sendmail's to_addrs whether they are "To", "Cc' or 'Bcc" recipients. The Bcc header may be transmitted to SMTP relay hosts, but not to the final recipients. Addresses inside the envelope (in the message) aren't used by SMTP at all for routing. so send_message does the correct thing by extracting the "to", "cc" and "bcc" addresses to_addr.
 
Anybody got a favorite way of measuring total memory usage from within Python? I can see a few ways, but wondered if you guys had any opinions.
 
9:23 PM
I'd use a dedicated memory profiler
Assuming you want to profile your code. With you, it's probably something weird instead :P
 
@bashar: There are gazillions of existing questions on SO on Term-Frequency computation on text, please go search SO for them. Start by solving that part before you move on to the sentiment parts.
 
9:56 PM
@AndrasDeak You know me too well. I just realized I need to get the high-water mark of the process memory consumption. So if I call a C function from python, I need to somehow capture the memory allocation of that C function. Yikes.
So in principle, I could get the total memory used by the process and children with psutil.
BUT if for example, one of the C functions allocated 10 jigabytes of memory, and then frees it, I can't really know.
resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
some kind of russian memory utility
 
sounds perfect for you ;)
Assuming your name is Russian, and not some other Slavic, or other non-Slavic :D
 
Yeah, now lets see if it actually works :-)
 
10:12 PM
welp, it doesn't work
I called a C++ function that basically allocates 2 gigabytes and then free it, and the resulting code showed nothing...
sad
 
Did the alloc succeed?
And did it not get optimized away during compile?
 
Yeah, it succeeds, and no its not optimized away
I actually revised it with just using a C-style malloc()
 
10:28 PM
I'm looking to use Python to interact with a graph db. I need to query the graph db to ask "Tell me the customers who live in Europe" then "tell me how many of those customers have blonde hair, black hair, and gray hair"
Essentially I'm looking to make a graph query and get SOLR-style facet counts in the response.
Anyone have any recommendations on databases?
 
10:42 PM
Yeah, so .ru_maxrss seems to give garbage, parsing /proc/self/status seems to be the most accurate way
 
11:30 PM
unclear. yet another case of conflicting package versions installed under anaconda/pip. Title is grossly misleading. stackoverflow.com/questions/65909788/…
 

« first day (3757 days earlier)      last day (1174 days later) »