« first day (3206 days earlier)      last day (1745 days later) » 
00:00 - 18:0018:00 - 21:00

6:02 PM
@roganjosh also I'd drop the conversation (comment) with the OP of that question - its just going to turn ugly based on previous experience.
Also, you're always going to have high CPU usage unless you put breaks into the Mongo process calls & Python code (that's how it works - asking on Raspberry PI or on of those would benefit OP)
 
I already asked for it to be closed
 
Also, cause its just cool & I'm not sure people know about it yet: I'd use the new client change streams for what he's asking
 
Two different line here; someone asked in chat about flask-migrate and disappeared when I asked for an MCVE, and someone who wants to multiprocess out of an unspecified hole. Standard day :P
 
yeah, that's why I haven't answered a question in a year outside of chat
 
6 months ago I was all over the main feed. cba now, it's too much tbh
 
6:08 PM
Hi guys I have a doubt
I have a pandas dataframe that has two columns containing values as lists
 
also, I hope he watched the Edinburgh talks as that was the only conference I remember being about Python 3 (one of the Cython core devs did it - was awesome)
 
I am trying to check if one particular value of the 1st column is in the 2nd one
 
@RaphX You shouldn't have a df with lists in columns
 
@roganjosh Why?
 
it's so rare to have a legit good use case for columns with lists.
 
6:09 PM
Cells (referenced by row and column indices) should be scalar values. Otherwise you trash pandas
 
and you lose out on every good bit of pandas performance wise
 
@roganjosh absolutely agreed
 
You might as well have a nested list
 
@JGreenwell That was a feature that really attracted me to CouchDB, which already had it when I got to it a couple of years ago. Quite a neat database for certain purposes.
 
@DeveshKumarSingh oh, i get a unicode decode error on that one. maybe my codepage is out of date.. I'll have to try it on my workplace pc as well. Does '\U+1E9E'.lower().upper() also return 'ẞ' for you, or is it 'SS' then, if you don't mind testing it for me?
 
6:12 PM
I was not given "a choice" with MongoDB - not that I dislike it just didn't matter at the time :) - (I also like CouchDB)
 
I am actually comparing addresses with a particular format and therefore was splitting it into parts to see if it was matching a value , hence the list @ParitoshSingh@roganjosh
 
@RaphX sounds like a XY problem, please have a look.
 
@RaphX - maybe you should change your name to RaphXY? ;-)
 
@JGreenwell I will look it up. But I'd bet money on this issue just being "I've heard it's faster"; there are many numpy questions like that
 
@RaphX i take that to assume all "lists" have the same length?
or well, i suppose depends on whether you mean normal addresses or not
regardless, MCVE for your question?
 
6:14 PM
Perhaps we could go a little further back and ask what problem you are trying to solve. Your terminological inexactitude is sometimes difficult to understand clearly.
 
I think this is the umpteenth time they've been asked for an MCVE
 
Righty ho.
 
Actually I wanted answer for that but since you asked now I have to make one :P
 
@roganjosh oh, with that OP - nah, that's an XY problem (or a I only understand Development and have wandered into engineering land problem): I just thought it was cool in general and likely how I would set up something I needed to have "breaks" due to CPU limitations (I'd hook them to the listeners) :)
 
@RaphX answer for?
 
6:15 PM
@RaphX why is it not the other way around? make one first, don't ask without one.
 
^
That. Totally that. I don't know why we're still in this cycle
 
@RaphX yeah, no.
 
@JGreenwell aside from the SO side, what is it you're going to be lecturing? It was you that's transferring to academia?
 
Sorry again @AndrasDeak
 
@Arne here you go
In [7]: '\u1E9E'
Out[7]: 'ẞ'

In [8]: '\u1E9E'.lower()
Out[8]: 'ß'

In [9]: '\u1E9E'.lower().upper()
Out[9]: 'SS'
 
6:20 PM
Nice!
 
...
 
@AndrasDeak did you get back okay? (or is this your bot talking? :P ;)
 
thanks a bunch for testing, interesting results for sure
 
and that throws a unicodedecode error too
In [2]: '\U+1E9E'
  File "<ipython-input-2-03031666d092>", line 1
    '\U+1E9E'
             ^
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 0-1: truncated \UXXXXXXXX escape
I am not sure how I posted that first thing
 
I'm waiting to hear on one course for this Fall but so far Software Engineering Fundamentals & Advance Structured Programming: OOP in Fall; Spring I have at least Data Engineering (Junior level) course
 
6:23 PM
@JGreenwell sat on the stairs all the way from Vienna (2.5-hour train trip), but otherwise okay, thanks for asking :)
 
ouch & glad to hear it
I'm also going to be working with ETI on some research things
 
just for the record, I sifted through the transcript right now and was accused of 1) being dutch (I'm not, and Arne is a swedish name, which I'm neither), and of 2) calling IBM Watson's recognition capabilities "cancer", which is also not what I meant. This Sam guy has an impressive ability of reading into things.
 
We missed the hard handedness of AD on that one
 
i feel you @AndrasDeak :) wish you a safe one
 
for sure
 
6:26 PM
Here is the actual problem
 
@JGreenwell Nice! Florida! been there once! It's an interesting place for sure. with so many florida man stories
 
I thought splitting would be a good thing thats why the lists came up
 
@roganjosh That was clear misunderstanding. As far as I know Flask-Migrate is an extension that is not needed when you are actually running the application, it's only used when we run a flask db command. For that reason we do not need to have it initialized along with the application, we can just enable it for CLI use.
 
lol, I'm from IL (Illinois) but moved all around the country while in the military and meet my wife when I decided to take leave (military's term for vacation) in Florida .... and now I'm stuck :)
 
ohh nice, I lived in Indiana for 2 years, IL is nice! Been to chicago like 25-30 times
 
6:29 PM
@RaphX your df in MCVE does not have lists in it. does your real df have lists inside?
 
I was making lists to solve this problem @ParitoshSingh
 
But I for sure don't miss the cold of the mid west. brrrr....
 
The real df doesnt
 
@anky_91 thanks
 
@Debendra sure, well I'm glad it's fixed, but you really should say if an issue solved because I know people will come and read through this transcript. You're welcome.
 
6:31 PM
I have a python flask web app and I have it running in azure - I'm trying to make it create and write to a file but I think I need to add permissions to the dockerfile? Does anyone have any suggestions?
 
@RaphX you need to compare the first or the second split? or both?
 
@cookies did you just try it as-is? Did it work? Or did it not work? I don't think you need to add any special permissions
 
@RaphX so given a string such as "12-14/6/6", the text after the first slash is "6/6". Do you need to check "6/6" or just "6". and if it's just "6", what's the criteria for picking how much to check?
 
^^
 
in docker or Python? if docker just ADD --chown=<user>:<group> <hostPath> <containerPath> (like admin:admin)
 
6:34 PM
@JGreenwell so add to the docker ADD --chown=admin:admin?
 
@ParitoshSingh Yeah just 6, sorry forgot to mention that , I just need to check the middle section (after the first slash but before the 2nd one)is present or not in the second column
 
@RaphX okay. now, i have a question to you. given a string such as 'a/b/c' can you write code that picks the stuff between the two slashes?
 
if its simply a permission issue (and there is a admin user in the admin group -- otherwise replace with whatever your's are named)
 
@RaphX try to do it yourself first
 
I can do it, but i'd like you to do it instead.
 
6:36 PM
@JGreenwell Thank you - how would I check what I am named if it isn't admin?
 
@ParitoshSingh thanks
 
you'd see what groups/users you have and what the permissions are - with Azure I'd use powershell and just check but I'm sure they have some GUI or Server tool for that too
 
@roganjosh Yeah it is fixed, there was nothing problem with the setup, but there was little typo in database configuration variable. It should be SQLALCHEMY_DATABASE_URI.
 
@JGreenwell and should the ADD be at the top of the dockerfile
@JGreenwell thanks Ill have a look now :)
 
@RaphX absolutely with @ParitoshSingh and @AndrasDeak you would learn by making mistakes. :) try and post what you have tried. :)
 
6:38 PM
Here are the powershell docs for Active Directory with Azure if you want to read up on what I meant by "use powershell"
 
@Debendra that's in the basic setup guide for flask-sqalchemy. I don't understand how you go as far as flask-migrate before you had issues
 
do anyone is there any difference between db.JSON vs sqlalchemy.dialects.postgresql.JSON when using postgresql ?
*know
 
@RaphX i would have splitted too(for a hint) don't know about others
 
but again you can probably find a GUI or Server tool if you don't need to automate it
@Debendra if db = sqlalchemy.dialects.postgresql then no there is no difference (otherwise what is "db"). Note, there shouldn't be unless we are talking Binary Json
 
@roganjosh I failed to detect typo actually, it was well working before but as I start changing code It was mistakenly placed.
 
6:44 PM
@JGreenwell So just double checking - if I add: 'ADD /application --chown=admin:admin' where /application is the directory which I want write permissions and admin is the username that should work?
 
@anky_91 I know that strng.split('/')[1] will give me the 2nd thing
 
Jul 19 at 13:15, by Andras Deak
Jul 9 at 14:24, by Andras Deak
Jul 4 at 11:59, by Andras Deak
Jun 27 at 11:26, by Andras Deak
Jun 6 at 11:37, by Andras Deak
@RaphX Please try harder. And read how not to be a help vampire. You are seriously depleting patience of the users here, which is a non-renewable resource.
@RaphX Try harder.
 
I would test it somewhere you don't care (I always do this when increasing permissions) but yes if all the other stuff is setup it should work with docker
 
@RaphX yeah, carry on from there
 
@JGreenwell Thank you - if I get stuck ill come back with hopefully a more detailed issue
 
6:46 PM
anyway, I'm off (and there are Docker experts way beyond me in here) so hopefully it gets you started
 
Ok
 
Thank you.
 
7:07 PM
df['apt'].str.split().str[1].isin(df['address'].str.split('/').str[1])
Is this good enough? @ParitoshSingh@anky_91
 
does this solve your problem?
 
Yes
 
Congratulations
@RaphX
 
then great, well done.
 
:)
well done
 
7:09 PM
can we* expect you to make that kind of attempt on your own next time too? break down your problem first, work with strings before worrying about dataframes.
 
Ok
 
^can we*
 
@ParitoshSingh *we will, @Raphx
 
Ok
 
I am happy with your code: I had df['match']=[a in b for a,b in zip(df.address.str.split('/').str[1],df.apt)] just an addition , if the df size is large (for smaller dfs it doesn'tmatter), a list comp might be faster since str accessors are not vectorized in pandas, but yeah, please ponder on what @ParitoshSingh said :)
 
7:15 PM
interesting, i didn't know that string accessors are not vectorized. (and i didn't know they were called that till today :P )
 
@ParitoshSingh yw ;) :P
yes they are slow slow slow... :(
 
It sometimes really makes me wonder why is it exactly that strings have such a bad time with vectorization, and can something be done about it. Then i realise there's probably people smarter than me who have looked at the same thing before and probably in more depth
so i just assume there's a good reason they're weird, and let it be.
 
because they are strings, I am sure that is the case with every other language(IMO not sure)
 
docs say it's vectorized? Is that not correct?
 
i doubt. its too slow. i will come up with an example for you(not today @Saturday night :P)
I have seen a list comp to be faster
in my real time exp
Besides , this is a good read
My concept of vectorized is fast(may be I got that wrong)
 
7:29 PM
ah, no. vectorized is basically "work on a batch at once". It's basically like parallel processing to be honest if you want to think of something similar
ofcourse, working on a batch at once is generally going to have the added bonus of being fast at going through the whole data usually
 
yes, that's where I went wrong. :) It doesnt in some cases apparently
 
however, the post you linked does remind me that strings fall back to loops. thus leaving me more confused than before
either the docs are lying, or the statement "Strings fall back to loops" isn't always correct?
anyways, definitely too heavy for a saturday/sunday :P
 
"fall back to loops." didnt understand that. :/
 
oh, as in, operations that "look" like they're working on a series, are, under the wraps, going through a loop and working on 1 value at a time
such as most cases of df[column].apply(f)
 
yes there are a lot.
@ParitoshSingh right
even replace()
its a loop under the hood
most str accesors IMO
 
7:44 PM
hey guys, for coding interviews what are the most important things to know about searching algos?
for sorting algos i know bubble sort, merge sort and their time complexities
 
`sqlalchemy.exc.NoReferencedTableError: Foreign key associated with column 'customers.subsidiary_id' could not find table 'subsidiary' with which to generate a foreign key to target column 'id'`

I am using latest flask with blueprint.
I am getting the above problem even I have specified table name for every model.
However only importing Subsidiary model within the customer class file solves the issue.
 
8:14 PM
cbg
 
8:25 PM
rbrb
 
00:00 - 18:0018:00 - 21:00

« first day (3206 days earlier)      last day (1745 days later) »