« first day (3378 days earlier)      last day (1570 days later) » 

12:01 AM
If you move the field, then the apartment info form wouldn't need a field
 
@Code-Apprentice Thank you so much for your time. What do you mean move the field?
 
to be more clear, if you take my suggestion and add info = OneToOne(ApartmentInfo, ...) to RentalProperty and remove rental_property from AparementBasicInfo. Then CreateApartmentBasicInfoForm wouldn't need the field.
FWIW, I've never used OneToOneField that I can remember. If two models really have a one to one relationship, then they probably should just be a single model.
 
@Code-Apprentice I will do that. It is quite late here so I plan to do that in the morning
 
Getting to the end of the day here, too
 
I was thinking having a long model might not be okay for readability but if the issues continue I might just have to make it one model
I guess all these are due to my lack of experience
 
12:08 AM
@superv yah, the way you are breaking up pieces of the model makes a lot of sense. I think the biggest issue is with how you define the relationships.
and understanding how to traverse those relationships in a template or other code
 
What if i just remove the OneToOne relationship and just use foreignkey
Would that make sense?
 
@superv yes, it can. Let's talk about this in the context of the relationship between two specific models.
If you changed rental_property = models.OneToOneField... to rental_property = models.ForeignKey... in Location. Then you can potentially have multiple Location instances have a foreign key to the same RentalProperty. In this particular situation, would that make sense?
On the other hand, if you have location = models.ForeignKey(Location...) in RentalProperty, you can have multiple RentailProperty objects with the same Location. Does this direction make sense?
 
Hello, sorry to interrupt with a totally different topic, i am wondering which way is better to pass config variables in many files inside a project, i am hesitating between (list, yaml file, class, .cfg file and dataclass) any advice is welcome 🙏
 
I think the later might make more sense. Say if the RentalProperty represents a single apartment. But the Location is the address for the apartment building.
@user3821178 no need to apologize for the interuption. We are probably monopolizing the room than we should. Your question is incredibly broad, so it is difficult to answer.
 
thank you for the reply, i have some settings that i pass around many files and it endup becoming repetitive, i am looking for a more pythonic approach to send those settings data(mostly integers) in the different modules
 
12:22 AM
@Code-Apprentice I have to really get the concept of the OneToOne and foreign key very well. Thanks so much for your time. I guess I should go to bed now so that I can be productive in the morning. I will go through your suggestions again and implement them. I will let you know how it goes. Thanks again
 
@user3821178 What do you mean "pass around"? Do you pass these as arguments to functions?
@user3821178 I suggest that you create a minimal example that reproduces the concept you are asking about.
 
most of the time i set them as global variables in the modules that need them then if i want to use them in a function i just access them from
 
@user3821178 one option is to create a settings.py module with all your global variables. Then import settings wherever you need it.
 
yes thats what i intend to do but inside the settings.py, i wonder what to use either a dict, a class, a dataclass...
to store the global variables
 
I usually just have simple variables, strings and integers, etc.
 
12:42 AM
i think maybe a dict will be better for my case as most of those settings are needed per module. Thank you so much
 
 
5 hours later…
5:27 AM
cbg guys o/
 
 
1 hour later…
6:30 AM
Since I did not receive any response from the person who posted that long answer, I am gonna quote his answer on my question post and will give the credit to his long answer.
I hope God of SO will understand and forgive me if by any chance it makes me sinful.
 
6:55 AM
So I posted the answer finally: stackoverflow.com/a/59746405/1944896
 
7:20 AM
@user3821178 sounds reasonable
 
cbg everyone
 
morning cbg
 
cbg
 
 
3 hours later…
10:31 AM
@AndrasDeak Can we cast close vote on this question being duplicate of this question ?
 
11:23 AM
same error, same method i.e. cache.set, came calling i.e. outside request
 
11:41 AM
Hello in Flask framework, the logger is flask.app. But can the root logger still be used?
 
 
1 hour later…
12:47 PM
@TheLittleNaruto I don't have domain knowledge to be sure
 
^^ Hammered
Cbg
@variable You can always configure and use your own custom logger, but I think the default Flask logger takes care of the normal web-server-ish logger setup, so why duplicate?
 
1:14 PM
@PaulMcG Im trying to understand this concept. I am confused since flask has its logger (flask.app) and there is also the python root logger. And in python, since propagate is true by default, so will the logs from the flask logger propagate to the root logger? If so then what is the default logging level of root logger (usually in python it is warning). But I dont see the logs displayed twice. SO Im bit confused
 
@PaulMcG Thanks Paul and cbg o/
@AndrasDeak No problem :)
 
@variable I'm sorry I have to break off for a bit - sudden work issue
 
another day, another dupe of this but I can't easily find a good target (I can find pleny of bad ones). Convert dataframe column from Object to float or int. Convert the dtype of several dataframe columns from the default when read in from CSV
@ParitoshSingh Just found out.
 
1:45 PM
@AndrasDeak You and Jon are right. Turns out it's even worse than that, since count() is an infinite iterator, it breaks if we ever use zip_longest/ iterator lengths differ. I posted an answer saying that.
 
@variable why do you think it would be displayed twice?
 
@smci I can't be right on the 15th of January!!! It's far too early to use up my being right once a year thingy... take it back! :p
 
@JonClements Umm I'm still reading about yesterday's SE events... would affect one's digestion...
 
morning cabbages, all
 
@smci it's not something I can say I care about any more... I'm just glad I made up my mind in October
onto happier subjects?
@inspectorG4dget morning cabbages mate
 
1:56 PM
@JonClements Indeed. If you gc a cbg what's left?
 
@smci how about this answer
 
@smci always cbg in one form or another because it's eternal? :)
 
bdw cbg :)
 
@variable how are you running the flask app? The development server?
 
Oh gawd, just reviewing a django project where the previous dev left 'em... and their login code is errr.... "interesting"
 
2:00 PM
@anky_91 No, Convert dataframe column from Object to float or int is about doing dtype conversion after the read_csv, not about reading in as float in the first case. But all these things are vaguely related, and we get more dupes daily, just a big headache...
 
umm, cannot find a relevant one either
 
@JonClements hey puppy! It's been a while. What's shaking?
 
@anky_91 Yes it feels like a trash compactor with maybe a snake, but definitely no princess...
 
lol
 
@inspectorG4dget oh you know, this, that and t'other :)
 
2:10 PM
@variable I assume you've already gone through the docs which mentions under which circumstances the default handler is/is not added?
 
right on. I recently discovered that a tool I wrote for my thesis is actually useful in the real world.

Mind. Equals. Blown!
 
2:47 PM
Greetings all!
 
Hello :)
 
3:00 PM
@roganjosh blah blah about some not liking sharing stuff... this one came onto my random play list for the first time in ages - I don't mind that either
(well - it makes sense when you're drinking Captain Morgan at 4am or something... :p)
 
@JonClements hahaha. I'll have to stack that up on my playlist as I can't listen just now. I feel a tad bare in that I've not encountered anything new recently to share back :/
 
@roganjosh I was just browsing on stuff and play lists
was quite curious to seen that Maroon5 have done anything
been thinking a long while they were a 1 hit wonder
(love the lyrics)
 
I've mainly had podcasts or Mind Field playing recently, not music. I have to move out at the end of the month and thinking I need a different job/contract in the process. I just need to figure out what I what i actually am/do these days :)
@JonClements yeah, it's a nice distraction from the mainstream chart music these days. It comes on like every night on my drive home cos they play the top 5 songs at 7pm every night
 
What to do with stackoverflow.com/questions/59735435/… ? Leave open? Close as 'Needs focus'?
 
My programming has always been under a title of "data scientist" but I think I'm too far behind the trend on big data now. "Operations research" or "operational research" is probably more accurate but I have a feeling most companies don't really know that term.
 
3:14 PM
@roganjosh Always a "fun" choice
I just go for - "I'm just a guy, ya know?" :p
Got a few database updates to do then planning to finish at 4 and have something nice to eat and raise a glass to Shog
 
@smci it's a viable question, even if the use-case is not advisable.
there might be dupes around, though
 
Well, as of the last 2 minutes, I can't do anything on mobile chat but flag comments now; reply option is gone :/
 
@roganjosh if it's still the same bug I remember, then you can tap the message and scroll a bit for the options to appear?
 
Nope, that's gone. I've been replying to you just fine using that trick... until now :)
The options are back, I guess they were moving icons about
 
3:40 PM
@MisterMiyagi Do you think that sort of question should be downvoted? What's your threshold for deciding if a question shoud be downvoted?
 
I downvote questions that directly result from very bad software design decisions.
In this case, I find it viable to want to set nested attributes (hence the answer), but I find the approach not viable to theunderlying problem of the OP (hence the downvote).
it's an XY problem, basically, where X is viable by itself but not for solving Y.
 
I didn't downvote but VTC as needing more info because I think there was enough feedback on the "why" which the OP hasn't responded to
 
I'm debugging some slow code. This means I frequently have 10ish minutes of just waiting. It's not long enough to do a backlog task, but it's long enough to do something. I've cleaned my desk. I'd like to do some simple programming challenges to stay sharp. Any suggestions?
 
I think the "why" is pretty clear, it's just a bad idea.
 
@inspectorG4dget project Euler?
 
3:54 PM
good idea. Thanks
 
4:07 PM
Hi guys, got a short question in regards to a quite simple python problem
 
ask away... or ask us. Just ask
 
Please Shoot. :)
 
shoots
 
scores
 
dead
 
4:09 PM
english is a funny language, isn't it. :)
cbg folks!
 
cbg,.. indeed :) versatile i'd say
 
well AFAIK, it's the only language where Booty Call and Butt Dial kinda mean the same thing... but they really don't
 
I have a big datafile (like 150.000 rows). I import it and format it correctly. I then want to filter the rows based on a number of criteria
First criteria: a certain column has to match a specific value
 
answer = []
for row in rows:
    if row[col_idx] == value:
        answer.append(row)
 
@Leon before you proceed, the question should have a mcve or it has to be absolutely obvious :)
 
4:12 PM
Second criteria: I then want to filter this data for a specific data
I think my question will be quite obvious
 
"match" <- do you mean an exact value, or do you mean like a regex match?
 
exact value
 
does case sensitivity matter? do you want to strip our leading/tailing whitespace?
 
first things first, before we proceed further, how are you importing the data? are you using pandas?
 
df = pd.read_csv (r'D:\project\Data\Short_Interest\mergedshort.csv')
df['Date'] = pd.to_datetime(df['Date'], format="%Y%m%d")
short = df.loc[df['Symbol'] == 'ARAY']
this is what I have now
 
4:14 PM
@Leon how did you import?
 
(aside: I myself prefefr to use monkeys instead of pandas - sometimes they get confused and start writing Shakespeare)
 
@ParitoshSingh beaten
 
I filter on the column value 'ARAY'
 
then use short to index back into df to effect the filter
 
4:16 PM
I have a variable which is can either change or remain same after each iteration, if the said variable doesn't change its value, I wanna to break out from the for loop. How to do that? I have tried setting, px = x, after the check that px == x?, but px keep getting changed when x is being changed, which always satisfy the check.
I have tried another method of append the x into an array, but in that way I am getting the same issue.
The last element of the list changes when x is changed.
 
@inspectorG4dget lol
 
@AjayMishra does the variable or the value of the variable get changed
 
@Leon okay, so what do you want to do after you filter?
 
@MisterMiyagi value.
 
@anky_91 Filter for 1 specific date and 10 observations before and after this specific date
 
4:19 PM
@AjayMishra then you must create a proper copy.
 
What do you mean?
How?
 
@AjayMishra please provide MCVE
 
@Leon you should create a mcve a short one on what you want to do
 
@AjayMishra without knowing any details, px = copy.copy(x) should work for most cases.
though since you would be changing the value yourself, how come you cannot detect that?
 
@MisterMiyagi It worked, thanks. I ain't t changing the value myself.
 
4:23 PM
countdown = 2
while True: # or some other condition
    countdown -= 1
    if someCondition():  # change x
        x = 5  # some new value
    if countdown <= 0: break
 
@anky_91 Okay, and post it here?
 
if its not too big you can take a took here , else you can create a github link and share
 
Date 	Symbol 	ShortExemptVolume 	ShortVolume 	TotalVolume
0 	2011-01-03 	AAWW 	0.0 	28369 	78113.0
1 	2011-01-03 	AMD 	0.0 	3183556 	8095093.0
2 	2011-01-03 	AMRS 	0.0 	14196 	18811.0
3 	2011-01-03 	ARAY 	0.0 	31685 	77976.0
4 	2011-01-03 	ARCC 	0.0 	177208 	423768.0
5 	2011-01-03 	ASCMA 	0.0 	3930 	26527.0
6 	2011-01-03 	ATI 	0.0 	193772 	301287.0
7 	2011-01-03 	ATSG 	0.0 	23659 	72965.0
8 	2011-01-03 	AVID 	0.0 	7211 	18896.0
9 	2011-01-03 	BMRN 	0.0 	21740 	213974.0
10 	2011-01-03 	CAMP 	0.0 	2000 	11401.0
This is how the data looks like
 
it is too much, along with code
put it in a self-contained pastebin or dpaste or github gist which can be run as-is
 
I want to create a time series for a specific symbol, and for a specific time window around a date
df = pd.read_csv (r'D:\project\Data\Short_Interest\mergedshort.csv')
df['Date'] = pd.to_datetime(df['Date'], format="%Y%m%d")
short = df.loc[df['Symbol'] == 'ARAY']
 
4:26 PM
you can use a StringIO instead of your csv so that it's self-contained
 
All I have so far, this filters for the symbol, but not for the specific date and time window (for example +10 - 10 observations)
 
@inspectorG4dget Oh my. Why was that your default for a dataframe?
 
@Leon Check what @AndrasDeak says. :)
 
@roganjosh it wasn't. I misread a previous message as OP knowing when x will change. Also, I didn't think it was a dataframe
 
@Leon A suggestion (my personal suggestion); take a step back here and formulate a complete problem rather than adding in multiple messages. I'm struggling to follow
 
4:28 PM
@roganjosh at that time, it was a "quite simple python problem". Not a 150k data frame.
 
haha, I can't imagine that filtering a dataframe for two specific column values is a difficult problem.
 
Yeah, I'm seeing that now. It's hard when looking back on a transcript how the conversation flowed sorry
 
all good. I'm trolling the corp slack channel while I wait for pickle to crsah on me
 
@inspectorG4dget it wont, the channel would IIUC :P
 
@Leon nobody said it was a difficult problem. It is, however, evolving over multiple messages. If we're in agreement that it's not difficult, then you could have posted something clear and self-contained
 
4:31 PM
@anky_91 that'd be a refreshing change of pace, tbh
 
@Leon may be we aren't as intelligent sorry about that
 
yeah, you have to find smarter people
 
I am bad at explaining :/
 
@Leon as we've said: take a step back, put together a self-contained and runnable example dataframe and show what you want to do or how what you're trying to do doesn't work
there's no rush, take your time
 
Working on it!
 
4:34 PM
thank you :)
 
it'll benefit us all
 
@roganjosh Hi. I have not understood this please can you tell me
 
@variable I'm not sure what part is unclear (I mean that genuinely; what part confuses you?)? It talks about the circumstances that the default logger is or is not added
I'm also not sure why exactly you want this. What's the end goal?
 
Hereby a running example of my problem. pastebin.com/cj9CpJ6v
I want to add a criteria for a specific date, for example, to filter for the date '2011-01-04'
 
There is no 'default logger' word mentioned in the document. I'm trying to understand this: during a request when I use logging it logs to the app.logger Or the werkzeug logger? And why does it not propagate to root logger. Just trying to find documentation on this please
 
4:50 PM
@roganjosh Is it clearer now?
 
@Leon short = df.loc[(df['Symbol'] == 'ARAY') & (df['Date'] == '2011-01-04')]?
 
And what if I want to include +1 and -1 of the observations around '2011-01-04'
with observations I mean rows
 
I'll be home soon, it's a nightmare on a phone to type code, but you can set the datetimes beforehand and filter on between
 
I do not want to use datetimes, as in my dataset some dates are skipped, so I therefore want to use rows
 
@variable "default handler" even.
 
4:57 PM
Use a forward shift and a backward shift to re-align the date in two new columns. build conditions for those columns.
 
32 mins ago, by Leon
I want to create a time series for a specific symbol, and for a specific time window around a date
 
If, im interpreting you correctly in that you want rows above and below the rows where date matches '2011-01-04' without actually worrying about what date they themselves contain
Which is a big if
 
I'm getting too confused with both questions. Taking a break till I'm back at a laptop
 
@ParitoshSingh Exactly
 
cool, explore .shift to create new columns. See if you can figure out the actual conditions themselves
 
5:05 PM
what's the expected output? there is no mcve without expecte output IMO@Leon
 
5:29 PM
@JonClements I always wondered the WWE (the wresting one) song. it's not as popular here, that's it. Thanks :)
 
@variable Right, back at my laptop. What exactly is it you're trying to do? I suspect that it's wasted effort because you'll want a deployment server and it's going to redirect stdout elsewhere anyway once you're running
 
@AndrasDeak I'm still amazed the syntax highlighting doesn't give people a hint...
 
@roganjosh I'm tying to understand flask logging better. Yes presently I am using the development server (also known as werkzueg I understand). I am confused because logging writes to flask.app logger but I read on that link that werkzueg writes to logger by the name of werkzueg flask.palletsprojects.com/en/1.0.x/logging/#werkzeug
Bit confused why werkzueg writes to different logger when there is this flask.app logger.
And when I use a production server where will it write to. I am trying to understand this.
 
It doesn't. Flask has its own log handler
I'll be honest; I don't understand the inclusion of Wekzeug at the bottom of that doc that we've both linked to
It is, however, a subtitle under "Other Libraries"
 
6:08 PM
Ok. Its not a subtitle though
 
Guys do you know how can I insert a float inside here? position: {lat: {{ latitude }}, lng: longitude} I get an error because of double curly brackets
its javascript
 
Yes it is. Check the font sizes. And if you don't believe me, look at the page source to see that werkzeug is a subheading to Other Libraries
@DzITC This is the python room. There is another room for JS
@DzITC I'm not sure that's valid JS tbh. Are you sure it's not from a template rendering language like Jinja2 or Django?
 
Omg ok
 
@variable It definitely takes a keen eye to see it, I'll give you that :D
 
If i reopen a dupe marked by Community , how shall I address it?
 
6:15 PM
Probably they are trying to say werkzueg is just like any other library which has its own logger implementation. And if you wish to use it, then here is the name of the logger.
 
I think you're probably correct. Flask defers plenty to werkzeug so I guess they are suggesting that people might want to log those actions separately <shrug/>
 
without loss of generality, I have a function that takes a list of strings and converts it to a list of ints (['1', '2', '3'] -> [1,2,3]). I have a LAAAARGE list of such lists, and I'd like to apply the intifier function to each sublist in that list. This takes a long time. So I'd like to throw multiprocessing at it. The module has mp.Pool, which looks attractive. pool.map produces a list which causes my memory to blow up. So, I'd like to replace each sublist in-place
So I thought I could use pool.apply_async, but that turns out to be single-process only [ 1). How can I do this without writing my own process pool?
 
did you apply apply_async to sublists, not the entire list?
note that subprocess generally doesn't do "inplace", since each process has separate memory
 
I did apply_async on the big list. I know that the subprocess will have it's own memor,y but I can allow it to use "a new sublist" worth of memory
 
6:32 PM
Heya. I have a 1000x1000x3 numpy array data.shape # (1000, 1000, 3)
I want to turn it into a 1000x1000 array, where each number is based on the three numbers per cell
in other words
new_data[x,y] = old_data[x,y,0] * old_data[x,y,1] * old_data[x,y,2]
What is the easiest way to achieve this? I would loop through the entire array and make a new array, but there must be a better way
 
with numpy arrays, there's usually a better way compared to iteration
 
a.sum(axis=2)?
 
looks like numpy prod has you covered
 
My example has a simple multiplcation, but it shoudl actually be a more complex formulabtw
 
<rubs eyes> * not +, doofus @roganjosh
 
6:37 PM
Sorry, my example was to concise haha
 
@MitchellvanZuylen that's just a matter of breaking down a complex formula into a sequence of simpler steps.
Same as with any logic that you'd build while iterating
The process of writing the logic itself doesn't change, just you start thinking in terms of vectorized operations
 
@MitchellvanZuylen new_data = old_data.prod(axis=2) maybe?
 
With the caveat that it wont always be easy to figure out, or worth the effort. But usually there's good payoffs for vectorization.
 
The real formula would be new_data[x,y] = (0.2126*data[x,y,0]) + (0.7152*data[x,y,1]) + (0.0722*data[x,y,2])
 
that doesn't seem bad at all. Just scale the values first before using numpy prod
import numpy as np
inp = np.arange(12).reshape(2, 2, 3) #dummy data
scale = np.array([0.2126, 0.7152, 0.0722])
out = (inp * scale).prod(axis=-1)
effectively just utilize broadcasting to scale all the numbers first based on the coefficients as needed
Then take the product
 
6:45 PM
Thanks! Will try this :)
 
I'm not always great at visualising beyond 2D arrays so I was working through it step-by-step to be sure I was along the right lines. This is something you could do yourself, with much smaller examples.
 
^ likewise. Small examples make things a lot easier to figure out
 
a = np.random.randint(1, 100, (5, 5, 3)) gave me a random sample that had the first two dimensions the same
Then b = np.multiply(a, [0.5, 0.2, 0.1]) allowed me to confirm that I was multiplying along the right axis
Once you confirm the approach, you can then drop your real values in. But I think Paritosh and I are on the same lines here; just make small examples and build up from there
 
7:13 PM
@MisterMiyagi ugh, system ate my second dupe target. Fortunately I still had it open
@inspectorG4dget numpy array and .astype(int)? Copies but much less memory than a large list
If you have the list I wonder if np.array(lst, dtype=int) works. My guess would be yes.
 
7:58 PM
Gah, that guy is unworkable
and to suggest I need to drink coffee? No, tea, mate, thanks :P
 
Look at the PDF and look at the html output. They are different. I provided a minimal example and not my complete document which I do not wish to disclose. I do expect my pdf document to look like my html. I am sorry this confuses you. I need help not irrational, half-baked, illogical sarcasm. — Mike C. 31 mins ago
Yeah, no
 
PaulMcG is his favourite, I just get insults if I try to reply.... even though I'm 99% sure his issue is in the Jinja template and he's ignored that request. I'm not sure how he expects us to help render the template.... if he doesn't give the actual template he's rendering
 
The obvious solution is to take the hint and let them suffer
They are not the ones doing you a favour
 
Oh, I have :)
 
Solutions please — Mike C. 46 secs ago
just close and ignore
the problem seems clear to me but it's no MCVE, for the record
 
8:13 PM
Closed. He was so quick with replying with snarky comments, but couldn't be bothered to update the question to describe just what he wanted.
Instead we were supposed to play "One of these things is not like the other" with the two images he posted.
 
I'll take the high road and not delvote the question now :P
 
8:27 PM
@AndrasDeak unfortunately, that intification function is only a placeholder for the MCVE. The actual function is somewhat more complex
 
@inspectorG4dget It was a back-of-my-mind suggestion but I dropped mp a couple of years back. You could chunk the list, then send each chunk to a mp.Process and then stitch them back together. I'm not really convinced this is useful, but you can be explicit in what is copied over to a process (I think)
 
@roganjosh that's absolutely right. I might have to resort to that
 
is anyone here familiar with containerizing a python app? docker specifically - my question has to do with windows vs Linux containers
basically all our development is on windows so I realized that when I export my environment (either from conda or pip) and try to build my container it fails because its including dependencies designed for Linux - however should I switch to windows containers? or just rebuild the environment on a Linux machine?
 
@inspectorG4dget are we absolutely convinced that it can't be vectorized?
 
@inspectorG4dget so much for C in MCVE ;)
 
8:35 PM
@roganjosh actually, that's what I'm trying now
 
That might be the better question IMO
 
@AndrasDeak ah crud! :P I'm actually trying your numpy reader right about now, so lemme see. A problem is that there are empty columns, which numpy can't floatify. a[np.where(a=='')] = None and a = a.astype(float) also fails
 
Then I can pretend to think about it while others solve the issue :P (I do actually start trying to build solutions but I'm not as quick as others in here by a long way)
 
@inspectorG4dget what is an "empty column" in a list of lists?
 
sorry.. a missing value in one of the inner lists, which presents as an empty string a string of length 0
 
8:41 PM
Well, you could convert those to np.nan but I'd be interested if there's a way that you can get out a jagged array without doing a python loop and getting lists at the end
 
are all the lists the same length?
 
yes
import numpy as np


def master():
    L = [list(map(str, range(i,i+3))) for i in range(0,10,3)]
    L[0][1] = ''
    a = np.array(L)
    a[np.where(a=='')] = np.NaN
    a[:,1] = a[:,1].astype(float)
    print(a)

if __name__ == "__main__":
    print('starting')
    master()
    print('done')
$ python multi.py
starting
Traceback (most recent call last):
  File "multi.py", line 14, in <module>
    master()
  File "multi.py", line 9, in master
    a[:,1] = a[:,1].astype(float)
ValueError: could not convert string to float: 'na'
that doesn't seem right
 
are your real strings float-valued? or ints?
or something custom?
 
they're a combination of ints ('5') and floats ('5.0'). But sometimes, the value may be missing, in which case the string will be empty ('')
 
OK
>>> L = [list(map(str, range(i,i+3))) for i in range(0,10,3)]
... L[0][1] = 'nan'
... a = np.array(L, dtype=float)

>>> a
array([[ 0., nan,  2.],
       [ 3.,  4.,  5.],
       [ 6.,  7.,  8.],
       [ 9., 10., 11.]])
 
8:49 PM
That cheats
Wait, hold on. Where does the setup and the solution start in your example, @inspectorG4dget
 
ahh... Hang on, I'll add documentation
 
Can you just change the missing values to 'nan' on creation? Or is there some API?
 
def master():
    L = [list(map(str, range(i,i+3))) for i in range(0,10,3)]
    L[0][1] = ''
    # setup is done. In the wild, L comes from reading a csv file; actually, multiple csv files

    a = np.array(L)
    a[np.where(a=='')] = np.NaN  # trying to nan-ify the missing values
    a[:,1] = a[:,1].astype(float)
    print(a)

if __name__ == "__main__":
    print('starting')
    master()
    print('done')
@AndrasDeak that miiiiight be an option. Lemme see if I can get creative here
 
can you read the csv files directly? Or are there things going on earlier in the pipeline?
 
I wrote the pipeline, so I can mess with it
 
8:56 PM
convert them to floats when you read them? :P
 
trying your pipeline hack
 
I'm not sure "hack" is the right word. But I'm unsure why I'm struggling to see this as a clean 1-liner :/
 
you're right - "hack" isn't what I was looking for; "modification", perhaps
 
fixing a leak? :P
 
:)
 
9:00 PM
that works, too
 
I almost wanna suggest pandas for the conversion to float, which I suspect it can do here. I'm just sticking to numpy for now, though. Tricky to change the dtype at the same time as handling empty strings. I suspect pandas pushed this into cython
 
actually, why didn't I think of that? Them black and white polar bears might in fact be the best solution for this
 
I didn't suggest it on principle
 
Well... there's gonna be overhead
 
9:14 PM
What do you call a class whose sole purpose is to store related static methods and which is not intended for being created instances of?
 
My answer depends on "are you setting up for a pun or is this a serious question?"
 
A module, and ditch the class?
 
In other words, a class which methods could very well just have been functions, but for cohesion reasons, the methods of a related logical entity was put together in a class.
 
yeah, sounds like a module in python
Can you give a single feature of classes that would give you something more than a module would? Or any namespace?
 
in general, instance specific memory - you can have only one instance of a module, whereas you can have multiple instances of a class, each with their own internal namespace and memory
 
9:26 PM
The use case sounds like static methods. No state. If you have multiple function copies with different mutable default args...well, don't
 
@AndrasDeak, Wouldn't it be overkill to call it a module if it's only a minor class with a few static methods?
 
IMO no
 
So there is no limit to how small a module can be? Is that it?
 
come to think of it, you could have multiple such classes in one module, each of which may contain static methods with the same function name
 
A module could be 1 line. You have to make a judgement on structuring the application. When you get to the point of a class of just static methods, I think it either should be refactored or deserves its own module. But that's opinion
What, exactly, do you see as the detriment to splitting this into a module. I'm open to be shot down on this
 
9:35 PM
if the functions are also coupled to other stuff in an existing module it might also make sense to keep them there
 
@roganjosh, What do you mean by "have its own module"? That I should separate the "module" from my application? -- so to have better structure and keep everything neat
 
but if this class is in a module of its own (to be used from elsewhere) then there's no reason to have from util_module import util_class rather than import util_module as utils
there's even SimpleNamespace if you just want to group stuff
 
@SebastianNielsen Yes, that. When I look back at pretty much everything I've ever written, I never think "I should have grouped this all in a module with classes" but, rather, "I wish I'd broken this up more". For sure I'm not authority on this
 
no MCVE, deleted and reposted from earlier stackoverflow.com/questions/59759716/…
 
@AndrasDeak ugh :( Why did they go at it again?
 
9:42 PM
because their question was at -3 and closed, and they are entitled to an answer
 
I forgot that rule. My apologies!
 
Ahh, okay, so the word I was looking for was "utility classes".
 
Andras, Josh: thank you for helping me fix my leak. Pandas to read the csv auto-NaNifies. Then some manipulation for column extraction and df.to_numpy().astype(float). Many thanks, you two
 
you don't need .to_numpy to cast the type but you probably know that
 
I did not know that. Please tell me more
 
hi, could you please tell me where is the mistake
raw_data.loc[(raw_data['FECHA_DEBUT'] == 0), 'FECHA_DEBUT'] = [(raw_data'DEBUT')]
 
Which mistake? You have multiple.
[(raw_data'DEBUT')] should probably be raw_data['DEBUT']
Have you read some tutorials yet?
 
@AndrasDeak ahh thanks
 
@inspectorG4dget You don't need to explicitly access the underlying array (which is.. also already an array :) )
@inspectorG4dget pandas is somewhat wonky in how it deals with dtypes (that's putting it mildly). If you're reading a CSV with a reasonable number of columns, you can specify on reading.
 
9:58 PM
@roganjosh I didn't know that, but I was casting to np.array regardless to (and this is probably also unnecessary) send the whole thing to sklearn
 
@ruben.lfdz one other mistake is that you probably need the same boolean slicing on the right-hand side
 
@inspectorG4dget Provided that you don't have object dtype, I don't think that's necessary. I'm unaware of a dtype that isn't supported by numpy otherwise
 
pandas datetimes are not numpy datetimes
 
The datetime types went through all sorts of loops. I think they're aligned now
 
10:01 PM
We were typing at the same time :) I'm not 100% on that
 
>>> df
0   2015-02-04
1   2016-03-05
dtype: datetime64[ns]

>>> df.to_numpy()
array(['2015-02-04T00:00:00.000000000', '2016-03-05T00:00:00.000000000'],
      dtype='datetime64[ns]')
 
I think they're aligned. Is there a reasonable test for this?
 
you're right
 
Oh good. That's a nice rarity :)
 
it even works with NaT
hmm, perhaps the new integer nan!
>>> ser
0      1
1      2
2    NaN
3      4
dtype: Int64

>>> ser.to_numpy()
array([1, 2, nan, 4], dtype=object)
gotcha
 
10:05 PM
I'm less sure about the full extent of .dt. operations. I've not kept up with all the release notes, so I'm not sure if there's some internal type that needs converting
 
@roganjosh oh thanks. I'll have to look into this some more
 
@AndrasDeak that was one thanks
 
Gonna go try get a game of pool in before bed. rbrb for a bit :)
 
@AndrasDeak working on it
@AndrasDeak both colums are int64
 
10:49 PM
@ruben.lfdz type doesn't matter at this step. Size does.
 
11:05 PM
@AndrasDeak that's the issue, I'm not sure how to sort that as column 'DEBUT' contains some cells with four number as in years and ceros as empty. I want to sort of combine them. thanks a lot
 

« first day (3378 days earlier)      last day (1570 days later) »