« first day (3832 days earlier)      last day (1341 days later) » 
06:00 - 16:0016:00 - 23:00

06:51
Happy duty this morning: I have to announce a performance thank-you bonus for a team that just delivered their MVP on time and to spec.
07:18
Have you guys encountered this oddity: when storing an object that implements __getitem__ in an numpy array (np.array(item)), seems to just loop forever, and ignore the len property. What the fix?
Implementing __getitem__ implicitly makes the instances of a class iterable. I had a similar issue in pyparsing and worked around it by implementing __iter__:
def __iter__(self):
    # must implement __iter__ to override legacy use of sequential access to __getitem__ to
    # iterate over a sequence
    raise TypeError("{} object is not iterable".format(self.__class__.__name__))
Sounds similar to your problem.
Okay cool, I'll need to take a look tomorrow. About to pass out now.
cbg-ning
08:12
Hi, does anyone know how to use `OR` pattern in glob?
For Example, if I have files:
X123foo.csv
X124bar.csv
X125baz.csv

In shell I can write: ll X*@(foo|bar).csv to get filenames with foo and bar, what will be the equivalent in python glob?
that's not supported
Python's fnmatch doesn't support such patterns.
Just do two globs.
I see, well actually I have 4-5 such terms, so I may need to do a loop. Would you suggest looping or shall I go the regex way? Or is there a better library for such task?
AFAIK glob will just get the entire file list and filter it using fnmatch. So if you have lots of patterns, getting the list and filtering it via regex or such would be easier.
Right, will do that. Thanks
08:21
Use pathlib.Path.iterdir (docs.python.org/3/library/…) and do your own filtering with a loop or list comp. pathlib.Path is a great class to learn.
Thanks. I know about iterdir, considering I do need the entire list of files in the memory, there's not much I have to gain by iterdir. Rather I think I will glob the basic pattern and then use list comp with regex to filter. I was just looking for a way to keep a similar search pattern between shell and my program.
pathlib.Path also has glob() and rglob() methods, and will give you Path objects instead of just strs. But do what you wish.
anyone here got googltrans experience? Basically it seems to be removing the first 2 characters from this string '瑛 莉・・・・・・。' Upon translating all I get back is '・・・・・・.' What google translate returns is 'Eri・・・・・・.'
im wondering if its an encoding issue
Hi, I have a small doubt.. I created a csv lookup table to get some string values based on the location index.. but when I get a value from it using logical operators the datatype i get is dtype : object.... I tried many ways to convert it to a string since I want to use it as part of the address to open some other files... How do u go about this
@JasonBrown Can you show us your code?
most of it yes
09:19
MCVE?
I just cant show the file paths
@cmk101010 pandas? strings are dtype object in pandas-speak for all intents and purposes
yes, from pandas...
oh and cbg folks
but i want to store the values as a string
I mean... its just using the basic translate.translate('string',dest='en',src=ja')
09:20
@cmk101010 so then yeah, they're already strings.
just use them.
@JasonBrown ...of which module?
mcve means you make an example code that we can run start to finish, which also shows your issue
ya il see if i can will take me a few minutes though
cool, no rush
09:22
So not googletrans after all nvm, that is the same project
that is googletrans, its just the latest version
3.0.0 doesnt work at all anymore
at least it just errors out for me
well i guess technically its unofficial version...
Alright, that's a no repro from me
>>> translator.translate('瑛 莉・・・・・・。', dest='en', src='ja')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/aran-fey/.local/lib/python3.9/site-packages/googletrans/client.py", line 182, in translate
    data = self._translate(text, dest, src, kwargs)
  File "/home/aran-fey/.local/lib/python3.9/site-packages/googletrans/client.py", line 78, in _translate
    token = self.token_acquirer.do(text)
  File "/home/aran-fey/.local/lib/python3.9/site-packages/googletrans/gtoken.py", line 194, in do
Lol thats why I couldnt run 3.0.0 it errors with nonetype
So... which one are you using? :|
err version 4
give me a sec to get the full version number
version 4.0.0rc1
09:29
Where can I find that / how can I install it?
trouble finding the release for it now... not sure why
maybe they pulled it cause it had issues? is that a thing? :P
pip install googletrans==4.0.0rc1
heres the release for it pypi.org/project/googletrans/4.0.0rc1
having semi issue reproducing it as mcve
different issue as an mcve in that it seems to be ignoring more of the string... lol
>>> translator.translate('瑛 莉・・・・・・。', dest='en', src='ja').text
'莉 ...'
09:35
Hi @ParitoshSingh, I did try it again, but am still getting an error... I'll attach my code below:
place = 'Alice Springs'
location_data = pd.read_csv('C:/Users/CHITHRAL KODAGODA/Desktop/Sem 4/ENGN8601/PythonCode/Lookup_table.csv')
location_data['Station_Number'] = location_data['Station_Number'].astype('str')
station_number = location_data.Station_Number[location_data.Location == place]
tz = location_data.Timezone[location_data.Location == place]
year = 2003
month = 5
file_loc = 'C:/Users/CHITHRAL KODAGODA/Desktop/Sem 4/ENGN8601/PythonCode/DataFiles/BOM/' + station_number + '/' + station_n
it looks like its not even passing the string for translation
the rror code i get is:
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-59-fe241b4d42f6> in <module>
      8 file_loc = 'C:/Users/CHITHRAL KODAGODA/Desktop/Sem 4/ENGN8601/PythonCode/DataFiles/BOM/' + station_number + '/' + station_number + '_' + str(year) + '_' + str(month).zfill(2) + '.csv'
      9 print(file_loc)
---> 10 a = pd.read_csv(file_loc)

D:\Anaconda\lib\site-packages\pandas\io\parsers.py in read_csv(filepath_or_buffer, sep, delimiter, header, names, index_col, u
you've misunderstood something
station_number = location_data.Station_Number[location_data.Location == place] here, station_number contains a string. it's not itself a string
you're dealing with pandas objects, you still have to access the stuff inside
so I have to initialise station_number as a string?
it's like trying to use a list containing your string, as opposed to accessing the string
initialize? no. you're dealing with a Series. (something similar to a list. like a container of items)
you simply need to index it to select an item from it
actual_station_number = station_number.iloc[0] or something along those lines.
09:40
hmm, could i use station_number = station_number.apply(str)?
i'll answer that question with a question. do you understand the difference between a list and a string?
@PaulMcG Nice explanation. A lot of people forget __getitem__'s legacy role in iteration.
yes, list can contain many strings in different indexes
string is a list of characters
I dunno, you wait all month for a star and then two come along together.
@JasonBrown Yeah, tot much I can say here except that that library clearly isn't in a working state yet. Find another one, I guess?
09:43
got any suggestions? heh...
Nope
ya, I havent had much luck finding one
@cmk101010 good. now, i'll build on top of this. a pandas Series is like a list. similar concept.
so, if you had a list, but you wanted a string inside it...would you be attempting to convert the list, or just access the string directly?
access the string directly.... but isnt that what I did with station_number = location_data.Station_Number[location_data.Location == place]
the data is stored as a string,,, and I am accessing it?
hmm ok it seems to only be the latest release, which means they broke something. Thanks Aran
09:46
@cmk101010 nope. this is your confusion. station_number here is still a series. a container. a list-like thing
seems fine in 3.1.0a0
when you use boolean indexing, you don't get a "single" item out in pandas, because your booleans could be True in multiple places
to phrase it differently, location_data.Location == place this line doesn't give a single index to select, it gives a lot of possible indexes to select. this operation would straight up fail in normal lists, this is specific to pandas series.
so basically when using logical operators to pull out things from a datframe... they are taken out as a series object?
so, the result of this operation stays a series
ohh
so its specific to only pandas ... so I get a pandas series object
09:49
yes, let me prepare a different example to show this behaviour
import pandas as pd
data = {"name": ["Ray", "Bob", "Jack"],
        "age": [20, 25, 25]
        }

people_df = pd.DataFrame(data)
print(people_df)

result = people_df[people_df['age'] == 25] # pay attention to this operation.

print(result)
print(type(result))
run this code and see the outputs.
does this help you get a sense of why it can't return a single item?
yes, thanks
great. now i'll take this one step further, and have you take a look at the operation you used for indexing
import pandas as pd
data = {"name": ["Ray", "Bob", "Jack"],
        "age": [20, 25, 25]
        }

people_df = pd.DataFrame(data)
print(people_df)

# result = people_df[people_df['age'] == 25] # pay attention to this operation.

indexer = people_df['age'] == 25 # putting the portion inside the brackets separately
# emphasises this is a two step process
result = people_df[people_df['age'] == 25]

print(indexer)
ok so basically going back to what u said,,, because of the possibility of multiple true values... it stores it as a list type
take a look at this. i took the "indexer" and assigned it separately. you're not indexing a single item, when you do this operation. you're using something called as boolean indexing. you can't do this in python lists.
this hanging } is SO AWFUL. someone should update PEP8 to forbid this abomination
09:57
@ThiefMaster ha. guilty as charged, my IDE spaces it like that :P
for a long time i thought this was the norm. guess not!
alright.. so this is a speciality of pandas?
yes, and of numpy as well
assigning these values as a list into the variable
cool
and a few others. it's great for libraries that deal with higher dimensional arrays, and pandas sits on top of numpy
@PaulMcG did you see the note about the Nutshell?
10:42
there is some error in my code .
df1[4] = df1[4].apply(lambda x: v[-1] if (v := x.split()) else "")
it is on colon syntax error
Upgrade to python... uh... *googles when this abomination was implemented* 3.8?
piR2 must have written that
@ItsOK please don't ask for help with fresh questions on the main site as per our rules
11:27
Hi, i just had a question regarding importing functions from packages.. why don't people just use from <package> import *...
does it consume more cpu by importing all functions?
@cmk101010 no, it clutters the namespace which is bad form
it also makes it difficult to find where a given name came from, and it can lead to clashes if a name is defined in two packages
star imports will make your life miserable. though sometimes you have to be there to truly understand why it was a bad idea
basically due to similar name conflicts between functions of diff packages and also since u can't make abbreviations?
and if you have 1000 names in your global scope and you try tab completion on a variable you're using there's a high chance you'll get a lot of garbage in the completion results
the former primarily. it's an absolute nightmare to debug a script with 10 star imports flying around, no idea where the bad function came from
11:32
for a more jovial reason, see The Zen of Python:
> Namespaces are one honking great idea -- let's do more of those!
to build on my question... then what is the difference between import <package> vs from <package> import *
Everything?
@cmk101010 have you tried playing around with this in an interactive shell?
(sorry for the pings)
and do we need to always have an import <package> command in addition to the importing of additional functions?
I just am using jupyter only
@cmk101010 no
@cmk101010 that works too
You do import foo to have the name foo. If you want the bar in foo.bar you can do from foo import bar. No need to import foo first.
@cmk101010 it might be too technical but you could give docs.python.org/3/reference/… a glimpse. There are some examples with brief explanations for various import statements.
@AndrasDeak so for the case of 'If you want the bar in foo.bar you can do from foo import bar.' ... but if in addition to that u need foo,, then u have to import foo... right?
raf
raf
11:38
I regret asking this question. Instead of asking there, I should have asked for suggestions in chatrooms. I am still trying to improve it. Can you please suggest me what else can I do improve. Due to this one, I am still not allowed to ask new question.
@raf step one: don't onebox it for no reason
@cmk101010 yes. But you can try this yourself
Use jupyter. Use a library you've used before. See what works and what odesn't.
@raf don't you have deleted downvoted questions? It seems weird to me that a single question at -1 with a few non-negatively scored other questions gets you banned
yes,, thats pretty much what i have concluded from trial and error
raf
raf
@AndrasDeak I don't think I have ever deleted any question. :\
@raf weird
raf
raf
btw I have a closed question as u can check
11:42
Already found it. That probably doesn't matter too much.
raf
raf
can you suggest me what to do to get myself out of this banned! If it continues, I guess I need to wait 5 months for each new question! :(
Yeah, I'm not sure. The question is not great, but it's also not terrible. I wouldn't downvote it but I also wouldn't upvote it. It's mostly on topic, and I can't really see what you could improve on it.
raf
raf
@AndrasDeak sorry. what do you mean by the term 'onebox'?
@raf this, when you post a link to a question or answer and it becomes this huge banner
raf
raf
@AndrasDeak ohho I am sorry.
11:46
no worries
raf
raf
Now I am afaird, what if anyone from here see it and put another downvote on it! :|
That's always a risk. Want me to move the message?
raf
raf
@AndrasDeak should I take the risk? :)
the thing about taking risks is that you should decide that for yourself
raf
raf
can you remove the question banner, keeping only the URL?
11:50
nope
raf
raf
if not, you can remove it.
was import * always a bad thing? one high profile library I know that encourages its use is pygame, from pygame.locals import *
there's a couple high profile libraries that encourage the * import behaviour. And i wish they'd stop.
@python_user it took a while for it to be well-known that it's an anti-pattern. For a long time matplotlib's recommendation was from pylab import * which fused numpy and matplotlib.pyplot's namespaces into one mess. And there were some clashes there, like np.hist vs plt.hist I think.
My experience is that it's mostly GUI libraries that do this: pygame, tkinter... because it would be too exhausting for users devs writing the documentation to spell out every name on import
tkinter is an extra sore point because it's right there in the official docs
11:54
just what i was thinking of :P
ohh yeah, I have seen that, and now that you mention it makes sense, I have seen some PyQt tutorials use that as well
In my opinion, the * import has a place, for library internals to clean up their interfaces perhaps. But normal code that's merely using libraries shouldn't need * imports, i can't actually think of a single exception that's good enough. the programmer convenience in this regards doesn't nearly match the downsides of hard to follow code.
Hmm, I don't see np.hist now, so either it was deprecated and removed or I misremember
but, since it's my opinion it doesn't quite amount to much. Just see if it makes sense to you or not.
I dont want to use * imports as well, after like 6 months I am pretty sure I wont remember where the name came from
unless I am working on the same project for 6 months
raf
raf
11:59
@AndrasDeak Thanks :)
no problem
raf
raf
btw, as u have checked that question, what would be ur recommendation? I mean which platform is better for deploying python web apps? so far, I have experienced streamlit only.
I am not sure, but have you checked Heroku and pythonanywhere?
raf
raf
@python_user no I haven't.
I have not used them but I see them mentioned in hosting, Heroku supports hosting of other language apps as well, so it may be a bit more, but there a tutorials out there
12:21
Most recently I used import * in my swf parser project from a couple days ago. The entire file I'm star-importing is a collection of constants like UI1 = "B"; UI2 = "H"; UI4 = "I" and it exists solely because I can never remember the meanings of the struct format characters
Another piece of common wisdom I'm ignoring here is, "if you have variables with identical names except for a digit at the end, consider a list or dict" because I can't touch type square brackets and I have to use these guys ten times a function for a hundred functions, so that's a lot of lost time right there
Define a custom file encoding that changes pairs of some other character to square brackets. ~ maybe?
:-P
although you might actually be using bit operations
I've got tons of bit operations, but come to think of it I'm specifically not using ~ anywhere
there you go
raf
raf
12:27
@python_user have u used any of them?
write a meta script that converts your script into a different script, with all those constants substituted for dict keys with square brackets
because that's how we save time. <insert automation xkcd here>
I don't trust ~ because it involves negative binary and I can never remember how two's complement works
perfect, even more things to debug when it breaks!
@raf like I said, I have not used them, I dont do flask or any web dev out of the dev server
My existing code only works with two's complement in exactly one spot, in the function that pulls N bits (not necessarily a multiple of 8) from the data stream and converts it into a signed integer
It's the one place where I can't just make struct do the work for me, because struct only works on whole bytes
12:42
Hi I had a doubt regarding adding arrays to dataframes..
my initial dataframe is of type: pandas.core.frame.DataFrame
I had a numpy array that I wanted to add..
but when just writing :
df['new column'] = numpy array
I get the error:
<ipython-input-165-40eeaa6c12f7>:14: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead
But when I used:
df = pd.DataFrame(df)
df['new column'] = nnumpy array it workled
which i dont understand why
I'm 100% sure I could replace all of my if signed: block with a single non-bit-arithmetic expression, but I'll let it stay as a monument to my hubris for now
error? looks like warning
yes it was a warning
but I didnt get any warning in the second case
yeah, because the warning related to something that happened earlier ,specifically how the df was created.
which doesnt make sense to me, since my df was already initially y a dataframe
12:45
theres the concept of views vs copies, when you take a small portion of the dataframe.
hmm,,, maybe that happened cos i used some logical operators to filter out some data?
and saved it to the same df
yes. if you try to mutate a view, you get the message. It's usually used to save people from undesirable behaviours, because pandas doesn't take guarantees on whether certain operations make views or copies. having said that, in your case if the result is fine, this is most likely a false positive
I personally don't really worry about it too much, but just understanding when it can occur and when it can cause issues should be good enough for now.
wait i still dont understand why i would get a warning though... since they are of the same lengths,,, i don't see the point of the warning since it should add the columns without any problems right? basically I would expect the warning to be there if the columns are not added properly or adding causes fudging of the data
raf
raf
@python_user oh okay, thank you. seems like I need to ask it on webapps related chatrooms. :)
@cmk101010 Hm, i dont think i'll do a great job at explaining this, but i'll try. the warning is because sometimes the desired behaviour is to modify the original df, and sometimes the desired behaviour is to leave the original df untouched. so, yes, your expectation is correct, it's just the df itself that you're thinking about is not. since you're reassigning the df to the same name, in your case you probably don't care about the original df at all.
12:53
stackoverflow.com/a/20627316/953482 gives df[df['A'] > 2]['B'] = new_val as a (non-MCVE) example of a situation where setting-with-copying produces a surprising result. Perhaps the warning gets raised even in circumstances where it's safe because it would be difficult to determine at runtime whether a particular expression is safe or not
But i'd encourage you to read a resource on this instead.
thanks
Or perhaps it's less about safety and more about the ambiguity of what behavior is actually desired by the user... I don't know, I'm short on practical knowledge. I'll set my link down and back slowly out of the conversation now.
@Kevin you nailed it, this one. pandas can't guess for the user what the intention was
this warning is often just false positives. maybe i haven't picked up some best practices for working with pandas, but this warning hasn't actually helped me yet. Though admittedly, usually when i subset a df, im no longer interested in the original
12:56
I basically just rephrased what you said half a page up. You win this round :-)
Of course we all win every round where we discover a new truth, because the competition is not you vs me, it's us vs the problem (◕‿◕)
6
And that's the truth! insert warm fuzzy feelings here
"And that's the truth!" banned from professional truth-discovering events because it can be chained together indefinitely to produce unlimited wins
And that's the truth!
2+2=4
"2+2=4" is true
"'2+2=4' is true" is true
'"\'2+2=4\' is true" is true' is true
'\'\\\'"\\\\\\\'2+2=4\\\\\\\' is true" is true\\\' is true\' is true' is true
'\'\\\'\\\\\\\'"\\\\\\\\\\\\\\\'2+2=4\\\\\\\\\\\\\\\' is true" is true\\\\\\\' is true\\\' is true\' is true' is true
Haha win counter goes brrrrr
And that's why you don't fool around with recursion limits, kiddos!
13:11
@ParitoshSingh doesn't the warning point to the official docs?
@AndrasDeak i believe so, though in this instance, i think the realpython article is better. (surprising perhaps? oh well)
doubly so because realpython
This reminds me of a problem that I've noticed with Microsoft Teams -- every time I join a meeting, it puts my name in quotes and adds "(Guest)" to the end. After four meetings, my name looks like '\'"\\\'Kevin\\\' (Guest)" (Guest)\' (Guest)' (Guest)
Stellar work there Microsoft
@Kevin oh yeah, I've seen that once, with level 2
it was extra fun because at least one of them was "Gast" thanks to a German locale for the user
Oh, interesting. I guess i18n/localization is one reason they can't just automatically check if the existing name already ends with "(Guest)" -- because it's not always spelled "Guest"
There's probably only a couple dozen variants, so it's tractable algorithm-wise, but off the top of my head I don't know if i18n libs typically let you enumerate all the locale-specific versions of a word
13:20
it would probably be beyond today's technology to keep track of a "guest" flag
They're probably making the guest experience intentionally crummy so I'll get frustrated and make an account. Crummier apps than these have tried, I am a rock
UIs that employ dark patterns automatically unlock my infinite stubbornness mode
Which is usually only seen when I'm reinventing a 17th century math concept, or a CS design that's conveniently available out of the box in 3+ popular libraries
"No, I do not want to use Twisted, I want to write ten different async functions that are all broken in different ways"
if I have a = ['1 2 \n 3 4'] -> how do I convert it into a numpy array?
Greenlets, get ye behind me, I can solve the dining philosopher's problem on my own
@pythonabsir I bet numpy has some kind of cool string-to-array parser, but until someone suggests it, how about numpy.array([[int(x) for x in line.strip().split()] for line in a[0].split("\n")])
I guess you don't really need the strip() call
The expression may look imposing at first but as long as you know list comprehensions, then the construction is fairly elementary
>>> import numpy
>>> a = ['1 2 \n 3 4']
>>> a[0]
'1 2 \n 3 4'
>>> a[0].split("\n")
['1 2 ', ' 3 4']
>>> [line.split() for line in a[0].split("\n")]
[['1', '2'], ['3', '4']]
>>> [[int(x) for x in line.split()] for line in a[0].split("\n")]
[[1, 2], [3, 4]]
>>> numpy.array([[int(x) for x in line.split()] for line in a[0].split("\n")])
array([[1, 2],
       [3, 4]])
Oops, my ancient literature reference was supposed to be "get thee behind me", not "get ye behind me", the latter being technically incorrect even if you consider "greenlets" a plural object because "ye" is the plural subject pronoun and not the object pronoun
13:44
@Kevin Great, used it for my case, thx great.
@Kevin thankfully I asked in the chat
I'm glad it worked out for you :-)
Granted, subject and object are the same here because I am speaking to greenlets and asking them to take an action with themselves as the direct object. But I don't believe the placement is valid. If I wanted to use ye and not thee, I would have to use intransitive get and say "ye get behind me"
Or perhaps "ye, get behind me" to disambiguate the imperative form ("You! Get behind me") vs the continuative/habitual form ("you have a tendency of getting behind me")
Perhaps we can justify backporting the present day vernacular Modern English mechanism of using the subject pronoun in place of the object pronoun, as seen in the meme "get you a man who can do both"
As opposed to the textbook-correct "get yourself [...]"
In [97]: a = ['1 2 \n 3 4']
    ...:
    ...: from io import StringIO
    ...: import numpy as np
    ...:
    ...: b = np.loadtxt(StringIO(a[0]))

In [98]: b
Out[98]:
array([[1., 2.],
       [3., 4.]])
@pythonabsir ^ you have to set the dtype to int if that's what you need
Oh, it hadn't occurred to me that it wouldn't try to guess
I guess there are enough integer dtypes that there wouldn't necessarily be one objective answer to "what dtype should this homogeneously typed data use?"
All of these values fit in a ubyte but problems will arise once you try to add 257
14:02
@AndrasDeak ah, wow, didnt know that.
@Kevin the np.array helper would, but string parser functions don't as far as I know
In [109]: np.array([1, 2, 3]).dtype
Out[109]: dtype('int64')

In [110]: np.array([1, 2, 3.0]).dtype
Out[110]: dtype('float64')

In [111]: np.array([np.uint8(1), np.uint16(2)]).dtype
Out[111]: dtype('uint16')
Curious, I would expect np.array to have less type derivation magic than loadtxt, just based on their goals
No, np.array is actually a high-level helper. The type is called np.ndarray and nobody calls that directly.
Oh, excellent. My main objection was that a class constructor deserves to be as objective as possible, and fuzzyness should be relegated to helpers and wizards. If np.array is not directly a type, then my confusion vanishes
And np.loadtxt has a dtype=float in its signature, so it's not that difficult to switch all values over. Or you could add a converter for each column.
14:12
A text parser is allowed to have a smidgen of magic if it needs to work on human-written input, but a high level helper can have an unlimited amount of magic. You know what you're signing up for so all bets are off
We await the day of the foretold Highest Level Helper, do_what_I_mean()
what is the best way to add a better to answer to duped question 5 years ago?
@python_user answer the dupe target
I thought of that, but the dupe target is way too specific, question in case stackoverflow.com/questions/35189709/…
I am surprised no one mentioned str.swapcase
in that case the dupe target is not good
yeah, that's a bad target
I'm sure there's a better one
Shall I vote to reopen, or should we wait until we have a more canonical target lined up?
14:25
for time being I am just going to leave a comment on the question, so at least someone can benefit
@Kevin nooo
there are a dozen dupe targets
Hmm, can you change a target without reopening and closing? I don't think I've ever actually tried
added the first three hits
@Kevin yes
No one man should have all this power
14:29
I was too busy asking about what we should do once we find a canonical, that I didn't take the time to look for a canonical
thanks, two of those dupe are newer than the question itself :D, but it works
time-backwards dupe targets are fun and good for you ;-)
@python_user all of them seemed to suggest swapcase at a first glance, so this is the right direction
The current day reader doesn't care who first asked about swapping cases, they care about who did the best job asking. (ok, they actually care about who got the best answers, but that correlates significantly with who asked most bestly)
I was like that when I started using SO, then I started to prefer dupes that are new, sometimes they have updated language features
14:34
ideally old targets should have new alternatives
soon there will be dupes with 3.10's match :p
I notice some high rep users editing their old answers to accommodate the changes as in the dict insertion order, that was actually helpful
I salute those users
Myself, I will lie complacently in the center of my domain like a dominant elephant seal, and let the smaller users grab snippets of clout by making constructive edits without my noticing
Seal of approval
15:07
The fat and lazy seals like me and the cunning and efficient seals must act symbiotically to protect the beautiful posts from the new user swarm
Actual on topic question: is time.sleep() noticeably less accurate in multithreaded processes?

long version: sleep()'s actual elapsed time is allowed to be much longer than what you request. In my project's prototype I'm satisfied with its accuracy. I want about 1/60 sec resolution. As I add features to my project, I may need to refactor the time-dependent code into its own thread. Will this make it more likely that a sleep call will take longer to finish? I know the GIL tries to keep things running cooperatively but I don't know how good it is.
im having a very hard time writing to a global variable using Parallel from joblib. Is there something funky that goes into modifying globals when you run parllelized code? Basically I just have this times variable which I wanted to use to do some analysis of certain parts of looping functions without having to change the function signature, but everytime I finish the parallel job my times dict is empty.
Perhaps it matters what the threads are doing, because the GIL can't switch threads if if it's ten calls deep into some non-Python C function. The main thread will be running a Tkinter GUI, and the time-dependent thread will be calling very fast OS API functions.
I'm trying to use ''.join as a binary operator to be used in functools.partial and am reluctant to write def op0(a, b): return ''.join((a, b)) OR lambda a, b: ''.join((a, b)). What I'm most satisfied with at the moment is:
from functools import partial

partial(lambda *a: ' '.join(a), 'Hello')('World', 'o/')

# 'Hello World o/'
is there a functool or something else that does this "starpartial" thing?
@piRSquared str.join
oh, not binary like that
almost, but it takes ... yeah, you see it
15:20
global times
times={}

def f(x):
   st=time.time()
   ###insert code here
   times.update({x:(time.time()-st)})
   return x_out

x_output=Parallel(n_jobs=6)(delayed(f)(x) for x in something)
@piRSquared yeah, defining op0 like that would be silly, because it would be clear
then if you print(times) it still comes out as an empty dict
@AndrasDeak right, and you know how I am. "Yam the future people!"
@Kevin The more threads you have, the less responsive each is. Aside from time slicing possibly blocking your "wake up window" with another thread's work, GIL switching is to some degree random – the more threads there are, the more threads may awake-sleep before the correct one.
@Skyler 1. global times does nothing there. 2. is this multiprocessing?
15:22
NB: The last part may have changed in Py 3.4 (?)
@Skyler too many names are missing there for us to be able to help
try an actual MCVE
@piRSquared Use + instead: reduce(operator.__add__, ['a', 'b', 'c'])
Since reduce is pairwise, there is no advantage from join.
@piRSquared since ''.join takes a single argument, you can't curry it. You can't curry half an argument.
@AndrasDeak yes to multiprocessing, and I guess then we can just insert a sleep call with integer input, the content of the function shouldnt do anything to effect the times dict (im pretty sure if we remove return x_out and gave any value to a list something this code should run as written) if you import the right libraries
ill give it a run in a second
@MisterMiyagi you mean operator.add?
I know the dunders are defined too but I thought they do the same thing as the non-dunders
yeah, it works with add
15:29
@MisterMiyagi I can work with that. This is all silliness as I have a viable solution.
@MisterMiyagi I suspected as much. I was curious if anyone had practical experience and had some ballpark figures I could contemplate. But no matter, I can run some benchmarks myself. Looks like tkinter's mainloop gives the GIL the opportunity to switch 27,000 times a second, which is very promising for hitting my 60 fps target.
@MisterMiyagi you're right, piR2 would never use a non-dunder function. Good catch.
Nice, my coworker and I had a talk about the style guide I made and it went quite well. He agrees on 90% of the things. This was productive :)
@Hakaishin neat. But you didn't say that you were making the style guide.
Even if having N threads reduces reliability to the Nth root, I still might be good, because I need like three threads if I want to get fancy
15:30
@AndrasDeak what do you mean?
at least my impression was that there was this style guide found in the field among burning bushes
Ah no, it was a compilation of things not to do I discovered while refactoring the code
Apr 1 at 9:19, by Hakaishin
@AndrasDeak <3 I am (not) looking forward when I present him our first new company coding style guide. Based on all his bad examples :P But I want to finish the refactor and show that it works, before shooting shots :P
"new company coding style guide" sounds a bit different from "this style guide I compiled based on his mistakes"
but anyway good job communicating it well
well, the company IT untill now is him and me :D Tomorrow a friend of mine is starting so I wanted to finish it and discuss it with him. So this style guide and company style guide, it's kinda the same :P
It may benefit you to tell the new guy that you did find the style guide among burning bushes
No author attribution, it just materialized from the company's collective will
15:32
@Kevin I'm not sure what this means :P
@Kevin haha L'etat c'est moi.
In other words, say "here is the company's style guide" rather than "here is the style guide I wrote", since the first phrasing may lend more legitimacy to it
@Kevin I've found threading performance to be 98% YAGNI. The remaining 2% are YAGDI.
@Kevin ah yes :) And now that we discussed it it is also officially true :)
Of course this is less likely to do anything if the company is just the two of you and the CEO and Carol in payroll
ooh, whats yagdi stand for
15:34
you aint gonna do it? :D
"Who decided that two spaces is insufficient indentation? Was it Carol? I bet it was Carol, she seems like a four space kind of lady"
The style guide doesn't touch on formatting at all :P
Her emails (written in multicolored Comic Sans) certainly go heavy on the decorative whitespace
Your name really does suit you :)
15:36
@Hakaishin Huh!? If not formatting, then what? No splats {**dict((1, 2))}?
Hello everyone, I hope everything is alright with you all. I have been working on a method called Conflation which looks into combining several distributions together (discrete or continuous). I asked the question and got a reply for it, however, I tried to modify the code but I am not getting the same results as the answer. The question is further explained in this discussion: "Conflation Distribution Method"
No triple comprehensions?
"Kevin" comes from the Gaelic(?) "Caomhain", meaning "gentle birth". Mom has never complained about my delivery, but apparently I cried a lot from ages 0-1 and probably onwards
Parts of dirtsimple.org/2004/12/python-is-not-java.html also it's more about good software practices not really style. It's style in so far, that programs that do not adhere to it do still work, but are just a bane to maintain. Thinking about it it should be called Maintainability/Readability guide. Like don't ignore warnings, don't swallow exceptions then just do nothing with them. Don't have 3k loc in one file with dozens of classes. It's more common sense than style related
@AndrasDeak basically the same thing but this mvce runs:
from joblib import Parallel, delayed
import time
global times
times={}

def f(x):
   st=time.time()
   time.sleep(x)
   times.update({x:(time.time()-st)})
   return x

x_output=Parallel(n_jobs=6)(delayed(f)(x) for x in [1.2])
print(x_output)
print(times)
I tried global times in a few places, but I am not too familiar with globals in python
15:44
It's meeting time once again -_-
ModuleNotFoundError: No module named 'joblib'
I can fix that though :P
Oops, let me anonymize my barely-a-secret last name... Ah, I'll do it next meeting
funky, joblib is a pretty standard library to see in conda isnt it
probably, but I'm not using conda
I suspect the main process doesn't do any work
@Skyler Random note: Please use a[b] = c instead of a.update({b:c})
15:46
@Skyler likely culprit: processes don't share memory
@MisterMiyagi i initially did do that actually, I thought I'd change to a function in case there was some kind of weird scope/access issues going on
Try creating your Parallel object with require='sharedmem'
sharedmem does the trick
any major concerns to watch out for when sharing mem between processes?
Pretty sure that actually prevents it from using multiple processes
according to the docs, sharedmem uses threads.
That's likely not what you want.
15:53
If you absolutely need processes, you'll have to have each process return its part of the dict and merge them all at the end
hmm, since this is mainly to be used to examine performance and debug reasons, any simple ways to just print to terminal when you're doing parallel?
it doesnt seem like the functions recognize the standard terminal io when in other processes
A plain ol' print should work just fine, unless the library purposely messes with the processes' stdout
i started exploring this dict angle when print statements werent returning anything
multiprocessing also ships with some processsafe shared memory primitives that should work for this. Not sure if joblib handles that properly, though.
06:00 - 16:0016:00 - 23:00

« first day (3832 days earlier)      last day (1341 days later) »