« first day (3292 days earlier)      last day (1655 days later) » 

3:00 PM
either way, @rshah, we can probably keep discussing what you actually want do for ages -- which part doesn't work as expected at the moment?
 
Well the code I have there, with your original comment to make the time variable global
That works, but only returns 10 times (once for each client jar run)
But I expect it to re run the clients and get more times until the 10 seconds (or 60 or whatever i set it to) has elapsed
And the program doesnt end either
 
def thread_client(output_dict, client, args):
    while not time_reached:
        time_remaining = get_time_remaining_somehow()
        client = client.popen(args)
        client.wait(time_remaining)
        output_dict[client_key].append(client.stdout.read())
        client.kill()
Something like this
 
and you are sure that starting, waiting and signalling takes less than 10 seconds?
can you please update the gist with your current code?
 
I'm operating under the assumption that it's acceptable if some threads take a little longer to finish, let's say ten seconds plus sixty milliseconds
 
@MisterMiyagi code is updated
 
3:03 PM
When I store a tuple in a python dictionary as a value and print it back, it no longer shows up as a tuple, why is this so?

>>> compatibility = {
10:(13, 14),
13:(16),
}
>>> compatibility
{10: (13, 14), 13: 16}

16 no longer shows up as a tuple
 
Yes its acceptable @Kevin
 
@kauray Common mistake, there: (16) is not a tuple. Try (16,) instead.
>>> x = (16)
>>> y = (16,)
>>> type(x)
<class 'int'>
>>> type(y)
<class 'tuple'>
 
^ beat me to it
 
Oh.....I used to cmake lists like[16], that used to work, so I kinda guessed something similar would
Why does it work with lists
 
because lists have different syntax than tuples
Ultimately [] makes a list and a comma makes a tuple
>>> y = 16,

>>> type(y)
<class 'tuple'>
 
3:06 PM
Oh thanks!
 
@rshah how do you know that each client runs only one process? you are still discarding all but the last one for each client.
 
I dont know that. The client sends one request and when it receives a response it finishes
 
Parentheses in Python have a lot of different jobs to do. They can denote a tuple, or they can dictate order of operations, and a bunch of other stuff on top of that. In order to avoid ambiguity, there has to be different syntax for "a tuple with one element" and "an expression inside parentheses". Brackets, on the other hand, have fewer responsibilities. They aren't used for order of operations, so there's only one possible interpretation for [16] when it's in an expression context
 
but you just said the problem is that the process runs "once for each client jar run"
how do you know it runs only once when you don't know that each client runs only once? oO
 
@rshah I expect popens ought to be a dict of lists, like we decided earlier. I recommend a defaultdict, while you're at it
 
3:09 PM
I know that each client runs once because thats the java program's behavior, the process could probably still be running
 
@AndrasDeak do you want to active pedantic mode? [y/N]
 
@Kevin yeah I can change this too
 
Do we know why Python does that or is it one of those great mysteries of the universe?
 
@Kevin oh i get that, (16) could simply denote an arithmetic expression, hadnt though of that
 
@MisterMiyagi <enter>
 
3:10 PM
That sounds like it @kauray
 
@Simon Everything about Python is, in principle, understandable. But it might not be obvious at first unless you're Dutch.
 
Yam I better change nationality
 
Why can we have tuples as keys in a dictionary but not lists?
 
that's a deceptively simple question ;)
 
Lists are mutable and using a mutable object as a key can lead to surprising behavior
 
3:13 PM
@kauray Because keys must be hashable, and should not be mutable.
 
A dictionary can only operate quickly and correctly if its keys' hashes can't change while it's not looking
 
ah thanks
 
a = 5
{a:5}
 
You can use a mutable hashable object as a key, but if the hash changes when the key mutates, Bad Things happen.
 
Have I misunderstood hash?
 
3:16 PM
@Simon it's hard to tell from that
 
I can define a variable as a key to a dictionary.
 
indeed
 
Yet a variable is mutable
 
Remember that (ordinary) assignment does not mutate. If you do a = 10 after that, the dict won't change.
 
@Simon 5 is not mutable
 
3:18 PM
The object 5 still exists in memory. The fact that the name "a" is no longer bound to it has no bearing on the structure of the dictionary
 
you might be misunderstanding mutability, in which case (re)read nedbatchelder.com/text/names.html
 
sounds like the hashing part isn't the issue here, but your understanding of python variables might be
 
@Simon you can define the value of a variable as a key
 
@AndrasDeak Ah yes, there is the answer
 
Typically, the hash of an object should be based on the value of the object. So if x & y have the same value then hash(x) == hash(y). If hash(x) != hash(y) then when you try to find the x item from the dict using y, it will fail.
> It doesn't make sense for a = 2 to turn the number 1 into the number 2 (that would give any Python programmer way too much power to change the fundamental workings of the universe)
 
3:26 PM
Once I used ctypes to actually find the pointer to the 2 object and change its value to 4, but Python would die almost immediately afterwards, so I couldn't have much fun with it.
 
I don't want to go down that road
 
consider the answer to be "yes"
 
In the ideal case, an object gets collected as soon as nothing is referencing it any more. Some objects will hang around a bit longer, if there are any circular references to the object that aren't actually reachable.
 
just out of interest: is anyone here using PyPy at all?
 
Whether "collected" means "memory is deallocated and its contents are zeroed out" is probably an implementation detail
 
3:31 PM
@MisterMiyagi Installing or uploading?
 
None of this applies to smallish ints, of course, since they're interned. They stay alive indefinitely.
 
Interesting
 
@Simon pardon?
 
Are you trying to publish code to Pypy or install something
 
PyPy, not PyPI
 
3:32 PM
I'm going to make a real math library named "pyπ" just to confuse matters even further
 
This is a sign I need a break. Rhubarb all
 
"I used pypi to install py{GREEK SMALL LETTER PI} on my PyPy distribution"
 
so if you export the value of py, you'd have pyπ.pi in the end...
please go ahead
 
Incidentally I am somewhat baffled that I can't copy-paste the pi symbol while preserving the nice serif on either side
I can see the serifed pi in MisterMiyagi's post, but if I copy-paste it, mine looks like a staple
Not sure how it looks on other systems, but this is what I see
 
same
 
3:42 PM
hands @Kevin a backtick
 
we're not worthy of using such symbols. Mr. Miyagi has perfected his art over years of training.
 
wim
π and π is same just one in codeblock
 
Ah, so it's a matter of fonts? Let's see: π
 
pyπ.pi in
 
I just did a copy from Mister Miyagi. I see serifs in the edit window: pyπ... And it's still looking good.
 
3:44 PM
Ok, that explains that. It's strange to me that the regular font would have a less fancy glyph.
@PM2Ring Looks like a staple to me.
 
wim
no it's the other way round, the codeblock one is sans serif
 
@Kevin it's not typeable though. Go for: import ₱ɣᴨ
 
wim
how does uppercase look ... Π (code) Π (plain)
 
that's just some side product
 
user11867329
Someone got experience with python applications on a KUSANAGI server?
 
3:47 PM
@Kevin Interesting! I blame Windows "creative" font handling. BTW, upper-case Pi has no serifs. ∏ Kevin'd by the badger.
@AndrasDeak I see what you did there.
 
thanks ;)
I'm sorry I couldn't make something more substantial out of it
 
Cabbage :)
 
cbg
 
I'm 100% prepared to blame Windows for all of this
 
@Kevin no, that's how it is for me too
sans in PM's message, sans in wim's message
 
3:53 PM
Ok, so uppercase code, uppercase plain, and lowercase plain are all sans serif, and only lowercase code is serifed? And PM's "∏" is actually N-ARY PRODUCT so we can ignore that...
 
well product is capital sigma, traditionally...
 
Incidentally I would like to thank 3.8 for introducing math.prod, which until now I had to implement myself once every six months or so
 
ah, about time
 
Now for completion's sake we need equivalents for exponentiation and tetration
 
math.uparrow
actually, that's just the binary op
 
3:59 PM
docs.python.org/3/library/logging.html#logging.Logger.setLevel this says that When a logger is created, the level is set to NOTSET (which causes all messages to be processed when the logger is the root logger, or delegation to the parent when the logger is a non-root logger).
But when I tested the new logger gets created with level 30 which is WARNING
 
Is it possible that your new logger is not the root logger, and is secretly inheriting from the real root logger?
I have no idea how you'd verify this either way, but <shrug>
 
I did logger = logging.getLogger(name) in a fresh empty projecr
and then print(logger.getEffectiveLevel())
 
"If a value other than NOTSET has been set using setLevel(), it is returned. Otherwise, the hierarchy is traversed towards the root until a value other than NOTSET is found, and that value is returned."
root is always at WARNING by default
logger.level should be NOTSET
 
I was concerned why documentation says that When a logger is created, the level is set to NOTSET
Another question I have is: I have read about logging in library- that we can use the Null handler. But I am interested to know- how does null handler at child logger prevent root logger from displaying logs
 
@variable because that is what happens.
 
4:12 PM
As the log are propagated to the root handler which by default uses lastResort handler
So does Null handler automatically stop propagation?
 
did you try?
 
I just want to be sure that I have understood it right
 
wim
The part about adding a null handler is to prevent spammy output like No handlers could be found for logger "myapp" in the case that someone logged an event before configuring the logging system
it's not a very good advice, in my opinion, and there are two sensible defaults for such a case
1. just drop the log event
2. log the event to stdout
stdlib logging will do 1 for debug or info events anyway (since root logger is at warning level)
for warning or above it will print the "no handlers" thing to stderr, which is kind of annoying (and it's why they advise to add the null handler thing in your library)
the dilemma is only appearing when other party's code uses your library as a dependency and they have neglected to configure logging in their app. it's not really an issue for your own code because you are in control of your own app and should configure your loggers at your app's entry-points.
 
4:30 PM
When a module A is imported in module B, and say the module A has code logging.getLogger() then the default logger varies based on whether you run the module B or module A directlt
Root logger*
 
wim
logging.getLogger() is the root logger
usually you would do logger = logging.getLogger(__name__) so you get one logger per module
 
"How can I configure the behavior of any logging that occurs before I can configure logging behavior?" seems like a real walls-n-ladders problem
 
wim
@Kevin It's more a question like "what should the logging system do when it receives an event which is above propagation level but it has not been defined yet where the events should go?"
and it is up to the logging framework itself to make that decision.
stdlib does this weird halfway house thing (complain to stderr for WARNING or above)
 
Yeah.
 
wim
better frameworks make, err, better decisions
probably people in this room know I'm a big fan of structlog, the default behaviour for structlog is to just auto-configure the logging system with some sensible defaults - namely a PrintLogger streaming to stdout (solution 2.)
 
4:33 PM
I just use print :3
'Course, the vast majority of my programs are executed on a computer with a monitor, while I personally watch stdout. If I was deploying to an embedded system or something, I'd crack open logging's documentation
There is a case to be made for "always use logging, even for applications which you can monitor yourself" but I need to keep my minimal viable prototypes minimal or else my attention span will run out 95% of the way through
 
@wim every time you say "I'm a big fan of <library for X>" I take a look at it and immediately feel the need to replace the duct tape we've used beforehand. Please write more "I'm a big fan of <library for X>" messages.
 
stackoverflow.com/questions/58490557/… too broad. Too much wrong with the code
 
Hi guys
Is this the correct way to apply a function to a dataframe series?
def currency_conversion_function(dataframe):
    if dataframe['money_currency'] == 'NZD':
        dataframe['money_Spent'] = dataframe['money_Spent'] * 0.57
    elif dataframe['money_currency'] == 'AUD' or dataframe['money_currency'] == '$ AUD':
        dataframe['money_Spent'] = dataframe['money_Spent'] * 0.62
    elif dataframe['money_currency'] == 'USD':
        dataframe['money_Spent'] = dataframe['money_Spent'] * 0.90
    elif dataframe['money_currency'] == 'GBP':
        dataframe['money_Spent'] = dataframe['money_Spent'] * 1.16
 
4:49 PM
that's technically correct but needlessly verbose
 
I know very little about dataframes, but I know when you have a lot of elifs with near-identical structure, you can usually replace them with a dict
 
wim
@MisterMiyagi will do - that was such a genuine compliment, thank you for the kind words
 
How can I improve it?
 
rates = {'NZD': 0.57, 'AUD': 0.62}  # etc.
dataframe['money_currency'] *= rates[currency_code]
 
Yes, or perhaps dataframe['money_currency'] *= rates.get(currency_code,1) if it's possible that the currency type is not present in rates and you want the fallback behavior to be "don't modify the value"
 
4:52 PM
yup
oops, I messed up both sides, sorry
rates = {'NZD': 0.57, 'AUD': 0.62}  # etc.
dataframe['money_Spent'] *= rates[dataframe['money_currency']]
there
 
Cool I got confused for a moment , thanks a lot ! @AndrasDeak@Kevin
 
But, uh, this is starting to look weird. The currency looks like a scalar. Does your dataframe have only one row? That looks off.
 
@roganjosh Well, it's mostly the same error repeated multiple times. A good answer needs to explain what the OP is doing wrong; a "Try this:" code dump isn't sufficient.
 
No it doesn't @AndrasDeak
 
I was just about to say. Does dataframe[key] usually return an object that can be compared to a string?
 
4:54 PM
@PM2Ring at least one other major issue is the implicit multiplication from brackets
 
@Kevin it can, but the result is a series, and when you put it in the if the implicit bool call will shout and raise
 
Also, I can't test right now, but I don't think you can use *= some_dict
I could well be wrong, but I think it requires apply
 
I guess RaphX wants row-by-row to use the proper multiplier
 
Yeah basically I have rows having different currencies, I want to convert them all to one currency using the appropriate multipler(in this case all other currencies to EUR)
 
@roganjosh Ah, right. I missed that on my first glance. I guess it should be closed as too broad. But I feel sorry for the poor OP. From the over-abundance of functions, I suspect that the professor is teaching Java disguised as Python.
 
4:59 PM
one pedestrian option is to use a list like conversion_rates = [rates[val] for val in dataframe['money_currency']], after which dataframe['money_Spent'] *= conversion_rates will probably work
a fancier option is to use a categorical dataframe that can be indexed directly with dataframe['money_currency']...this is left as an exercise to the reader
 
pandas devs, please implement the method dataframe.rowwise_map, thanks
 
@PM2Ring I can understand the sentiment :) I think 50% of my motivation comes from being irked by people stabbing at one bit they recognise as wrong and just ignoring the rest
 
@Kevin df.apply might do that
 
It does
 
pandas devs, please create an alias for apply named rowwise_map, thanks
 
5:03 PM
df['x'] = df['x'].apply( lambda y: y * some_dict[y])
 
But that's just a wrapper for a loop... whereas a vectorized lookup thingy would be faster
 
Its not been clear to me why dict lookups couldn't be done in parallel. I'm not sure they can ever be vectorized, but I imagine they could be done in parallel
 
I nominate the GIL as scapegoat
I'll hoist the GIL-shaped piñata, you guys get the sticks.
 
My understanding is that vectorization requires a single instruction set to be loaded into the CPU and I'm not sure if a dict lookup can fit that bill. Then again, maybe I've just invented the whole back-excuse for why it's not a thing
 
@roganjosh Yeah. It's the blind trying to lead the blind, and they aren't trying very hard. I was hoping that those answerers would improve their answers, or that a new kid would post something better. But I've now given up hope, and just close-voted.
@roganjosh Vectorization also requires the data to be in a contiguous array. You can't do that with a hash table.
 
5:10 PM
cbg
i wanted to say, series map looks useful here too
Though now with roganjosh's comment, im guessing series map is internally just a simple loop, similar to apply but specific to dict lookups
 
@ParitoshSingh ah, yeah, that's it
 
They're basically the same
 
at least it's not apply and it doesn't use a lambda
 
Map will still do the dict lookup but not allow the transformation part (*=)
 
Frank has now added some explanation, but it's not very helpful. Oh well.
 
5:13 PM
df['money_Spent'] *= df['money_currency'].map(rates) should probably do it
 
yeah, with the map, i'd opt for creating the array by mapping currency to the "multiplication factor"
AD nailed it
 
I suspect they run at about the same time
I would guess the python loop itself is dominant in the runtime. I'll have to play about when I get home cos I'm curious now :)
df['money_currency'].map(rates) makes me think it's just gonna recreate the whole series in a loop and the inplace multiplication after that amounts to very little
 
@roganjosh and I wouldn't be surprised if it could use compiled code under the hood, since it knows what types to expect
 
That's a good point. I think it will only be settled by timings :)
 
yup
let me know how it goes
 
5:19 PM
Twist: Raph reveals that his dataframe is only 20 rows long and so all of the proposed approaches are <1ms and therefore equally suitable
 
<suspicion that Andras knows the result increases>
 
"Nevermind guys, I decided to just use the csv module instead"
 
@roganjosh nah, I have no idea
 
i just realised the apply is weird. How would i be writing this with an apply in your mind roganjosh?
 
@ParitoshSingh what is "this"? I thought you were going to post something new. I have an apply approach (untested) earlier
 
5:23 PM
er, simply put. i'm either too tired or too out of touch to write this using apply without effectively writing it in two steps: create a new series that's effectively a map of money_currency. then multiply that series output with money_spent But it sounds like you're imagining it in 1 go
 
20 mins ago, by roganjosh
df['x'] = df['x'].apply( lambda y: y * some_dict[y])
 
the 'x' has to be money_currency right? which makes the lambda part stop making sense for me
 
Aha, ok
 
rates = {'NZD': 0.57, 'AUD': 0.62}


n = 1000

import pandas as pd
df = pd.DataFrame({'money_currency': ['NZD', 'AUD'] * n,
                   'money_spent': [5000, 4000] * n
                   }
                  )

temp = df['money_currency'].apply(lambda y: y * rates[y]) #TypeError: can't multiply sequence by non-int of type 'float'
if im assuming what you wanted correctly, we'd need some kind of apply on two series at once
 
I'll be home shortly. You can have multiple columns in your apply, you can transform a column in place, or create a new column
 
5:26 PM
Ok cool, sorry didn't mean to rush you or anything. Im just realising how out of touch im getting with code. It's been a nasty month
 
So yeah, the x on both sides of the assignment doesn't make sense in my example, sorry. You'd have the float column that you want to change,and another column with the currency string
No need to apologise, my example was a bit crap but it takes some fighting with my phone to post
 
user11867329
Is running "sudo apt-get upgrade" on a fresh Kali Linux proper procedure?
 
user11867329
Or will that affect system stability
 
#Setup:
import pandas as pd
rates = {'NZD': 0.57, 'AUD': 0.62}
n = 10000
df = pd.DataFrame({'money_currency': ['NZD', 'AUD'] * n,
                   'money_spent': [5000, 4000] * n
                   }
                  )
temp = df.apply(lambda x: x['money_spent']* rates[x['money_currency']], axis=1)
temp = df['money_spent'] * df['money_currency'].map(rates)

#Output:
%timeit temp = df.apply(lambda x: x['money_spent']* rates[x['money_currency']], axis=1)
429 ms ± 51 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)
 
@OTLT-LCar not a python problem
And kali linux is not a general-use OS
 
5:32 PM
One thing i didn't use is inplace multiplication/assignment. But it seems like there's a huge difference, assuming i did the apply part correctly.
%timeit temp = df['money_spent'] * df['money_currency'].apply(lambda x: rates[x])
4.38 ms ± 78.6 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)
For a more apples-to-apples comparison.
 
user11867329
@AndrasDeak I got $5 on python being the prominent scripting language used on Kali, but I agree it's not generally linked.

Also, what's it not being general-use have to do with it?
 
@OTLT-LCar that people asking about it usually shouldn't be using it ;)
 
user11867329
;) ;) ;) maybe you should stop hanging with unethical people then.
 
user11867329
I need to secure my new work's infrastructure
 
user11867329
and was simply curious
 
user11867329
5:37 PM
about the HIGH amount of upgradeable packages that comes with the latest Kali
 
user11867329
but all I get is a guy winking at me
 
Probably not going to be much help to be had on kali stuff, here in the Python room
 
user11867329
That, I assumed otherwise.
 
surprisingly, we're not penetration testers
 
user11867329
Me neither.
 
user11867329
5:42 PM
Standstill I guess
 
We're a multitalented bunch. Sometimes you can find an expert of an unusual topic in here. But sometimes not.
Try again in 23:59:59, or insert ten gems to try again now
 
user11867329
K but, lol, logically, like, from your experience.
 
user11867329
inserts 10 gems
 
I installed Ubunutu once ten years ago, and in my opinion you should not run sudo apt-get upgrade on your fresh Kali Linux
 
user11867329
Hmm, makes sense, it's like that gentoo I had to setup in the 80ies to play CoD
 
5:47 PM
Thanks everyone for the different suggestions @AndrasDeak@Kevin@roganjosh
 
user11867329
but I needed more pylons
 
user11867329
so the things didnt spawn
 
Always construct additional pylons.
 
user11867329
Depends, my first one was for a sneaky thing
 
user11867329
but got spotted
 
user11867329
5:49 PM
so... yeah, not always
 
@RaphX I'm currently trying to build the test case for speed. You're welcome; it's triggered some intrigue in my to find out what's fastest here
 
user11867329
FYI: Kali packages should be upgraded.
 
user11867329
Was wondering if it was like Tails
 
user11867329
apparently not, a little more like Knuckles
 
user11867329
Ok, now that that's fixed. How do I python the whole internet?
 
5:51 PM
very slowly, most likely
 
You can't without violating a lot of robots.txts, and that would be wrong
 
@ParitoshSingh Now that I see this on a proper monitor and not a phone screen, you've categorically answered it. Thanks mate.
 
user11867329
@Kevin, I think the first time I heard robots.txt was in HTTrack > Unchecking "Follow Robots.txt"
 
user11867329
That still a thing?
 
i think with a different n it may be worth exploring what happens. But best i can tell, the dataframe level apply absolutely obliterates anything else and overshadows the timeit test on its own
a series apply could possibly be improved upon by avoiding the lambda perhaps, but at that point, you might as well use the map
 
5:54 PM
I'm not sure now, though, how I can push map into a loopy corner
 
If you can generate the Series with map and then throw it away like that, I am utterly defeated :)
 
user11867329
@Kevin

sudo accidentally the whole internet
 
AAB
Hi,
 
user11867329
h
 
AAB
5:57 PM
Does python have any equivalent of C++ sort(arr.begin(), arr.end()) ?
I know about list.sort()
& sorted which return a new list
 
user11867329
Press "A"
 
user11867329
(amidoinitrite)
 
it would help if you could tell us what arr.begin() and arr.end() and sort(arr.begin(), arr.end()) do
 
AAB
say If I only want to sort the first 5 elements in an array how to do so in python
@Aran-Fey hi
 
Take a slice of the list
 
AAB
5:59 PM
arr.begin & arr.end are the starting and ending index
 
my_list[:5] = sorted(my_list[:5])
 
AAB
Won't that create a new list in the process?
 
it will.
 
What is your actual concern here?
 

« first day (3292 days earlier)      last day (1655 days later) »