« first day (4112 days earlier)      last day (822 days later) » 

12:02 AM
My position doesn't really include any people management, which I actively avoided. We have Team Leads; nobody wants that from me, myself included. But I'm not supposed to be leading solo projects for multiple customers
 
I somehow ended up in Project Management, then Team Leading, then Team Management and Department Management, with all the former etc... seemed like a good idea at the time but ultimately ended up getting nothing done... (properly anyway)
I'd like to think I'm reasonably competent at 'em but - in hindsight I should have just stuck with being the Senior Developer role - although it's quite useful to see things from the other sides of the development process as well.
 
Yeah, my path is very much blocked from that; I made sure of that. But the idea is that I can jump between projects to support in commercial discussions/shape solutions... yet atm I'm just building the whole thing myself
They have to backfill me for a vehicle routing project for 3 million deliveries a day. That'll be a long time coming...
 
someone like Amazon would surely be just: "3 million? Really... that's just piffling easy...." :p
 
Amazon might have more than one at the helm :P
 
You have my sympathy - I'm currently trying to backfill a potential 3 million customers and 270 million transactions between two systems that really don't want to talk to each other...
 
12:15 AM
All fun, hey?
 
and either of them seem to both time out or throw a hissy fit upon a failed transaction
I've resorted to using redis to keep track of small blocks of transactions (so they can be re-tried later) - and I can keep just reducing the keys down until they're both 0.
 
This is still the postgres work, ultimately?
 
postgres is related - but it's not the one mentioned previously
anyway... I'm going to call it a night... rbrb
 
12:32 AM
Same, rbrb
 
 
3 hours later…
3:27 AM
@MisterMiyagi that's why we have Python. You can't get more upfront about running code.
 
user17921218
4:12 AM
Hi everyone. Is there anyone available to help me with a question? Thank you
 
5:33 AM
@NathanMüller People come and go here - just post your question
 
@PaulMcG Looks like you've been in the movies
 
This morning was the first time I got stuck in my parking space because of snow and ice. Kinda stupid because I missed my meditation, but still cool to have a new life experience. I wish I could see all negative experiences as having this quality of being slightly interesting and novel :P
 
6:15 AM
Is there anyone who have tried Ant Colony Algorithm yet?
 
 
2 hours later…
8:29 AM
@miltonbhowmick Yes.
 
8:42 AM
Some time ago I saw a Python library that allowed querying Pandas dataframes with SQL. So kind of in-memory database. I have vague feeling the name was something "panda" related, but can't figure out the name of the project. Any ideas?
 
Dec 14 '21 at 22:51, by Aran-Fey
There's a pandasql module
perhaps ^
specifically that one seems like an abandoned fork of github.com/yhat/pandasql which is even more abandoned
 
FYI loopy walt's solution to Minimum depth of a ragged list is neat but I can't understand it.
 
Since the first part is python 2 specific, I bet there are shenanigans like comparing ints with lists using <
 
8:57 AM
As for the Python 3 part: int in map(type,a) just checks whether there is any integer at the current level and return True (1) then. The sum(a,[]) flattens the next level.
 
I would try the python 2 version but I don't have it installed :')
 
The ...-~f(... instead of ... 1+f(... exploits that ~a is defined as -(a+1).
As for the Python2 part:
>>> 9<[]
True
>>> []<[]
False
>>> min(1, 2, [], 3)
1
 
Python, ruining code golf since 2012
 
@AndrasDeak Thanks. Finally found what I had in mind, it was DuckDB: duckdb.org/2021/05/14/sql-on-pandas.html
 
So min(a)<[] checks whether there are any integers in the list.
 
9:02 AM
@JuhaPalomäki good to know
 
9:35 AM
Hi
 
@ChristopherMarlowe hello. Please don't ask for help with fresh questions on the main site as per our rules
 
ok sorry.
understood your point
 
10:28 AM
Async is slowly but surely creeping into our company too... Scary
 
10:41 AM
Relax, it's no worse than all the super bad alternatives.
 
 
1 hour later…
11:58 AM
@MisterMiyagi yeah currently we use zeromq and spawn 226 threads :D
@MisterMiyagi I guess that is some solice
 
I have to say async is making me much less uncomfortable with high-concurrency (1000+ tasks) situations. But it's a chore to program...
Still jealous of Golang in that regard. ^^
 
1:02 PM
@Kevin "OK, I shot the screen, but now I'm having trouble reading it."
 
i do have a dataframe which i would like to send it via email and to be displayed friendly to the end user. any suggestion would be appreciated !
 
@αԋɱҽԃαмєяιcαη solution: forget it :D
 
Assuming you just want an image then you could use the styler but you'd either have to take a screenshot at the end or save as a png (which I assume you can do directly, not checked). I imagine HTML in emails would set off alerts
 
1:30 PM
@roganjosh nope, i don't need an image, the brief description of that dataframe is Product Link, Name and Price
so the user can click the link from the email
 
Seems easy enough to make out of <table> and <a> elements
 
1:52 PM
Heya! I am writing a code for which I need the numbers to be exactly equal to the key of the dictionary

for example:

{5: 4, 5.5: 5, 6: 6, 6.5: 7}
This is a sample dictionary

Now, if the user input's a number 5.4325, so my program should assign it 4 points as 5.4325 can be rounded to 5, but if the number was 5.8762, then it should be rounded to 5.5 and assigned 5 points. One way we can do this is by using elif, but I have a lot of dictionaries and want to know if there is a better way of achieving this. Thanks a ton (in advance)!
 
So you want to round down to the nearest half integer?
 
Yess
 
python's round takes a 2nd argument
 
And the dict has a key for every possible half integer?
 
though youd want to accomodate the half int logic
 
1:55 PM
Your sample dict doesn't contain 3, for example. What if the user input is 3.2?
 
It is just a sample dictionary
 
@AndrasDeak Not a Mac user, then?
 
Used as an example
 
It might be easier to multiply everything by 2 and just use integers.
 
^ seconded.
 
1:58 PM
@MisterMiyagi There are multiple dictionaries and they spread across 3 files, so a lot of chaos can happen
 
Proposed approach:
>>> def half_round(x):
...     return round(x*2)/2
...
>>> half_round(2.49)
2.5
>>> half_round(2.51)
2.5
>>> half_round(5.4325)
5.5
>>> half_round(5.8762)
6.0
 
Yeah, if the dict does contain every half-int, then just multiply everything by 2 and turn the dict into a list
 
@Momobear "chaos"?
 
Or, hmm, did I misunderstand the requirements? I just noticed "if the number was 5.8762, then it should be rounded to 5.5"
 
@Kevin In your last sample 5.87 is returned as 6, but what I want is it to be brought down to 5.5
 
2:00 PM
@MisterMiyagi I'd argue that deadlock is a higher-probability circumstance in a cooperative scheduling environment. Apropos nothing in particular, did you know that when threads were invented in the 1980s they were called "lightweight processes"?
 
@Kevin Replace round with int and you should be good
 
@MisterMiyagi Sorry typo 😅😅
 
If you always want to round down and never up, then I agree with Aran-Fey that int should do it
 
@Momobear I wasn't wondering about that, but what you mean by it.
 
@Kevin I lose the half ints like 5.5, 6.5...
 
2:02 PM
>>> def half_truncate(x):
...     return int(x*2)/2
...
>>> [half_truncate(x) for x in [2.49, 2.51, 5.4325, 5.8762]]
[2.0, 2.5, 5.0, 5.5]
One of these days I must learn how to read requirements
 
@MisterMiyagi since i am assigning point on various factor, I am also performing various arithmetic operation on them. That's why
 
@Kevin Would (x >> 1) << 1 be any quicker, I wonder?
 
@holdenweb There's certainly a chance of failing to write cooperative code. I've had to debug quite a few loops that didn't actually yield on a fast path. Yet on the upside, "check that thing where it says await or sprinkle in some" is a pretty good strategy to approach issues that has little equivalent for threading/process, as far as I can tell.
Admittedly also, a lot of the ease depends on the async framework – I've found it much harder to write correct code in asyncio than in trio.
 
@holdenweb My prediction is, "a bit quicker"
 
Or do I mean ((x >> 1) << 1) / 2?
 
2:06 PM
@Kevin This will work. Thanks a lot!!!
 
@MisterMiyagi Right, and of course having umpteen frameworks will be really helpful to learners.
 
@holdenweb Didn't know that! It's always nice to have some historical trivia.
 
@Kevin 😂 Well you got mine right
 
As for skipping the dict step entirely, it should be possible to turn it into a linear equation that can take any numeric input. Start with y = m*x + b and solve for m and b
 
On the bright side, contracts having been signed (after a two-month onboarding delay) I can now tell the world that from tomorrow I'll be a Technical Architect at $Government_Department.
5
 
2:09 PM
@Kevin Is this for me or someone else?
 
@MisterMiyagi Happy to provide it. I was working in technical support at the time for Sun Microsystems. I think C was the first language they were available for (not surprising in a Unix feature).
 
@Momobear For you.
Here is a quick sample of what I mean
 
@holdenweb correct
 
>>> def f(x):
...     return int(2*x - 6)
...
>>> f(5.4325)
4
>>> f(5.8762)
5
>>> f(1001)
1996
>>> f(-123)
-252
 
@Kevin That's a great idea too! Thanks a lot for all the help
 
2:13 PM
@holdenweb $Gov dept being the financial branch or just a well paying branch? What branch I'm curious :P
 
The nice thing here is that you don't have to have an explicit half_truncate call, you can just regular-truncate at the end with int
 
@Kevin Yes, and it surely will remove a few of the dictionaries. I don't why but I don't enjoy using them a lot.
 
$Government_Department being my cute way of not saying, but in fact it's the Department for International Trade.
@Momobear Perhaps because you aren't using them to best advantage? That's not uncommon for people not familiar with the structure.
 
I don't enjoy using any data structure if I can replace it with a O(1) function
 
You got that right.
 
2:16 PM
@holdenweb I guess you are right, I need to study them deeper.
@Kevin I guess then you work in google
🤣🤣🤣🤣
 
Try counting the number of word occurrences (case-insensitive, any definition of "word" you like). That's a good problem to show off why dicts are popular, and allows for some onward learning about more advanced features too.
 
Nah I'm going to topple them as CEO of Kevin's Pretty Good Search
 
@holdenweb I didn't understand the problem statement well enough
@Kevin yeah! you do that!
 
To give an example of holdenweb's exercise: the input "Hello", should give the output {"H": 1, "e": 1, "l": 2, "o": 1}
 
Ahh, now I get it
@Kevin what is your typing speed in programming?
 
2:29 PM
Hard to measure, but I'm guessing... 15 wpm
For regular English sentences I can do about 70 wpm
 
For regular mine clocks near 40 to 43 and does not seem to go any higher than that, and I want it to
Really annoying sometimes
 
@Kevin Ahem: The input "I was not understood, was I" would give (something equal to) {'i': 2, 'was': 2, 'not': 1, 'understood': 1}. But your problem is broadly equivalent, and probably easier as a first task because you don't have to do any word-splitting.
 
There I go reading requirements wrong again
I could try to justify it by saying "my definition of 'word' is just a bit unusual ;-)" but I'm trying to cut down on goalpost-moving this year
 
[Applause]
 
2:58 PM
@holdenweb Uh fancy, that sounds like a lot of fun and exciting work
 
3:55 PM
I was wondering if there was a way to make GitHub ignore the presence of a language while it shows the summary, right now it calculates TCL to be 62 percent and it is shown as my repos main language but TBH it is just a theme file and python is the main file
 
Google led me to this
 
4:33 PM
@holdenweb congratulations!
 
Thanks!
 
@Aran-Fey Seems like I have to upp my google searching skills, thanks :)
 
4:59 PM
If I want to populate a column in a dataframe with the output I get from making a request.get() using something like df['Output'] = df['URL'].apply(lambda x: requests.get(x)) will this work? Or is it better to use a for loop?
 
I'm told that, when it comes to pandas, explicit for loops tend to be slower than the alternative. But the difference is surely minuscule compared to the network I/O that requests does
As for "will this work?", I don't know, but it should be fairly easy for you to test
 
Right was gonna test it in a bit but for some reason it felt weird to do something like it so wanted to see if there was a better way to do it
I normally make requests 1 at a time but I believe you can do it in parallel
 
If you can make async requests then I wouldn't use apply
If it was me doing it (well, how I actually do it when hammering a server async on localhost), I just use requests-futures to generate a list of results in order, outside of pandas, then assign the whole result back to a column. Much faster than trying to iterate the df (which apply will do serially)
 
A potential consideration: what should happen if one of the requests times out, or returns a 404, or raises some other exception?
 
requests-futures is slightly slower than other approaches (IIRC it's like 5% slower than some other approaches I saw benchmarked) but much easier to handle/reason about when I don't really dabble in python async
 
5:10 PM
Right makes sense I have been seeing async requests in my research but it's completely new so i'll have to do some reading
 
If you build a list of futures, you would see the exception and you can just return np.nan in your callback and everything will stay in order
 
What is the desired type of the Output column? Right now it's being filled with request.Response instances
 
That's what you get your callback to do - the response can be parsed async too
I'll see if I can post an overview shortly and rip out the irrelevant parts of the code
 
Mm hmm I was thinking the callback could do it. I'm less interested in "how to make the code do what we want" and more interested in "what do we want the code to do"
 
I just made an example but it would look something like this I think: df['Output'] = df['URL'].apply(lambda x: [i['C'] for i in json.loads(requests.get(x))['A']['B']])
I'm populating a list per cell not sure how i'm going to work with it yet
 
5:22 PM
But that's still using iteration over the pandas df
 
Yeah just showing kevin the desired type of the output column
But will still look into async to do it better
 
5:54 PM
I have a bit of an unusual problem. When the user presses a button, I need to start a thread that loads a file. Depending on the size and format of the file, this can take no time at all or up to half a minute. The user can also press the button again before the thread has finished, in which case I need to start a new thread. Generally I'd like to re-use the previous thread if possible, but is there an easy way to do that?
 
Make it so the user can't press the button again?
 
You mean something like keeping tabs on a group of idle threads? Would you keep them around indefinitely? Or kill them after a while?
 
@vaultah No no no, that's bad UX. Essentially, I'm making an image viewer, and if the user wants to scroll through the images faster than I can load them, that's perfectly fine. I'm not gonna force them to wait until I'm done loading a 50MB gif if they have no intention of looking at it anyway
 
Rough draft: at the start of the program, create a tasks queue, and 4 threads. Each thread calls tasks.get, processes the task, and repeats indefinitely. When the main thread wants a task to be done, it pushes it onto tasks.
 
@AndrasDeak Hmm, I don't think it's worth the effort to maintain a whole pool of threads. Just one idle thread should be enough most of the time
 
6:06 PM
If there are no tasks to be done, the threads block on get, consuming roughly 0 resources
 
What's wrong with just starting new threads? They shouldn't be costly if the button isn't pressed too often, and if it is you need many threads anyway.
 
I too am wondering this
 
Honestly, yeah, I should probably just do that
It just feels... I don't know... wasteful?
I heard starting a thread can be a bit of a slow operation, but it's probably not an issue
 
one man's wasteful is another's "with well-defined responsibilities"
 
@Aran-Fey Clicking a button should be slower. ;)
 
6:18 PM
make the button jitter on the page so it's harder to hit
 
It'll actually be controlled via the keyboard, so I don't think python has enough control over the real world to facilitate that
 
Randomly assign it to a different key!
Pro Tip: Only accept the [start] Button on an X-Box Controller!
That should keep them occupied while your code does its thing...
 
Haha, the power-off button would also be a fun candidate
I see we've tacitly agreed to use the xkcd definition of "random"
 
So i've come across something interesting using aiohttp
async def get(url, session):
    try:
        async with session.get(url=url) as response:
            resp = await response.read()
            print("Successfully got url {} with resp of length {}.".format(url, len(resp)))
    except Exception as e:
        print("Unable to get url {} due to {}.".format(url, e.__class__))


async def main(urls):
    async with aiohttp.ClientSession() as session:
        ret = await asyncio.gather(*[get(url, session) for url in urls])
    print("Finalized all. Return is a list of len {} outputs.".format(len(ret)))
 
6:35 PM
Still waiting for the punchline.
 
.format() looks so weird after having gotten used to f-strings :D
 
@Pherdindy classic
 
It seems strange to me to unpack a list inside gather... I wonder if asyncio.wait would work instead
 
that's exactly what I was thinking
Well, the first part.
 
wait takes an iterable of wossnames, so you wouldn't need to unpack
Disclaimer: I have a lifetime total of one hour's worth of asyncio experience
 
6:51 PM
@ThiefMaster definitely lol f-strings all the way for me
@AndrasDeak well I haven't been doing "that thing" for a while so it's for a different application :D
So in the example it had about 100 urls in the urls variable. So did it simultaneously make 100 requests at the same time?
I haven't really looked into the library yet but it does look to me that it does what I am looking for
 
At the physical layer, I don't think it's possible to simultaneously put more than one packet on the outgoing wire. So that's a bottleneck, albeit a very fast one
If your upload bandwidth is measured in mb/sec or better, you're probably good
 
Some libraries limit the number of simultaneous connections for a session, but I don't think aiohttp is one of them. So yeah, I'm pretty sure it sent all 100 requests at the same time
 
According to the speedtest I have 35 mbps for upload would be interesting to test the library out
@Aran-Fey sweet well i'm feeling optimistic about the discovery xD
 
7:15 PM
A typical HTTP request header is around 200 to 2000 bytes, so you'd need to make 17,500 to 175,000 requests per second to totally clog the outgoing wire
 
8:06 PM
Supposed to be in a meeting with @PaulMcG right now, but I figure his audio is off. Can we all shout? ;-)
 
@PaulMcG Nice, I always wondered why IMDB doesn't have a "what movies were both of these actors in?" search
It seems like it would be a useful tool if you can't remember a movie's name, but remember two actors in it
 
Hi there! I am using opencv-python verison 4.5.5.62 and I use this cv2.KNearest_create() in my code. When I run it I get this: AttributeError: module 'cv2' has no attribute 'KNearest_create'. What should I do? Thanks!
I tried downloading opencv-python 3.5 and tried to use KNearest_create and KNearest but it didn't work
 
8:23 PM
Just before you do cv2.KNearest_create() what do you get from print(cv2.__version__)?
 
I get "4.5.5"
 
And if you run model = cv2.KNearest() in your script?
 
AttributeError: module 'cv2' has no attribute 'KNearest'
 
In which case, have you called a file cv2.py?
 
8:28 PM
What do you see from print(dir(cv2))?
You might want to post in dpaste or something. I'm expecting a long list
 
@PaulMcG Much improved since my first visit. For extra marks, can you add role names to the search? (Users: never contribute, always want more!)
 
I can't send it it's too big
 
How sure are you that KNearest_create really is supposed to be in the cv2 module? Is it in the documentation? Do you have a link?
(Not a trick question, you won't get in trouble if you say "idk I'm just copying it out of this file on github")
Alright, that's close enough for my purposes
 
You don't have to delete your posts so much :-)
 
8:43 PM
@Kevin :)
 
I'm guessing the C* static member function static Ptr< KNearest > create got renamed to KNearest_create when they ported the interface to Python. So the name should be accesssible from the KNearest class' namespace. In other words, try cv2.ml.KNearest_create.
(*C++? I can never tell)
 
2 minute google search led me to cv2.ml.KNearest_create()
 
Me too :-) but I wanted evidence from the documentation, to make sure I wasn't recommending something implementation-dependent or ill-defined etc
 
Thanks
The problem was fixed
I also tried that
but I guess I did some mistake
Thank you again!
 
Ah, the old "fix only works the second time you try it". I know it well
 
8:47 PM
@Kevin :-)
 
This reminds me of my frustrations with tk/tcl, which is the third party library that powers tkinter. You can call many tk/tcl functions from Python, but there is no documentation for them in Python. You have to understand the custom language that tk/tcl uses in order to understand their documentation.
Similarly, in this case it can be hard to tell how to access KNearest_create unless you know what "::" means in C++
And of course it 's impossible to google was impossible to google the last time I did C++ work, 15 years ago
 
 
2 hours later…
10:36 PM
The Gtk python bindings are blowing my mind right now. First I noticed that super().method() doesn't work (crashes with "requires 1 argument but got 0"), which is easy enough to explain - they somehow made all methods static. But how on earth does super().method(self) cause the method to call itself?!
 
Is super still that super?
 
No it's the builtin super
 
that's what I meant with "that" super
 
But not for long... I'm about to replace it with a working version
 
(obviously)
 
 
1 hour later…
11:43 PM
@holdenweb Anything else?
 

« first day (4112 days earlier)      last day (822 days later) »