« first day (4125 days earlier)      last day (1048 days later) » 

00:24
Wow, I decided to check pypi for a reverse enumerate and found one uploaded just this year
 
5 hours later…
05:48
I have a variable named number_of_occurances_of_highest_frequency_letter which is a number that refers to, in a given piece of text, the number of times that the most-used letter [in said text] is written. That's... descriptive. And a horribly ugly variable name. Any ideas for a better name?
max_freq
@jeremy Thanks!
 
2 hours later…
08:06
Cbg
08:32
I have a list like [{'a':{'a': 1, 'b': 2}}, {'a':{'b': 2, 'a': 1}}] how can I create a Counter? dicts being mutable I cant seem to use the default Counter
I tried to groupby and then find the length of the result, but I can not sort a list of dicts either
What do you want to count? The occurrences of each top-level dict, or the nested dicts or keys?
the entire dict, this is a simple example, but my actual dicts are nested upto 5 levels
If you're sure the dicts are of consistent order, you can transform their .items to tuples and feed that to Counter.
in this case, both the dicts in the list are equal, so they should be in a group, if groupby worked (assuming I sort)
arent dicts equality un ordered?
{'a':{'a': 1, 'b': 2}} == {'a':{'b': 2, 'a': 1}} the inner dict is not the same order, but returns True
@Jake They are, but the primary alternatives you can dump their content to isn't.
frozenset of the items would be suitable, come to think of it.
08:41
thanks, I will try this
def deep_items(d: dict) -> tuple:
     return frozenset((k, v if not isinstance(v, dict) else deep_items(v)) for k, v in d.items())

Counter(map(deep_items, [{'a':{'a': 1, 'b': 2}}, {'a':{'b': 2, 'a': 1}}]))
I will check if this does what I want, will take me a while to understand thiss lol
why the recursive conversion to frozenset?
@Jake do you want these to be equal or unequal?
ie, is it 2 occurrences of the same thing, or 2 different things with 1 occurrence each
@Jake Because as you found out a dict cannot be hashed, even if it's nested into something else. A frozenset is hashable and order-independent. Use tuple if you want order to matter.
09:03
Is there a name for operations that have the same result whether they're nested or not? For example, all(1, 2, 3) == all(all(1, 2), 3)
something like associativity?
It's technically not associativity, but I might use that anyway because it's the first thing that comes to mind...
btw, reading the Haskell from first principles book right now, that ivory towery stuff seems infectuous, feel somewhat triggered by the use of all on int values :-P
I was also thinking about idempotence, but that doesn't exactly apply either
Hey, interpreting ints as booleans is perfectly fine. Everything in python has a truth value. It's true. :P
perhaps associapotence or idemtivity? :-P
09:10
As long as it's not impotence...
@Aran-Fey Nice monoid!
Argh, stop making me google things so early in the day!
A monoid is...
* A set, S
* An operation, • : S × S → S
* An element of S, e : 1 → S

...satisfying these laws:
* (a • b) • c = a • (b • c), for all a, b and c in S
* e • a = a • e = a, for all a in S
Lifted from here if the rabbit isn't deep enough already.
I haven't reached that part of the Haskell book yet, but it seems the best description indeed
Thanks. I'm still too tired to work out what my or e are, but I'll take the compliment
09:16
haha any operation, e is the identity element for the operation
For your specific case: S are all Python objects, is all, and e is True.
an easier example, take addition, where the operation is + and e is 0, since x + 0 == x, and any int + int returns another int
Ah, that's what you were going for, I see. I was confused because all is technically not a binary operation
nope, but you could see it as a chain of binary and operations
There was a nice* article how one can get variadic and keyword functions with nothing but 1-argument functions, types and currying.
So I wouldn't interpret the "binary operation" thingy too tightly.
09:25
the haskell book is actually quite nice even if you're not going to write much in Haskell, it kind of blows your mind in a nice way, giving a gentle introduction from the basics of lambda calculus up. Currying is actually one of the features I miss the most in Python at the moment
or at least, a decent way to add type hints to curried functions
@ParitoshSingh they should be equal
@MisterMiyagi thanks for the explanation
@MisterMiyagi well why didn't you say so?
@MisterMiyagi Actually, it would be easier to pick e as all(()). 🧐
Associativity with a unit element that's both left and right unit. This is "what is the matrix" all over again.
There's nothing as helpful as being mathematically correct.
09:34
@MisterMiyagi I wonder, how deep does the rabbit go?
Give it enough booze and it will drone on forever.
If you gaze long enough into the rabbit, the rabbit will gaze back into you.
10:06
@Arthi hello. Please don't ask for help with fresh questions on the main site as per our rules.
Hi, is it the right place for asking a question about deep learning topic?
@Pathi_rao hello, as long as it's about python it's technically OK, but we don't have many (any?) machine learning experts visiting here regularly.
Ohh i see. Thanks @AndrasDeak . My question is about a research paper related to Meta Learning (more specifically Prototypical Networks for One shot Learning). So I don't think this would be the appropriate place.
Yeah, I'm afraid not. You could try looking around at chat.stackexchange.com if there's anything ML-related.
Yea i did. The last message was posted a year ago xD. Thanks for the help. Have a nice day
10:13
No worries
 
5 hours later…
15:24
Does anyone have a strong suggestion for a multiprocessing wrapper that would handle a dict shared in memory (the dict has 200,000 keys and I want 35 processes on it). I assume that even though the collisions between processes accessing an individual key and modifying the value won't be drastic, all of the processes will lock the whole dict? I could just use stdlib if there are no nice alternatives, I just don't the state of play in this area now
wasn't that called a database? ;-)
The only database we have access to here is Redshift and that's super slow for stuff like this :'(
I doubt "locking the dict" is a thing even with threading, let alone multiprocessing
Although, saying that, I wonder if I can get away with SQLite here. I suspect the writes will be too fast for it to handle, though since every process will want to update the data on every iteration
But I don't actually know
15:28
I was just about to say SQLite should be able to handle multiple connections, although I'm not sure how it handles locking exactly
I think the SyncManager would do it, Andras?
If you go out of your way to do it, sure. I thought you wanted to avoid that.
@hugovdberg You can have an unlimited number of readers, but only 1 writer. The writer will acquire a lock and all pending writers will wait for up to 5 seconds (default, but can be changed) before writing. However, a single process atm is shifting 1000 iterations in 30 secs, so I think unleashing 35 of them will hit some serious bottlenecks on the locking mechanism
@AndrasDeak I was hoping to find a wrapper that handled it for me vs. building it all up, basically
I have to get some prelim results out for Wednesday but the current simulation would barely even finish by then, and I still have more dev work to do beforehand :'(
what kind of values are you storing? could you create a simple server that holds the dict and provides some GET/PUT method?
It's just a dictionary of lists, basically. I think that wrapping it up in a server will be quite time consuming vs. using the Manager in multiprocessing. I'll just go with the stdlib implementation if there's no "ZOMG you should definitely be using Wrapper X in 2022"
Interesting, it looks like the Manager launches a server on localhost. I don't think I remember that from all those years ago when I used to play with multiprocessing
Jay
Jay
15:46
Yeah, going with multiprocessing.manager is the way to go. It supports nesting as well, so you can also do CRUD operations on lists values inside the dict. You can probably go with in-memory sqliteDB, but I think manager.dict() would be a better choice, if you have time you can do some perf test against in-memory sqliteDB and manager.dict() for your data.
16:51
@roganjosh sounds similar to my issue :P
17:33
slots: If true (the default is False), __slots__ attribute will be generated and new class will be returned instead of the original one. If __slots__ is already defined in the class, then TypeError is raised.

New in version 3.10.
kinda cool
but should be True by default
@Mikhail context?
python's @dataclass
I think __slots__ are roughly at "Giant Space Flea from Nowhere" level for most Python users.
17:51
Holup just a minute, how does that work? If it creates a subclass, then it'll still inherit the __dict__. If it returns a new class, super() calls won't work. Does it just... create the slots in addition the dict, or what?
there goes another few hours of Aran's productivity
@Aran-Fey It throws away the __dict__ class and replaces it with a new one that has __slots__.
So if you have a super() in one of your methods, it'll just break? Because it implicitly references the original class?
I'm gonna need to take a look at the source code when I get home
I don't think super() breaks necessarily, but they would have to adjust the __class__ slot.
Or whatever that thing is called.
18:09
Isn't that implementation detail territory? I've always been afraid of touching those things
At least the data model has it explicitly: "__class__ is an implicit closure reference created by the compiler if any methods in a class body refer to either __class__ or super."
Wow, since when is the code in the stdlib so well documented? This is easily the best code I've seen in the stdlib ever
FWIW, there's a CPython implementation detail right afterwards. So I guess it's safe to say __class__ is not an implementation detail.
Hmm, alright. But modifying a closure variable still seems like a bold move
At a glance, I can't see anything of the sort in the source code though. I gotta run, but I'll test this with super() later
18:45
It would still be OK if it were an implementation detail, because dataclasses is part of the stdlib, right?
19:15
I guess.
@dataclass(slots=True)
class Foo:
    def __str__(self):
        return super().__str__()

print(Foo())
# TypeError: super(type, obj): obj must be an instance or subtype of type
\o/
Are you happy now? :P
Feeling validated in my growing dislike of dataclasses, yep
Plus, it's not every day that you find a bug wontfix in the stdlib with your gut
You could open an issue and probably be told that this is why False is the default
Is it not weird that Python uses bpo instead of github issues?
Not really. You see, that ancient piece of junk deters people from writing too many wontfix reports
Sanity through obscurity, I see.
cls = type(cls)(cls.__name__, cls.__bases__, cls_dict)
This is written that way to keep metaclasses work, right?
19:28
yeah
thanks
 
3 hours later…
22:48
cbg
    seconds = 0
    def time_program(timeout, start = 0):
        global seconds
        main_program_thread = threading.Thread(target=main_program, name="main_program")
        main_program_thread.daemon = True
        main_program_thread.start()

        timeout_seconds = int(timeout)
        counter_seconds = start

        while counter_seconds < timeout_seconds:
            if not main_program_thread.is_alive():
                seconds = counter_seconds
                sys.exit(1 if args.any_errors else 0)
I have the following code whose purpose is to execute main_program while a timer loop counts and quits if it is exceeded
It was my presumption that if a thread is killed with a SystemExit exception all threads contained therein are killed
Not sure if sys.exit behaves different across different OSes but it seems to me like main_program_thread is still running, or the threads spawned inside of main_program
Is there a way to kill everything in main_thread without killing the entire program
23:21
Not entirely sure I understand the setup, but generally speaking: No, you can't stop a thread from the outside. A thread only shuts down once it's done executing its target function. If you want to stop a thread, the thread needs listen to some sort of signal from the outside and shut itself down.

« first day (4125 days earlier)      last day (1048 days later) »