I have a variable named number_of_occurances_of_highest_frequency_letter which is a number that refers to, in a given piece of text, the number of times that the most-used letter [in said text] is written. That's... descriptive. And a horribly ugly variable name. Any ideas for a better name?
I have a list like [{'a':{'a': 1, 'b': 2}}, {'a':{'b': 2, 'a': 1}}] how can I create a Counter? dicts being mutable I cant seem to use the default Counter
I tried to groupby and then find the length of the result, but I can not sort a list of dicts either
def deep_items(d: dict) -> tuple:
return frozenset((k, v if not isinstance(v, dict) else deep_items(v)) for k, v in d.items())
Counter(map(deep_items, [{'a':{'a': 1, 'b': 2}}, {'a':{'b': 2, 'a': 1}}]))
@Jake Because as you found out a dict cannot be hashed, even if it's nested into something else. A frozenset is hashable and order-independent. Use tuple if you want order to matter.
btw, reading the Haskell from first principles book right now, that ivory towery stuff seems infectuous, feel somewhat triggered by the use of all on int values :-P
I was also thinking about idempotence, but that doesn't exactly apply either
the haskell book is actually quite nice even if you're not going to write much in Haskell, it kind of blows your mind in a nice way, giving a gentle introduction from the basics of lambda calculus up. Currying is actually one of the features I miss the most in Python at the moment
or at least, a decent way to add type hints to curried functions
Ohh i see. Thanks @AndrasDeak . My question is about a research paper related to Meta Learning (more specifically Prototypical Networks for One shot Learning). So I don't think this would be the appropriate place.
Does anyone have a strong suggestion for a multiprocessing wrapper that would handle a dict shared in memory (the dict has 200,000 keys and I want 35 processes on it). I assume that even though the collisions between processes accessing an individual key and modifying the value won't be drastic, all of the processes will lock the whole dict? I could just use stdlib if there are no nice alternatives, I just don't the state of play in this area now
Although, saying that, I wonder if I can get away with SQLite here. I suspect the writes will be too fast for it to handle, though since every process will want to update the data on every iteration
@hugovdberg You can have an unlimited number of readers, but only 1 writer. The writer will acquire a lock and all pending writers will wait for up to 5 seconds (default, but can be changed) before writing. However, a single process atm is shifting 1000 iterations in 30 secs, so I think unleashing 35 of them will hit some serious bottlenecks on the locking mechanism
@AndrasDeak I was hoping to find a wrapper that handled it for me vs. building it all up, basically
I have to get some prelim results out for Wednesday but the current simulation would barely even finish by then, and I still have more dev work to do beforehand :'(
It's just a dictionary of lists, basically. I think that wrapping it up in a server will be quite time consuming vs. using the Manager in multiprocessing. I'll just go with the stdlib implementation if there's no "ZOMG you should definitely be using Wrapper X in 2022"
Interesting, it looks like the Manager launches a server on localhost. I don't think I remember that from all those years ago when I used to play with multiprocessing
Yeah, going with multiprocessing.manager is the way to go. It supports nesting as well, so you can also do CRUD operations on lists values inside the dict. You can probably go with in-memory sqliteDB, but I think manager.dict() would be a better choice, if you have time you can do some perf test against in-memory sqliteDB and manager.dict() for your data.
slots: If true (the default is False), __slots__ attribute will be generated and new class will be returned instead of the original one. If __slots__ is already defined in the class, then TypeError is raised.
New in version 3.10.
Holup just a minute, how does that work? If it creates a subclass, then it'll still inherit the __dict__. If it returns a new class, super() calls won't work. Does it just... create the slots in addition the dict, or what?
At least the data model has it explicitly: "__class__ is an implicit closure reference created by the compiler if any methods in a class body refer to either __class__ or super."
@dataclass(slots=True)
class Foo:
def __str__(self):
return super().__str__()
print(Foo())
# TypeError: super(type, obj): obj must be an instance or subtype of type
seconds = 0
def time_program(timeout, start = 0):
global seconds
main_program_thread = threading.Thread(target=main_program, name="main_program")
main_program_thread.daemon = True
main_program_thread.start()
timeout_seconds = int(timeout)
counter_seconds = start
while counter_seconds < timeout_seconds:
if not main_program_thread.is_alive():
seconds = counter_seconds
sys.exit(1 if args.any_errors else 0)
I have the following code whose purpose is to execute main_program while a timer loop counts and quits if it is exceeded
It was my presumption that if a thread is killed with a SystemExit exception all threads contained therein are killed
Not sure if sys.exit behaves different across different OSes but it seems to me like main_program_thread is still running, or the threads spawned inside of main_program
Is there a way to kill everything in main_thread without killing the entire program
Not entirely sure I understand the setup, but generally speaking: No, you can't stop a thread from the outside. A thread only shuts down once it's done executing its target function. If you want to stop a thread, the thread needs listen to some sort of signal from the outside and shut itself down.