« first day (4619 days earlier)      last day (330 days later) » 

6:46 AM
I'm curious about the extent to which people have used swagger in their APIs. There was a person in our team that went whole-hog with it in the flask app I was developing, with huge amounts of auto-generated code with custom objects (it was almost akin to its own ORM) and it was totally unworkable for me. In the end we've ripped it all out when he left and now it's a documentation tool with "contracts" that are so weak as to be ignorable
Basically, I just copy/paste into https://editor.swagger.io/ and it gives me nicely formatted input/output. I'm not sure whether the tool was just badly misused on our end of whether the typical swagger experience is that you have custom validator objects everywhere in your code, which is auto-generated, and have some rigid process by which to iterate and upgrade your code
In principle it sounds like it should be a good, useful tool to get frontend/backend on the same page but it was yamming unintelligible to me when trying to push for a release deadline
 
7:06 AM
Is there a built-in way to constantly have a reference point for elapsed time measures ? Most of what I find pointing to something that either is something like or using something like time.perf_counter. - "only the difference between the results of two calls is valid."
 
I put swagger into everything I do.
 
Isn't that exactly what you want? reference_point = time.perf_counter()?
 
@CodyGray-onstrike never doubted that one for a second :D
 
I guess not. What I'm looking for is to save the current-time and check on arbitrary points in my script what time has elapsed without depending on the system time, since thus could change at runtime.
 
time.monotonic(), then
 
7:15 AM
but it states the same, that just the difference between two calls are valid.
 
Why is that a problem? You want to know how much time has elapsed, right?
elapsed_time = time.monotonic() - reference_point
 
To be honest I don't understand what they want to tell me by "The reference point of the returned value is undefined, so that only the difference between the results of two calls is valid." ^^
 
It means that time.monotonic() can't tell you "today is June 9th 2023", it can only tell you "X seconds have passed since reference_point"
 
well that's exactly what I would expect, so the disclaimer looks confusing to me.
anyway, thanks for explaining it to me.
 
8:09 AM
If I have a method called "wait_for_file", would you throw an "TimeoutError" if the file couldn't be found or return a boolean instead?
What error handling is best?
 
a) what does it actually mean to "wait for a file"?
b) consider drawing inspiration from `select()`
 
It should wait until the file appears on the disk AND is released.
 
@Thingamabobs this happens because other people have requirements that differ from yours, and might mistakenly try to use time.monotonic where it isn't appropriate.
 
makes sense. If I wouldn't knew about datetime I might would have been confused the other way around :P
@Warcaith I belief you doing this in a concurrent environment, otherwise in a synchronous scrip you could've made sure the file exists in the first place. But instead of making an maybe wait_for_ever function, why not synchronize the threads instead?
 
Because the product that we're writing tests for doesn't work in that way. I can't predict exactly when some files are created.
 
8:25 AM
@Warcaith so you want to wait for creation of file and wait for say, CLOSE event on said file?
 
@Warcaith When having multiple conditions and outcome, it looks like throwing an error seems more convenient, since you can easily handle multiple exceptions but a boolean tells you only if you had success or not.
 
I'm waiting for the file to appear on disk and also that it has been released by the process that creates the file:


        def wait_for_file(
            self,
            path: StrPath,
            timeout: float
        ) -> bool:
            return wait_for(lambda: bool(self.has_file(path)) and is_file_released(path), timeout)


        def is_file_released(path: StrPath) -> bool:
            try:
                os.rename(path, path)
                return False
            except OSError:
There's perhaps a better way to do it, which is why I'm here! :D
@Thingamabobs Hmm, you're right about that. :)
Would you expect the method to return the file (or the path maybe) instead of a bool?
 
I would expect you to know what you need.
 
:|
 
Sorry, but where should I know what you need for further processing ? I have no clue what the next steps are.
 
8:38 AM
My question was more, if you saw the method "wait_for_file" at first glance, would you expect it to return a boolean or something else?
 
I would probably expect it to block
If based on method name alone I would assume it blocked and did periodic checks for the file to exist
 
@Warcaith you're using the fact that os.rename do not work when the file is opened right? This might not be good if you want crossplatform support, just fyi
 
@roganjosh Yeah, that's what it is doing right now. If it reaches the timeout, it will return "False", else "True.
 
Ok, if there's a hard timeout, I would expect an exception
 
try:
    while wait_for_file():
        print('waiting')
except TimeoutError:
    print('time out, I do something else then or retry')
except RuntimeError:
    print('Something went wrong, need to do something')
 
8:42 AM
@NordineLotfi Using "os.rename" solution works on the platforms we use (Windows, Linux)
 
I see, I tried that once on some Linux distro and it didn't seem to work. I guess it depends on the kernel
 
@NordineLotfi Hmm, that might be the case. It's not an optimal solution, but I'll make sure to investigate that further at another time!
 
yeah, making tests cases might help with that :)
 
@roganjosh What would the return value be do you think? "None"?
@roganjosh And what do you mean by "hard timeout"?
 
9:04 AM
A fundamental concept in python is that the code should be readable as it was written in plain English. There are keywords like wait or has or is and you normally get what you ask for. It all depends on what you want to implement and in what context you embed it. It should be coherent.
 
cbg all
Man, I am really not happy with the code I'm churning out with this release.
 
9:20 AM
How comes?
 
@Warcaith A method like wait_for_file might have a function signature like wait_for_file(file_name, poll_type="exponential_backoff", timeout=30) where it will keep waiting for a file until the 30 seconds is up, polling for the file with exponential backoff but eventually it will just give up. That would be a hard timeout, and that should throw some kind of exception
 
@Thingamabobs I've got a (reasonable) deadline, but I have that major brain itch where I know there's a better way to implement this and know I won't get the time to refactor it when it occurs to me
 
Sometimes the customer pays you to gain experience. You might get a chance to do them better in future request or you are prepared for the next similar request.
 
 
1 hour later…
10:41 AM
A design question over something that's slaughtering me right now:
class Something:

    def __init__(self):
        self.a = None
        self.b = pd.DataFrame()

    def do_something(self):
        self.a = 1
        self.b = pd.DataFrame({'1': ['a', 'b', 'c']})
        self.c = None

my_thing = Something()
print(dir(my_thing))
my_thing.do_something()
There are two things wrong with this code: I hate that attributes can spring into life (i.e. self.c here, which doesn't exist until a mysterious method is run) but the fact that self.b is initialised as an empty DF is causing a lot of pain. Is there an easy path here? I feel like self.b should have its intended type defined from the start
Sorry, there was supposed to be a second print(dir(my_thing)) after the last line
 
Why doesn't self.c get initialised?
(and what is self.b's intended type?)
 
I could always have self.b = None in the __init__ function, which will definitely explode if I tried to use it at the wrong time, but then it communicates nothing to the user about what type self.b is intended to be
@OldTinfoil That's the "rock" in my "between a rock and a hard place"
 
Isn't type hinting enough?

class Something:
    a: Optional[int]
    b: Optional[pd.DataFrame]
    c: Optional[?]

    def __init__(self):
        ...
 
Basically I a) don't want to have attributes spring into existence like self.c does, but at the same time, if I do the opposite and initialise it with the type it's meant to take, like self.b does, it ends up causing chaos because I might be working with self.b as an empty df all along
 
@roganjosh Have you considered slots or do you need dynamic creation of attributes ? Because I really like the fact that you can define slots with a dict to document every attribute.
 
10:52 AM
I haven't, no. Maybe that's the piece of the jigsaw I'm missing...
@Warcaith They're not arguments
 
Why not initialise them all as None? Then you can at least perform simple truthy checks. Mind you, IIRC pandas will rip your arm off if you try to do: if self.b: ...
 
That's the "hard place" of my position :P It definitely will blow up with that if the user is doing something incorrect, but if I am naively wandering the module, I might just do my_thing = Something(); print(dir(my_thing)) and see that there is a "b" attribute but I have no idea now what form it will take because it's just None on instantiation. Perhaps I'm overthinking this, and that's actually just fine
 
In the thingy I'm writing I have decided to use slots on all classes since they are fixed when I'm done and subclassing enables one for dynamic creation of attributes. However, if you do help(Something) your class will be documented as intended.
But I don't know if that is suitable for everyone, since I have never worked in a dev-team that use type-checkers and other stuff I'm unaware of.
 
That's the thing; I don't use any type checkers but I do rely on dir() quite a bit. The fact that the "c" attribute wouldn't appear from the start is something I would find really annoying in my example case, so I think it should be in __init__. But putting it there as None by default, instead of an object of its intended type, is also causing problems
 
11:08 AM
What is the intended type, out of interest?
 
A dataframe
 
I'm assuming it's something custom right now
 
Let me go from an opposite angle:
class Something:

    def __init__(self):
        self.a = None
        self.b = {}
        self.c = []

    def do_something(self):
        self.a = 1
        self.b['hello'] = 'bye'
        self.c.append("cabbage")

my_thing = Something()
print(dir(my_thing))
my_thing.do_something()
print(dir(my_thing))
That's perfectly fine. b and c are initialised with empty containers, they're all defined up-front and we're all happy. It's when the default is an empty pd.DataFrame that things go really bad
 
what about just having the type at __init__ and then overwrite it with the instance later? :/
class Something:

    def __init__(self):
        self.a = None
        self.b = pd.DataFrame

    def do_something(self):
        self.a = 1
        self.b = pd.DataFrame({'1': ['a', 'b', 'c']})
        self.c = None

my_thing = Something()
print(dir(my_thing))
my_thing.do_something()
 
Hmm, that certainly isn't an angle I considered
I might actually just go with that. It ticks the box of exploding if the dataframe isn't populated but it (albeit in a roundabout way) conveys the type. Thanks. It's frustrating how many things you can do with an empty df without knowing it
 
11:23 AM
inserts my ranting and dislike for pandas here
 
The solution, beside type hints, is to write and read documentation. Help instead of dir.
 
What is "documentation"?
 
Something they used to have on Stack Overflow, but got rid of
 
looks aghast in Andras' direction
 
On a serious note, when working in a group project against a deadline, there is nothing even resembling documentation
 
11:34 AM
Can imagine it. In my profession we also just do what is necessary to get the job done with no law suits. There is a amazon mentality out there, pressing a button and things are done. Unfortunately that's not the case.
 
 
3 hours later…
3:01 PM
@NordineLotfi Yeah I've seen reportlab, I read it wasn't so easy to work with. I'm mainly needing to convert from vba to python, not looking to develop anything fancy (or parse pdfs).
 
@toonarmycaptain got you, so this is strictly for converting (I'm assuming) from Excel to pdf?
 
@roganjosh looks to me like a class got used in an unintended way. is there a good reason why do_something isn't part of the __init__?
if there was no way to actually solve the issue, I'd probably try and encapsulate all delayed attributes somehow. maybe just a dict, maybe a property if you want to run code on access (like raise an exception), or maybe a full blown subclass
class Something:
    def __init__(self, x):
        self.x = x
        self.delayed = {}

    def do_something(self):
        self.delayed["a"] = 1
        self.delayed["b"] = pd.DataFrame({'1': ['a', 'b', 'c']})
        self.delayed["c"] = None
 
3:27 PM
I second the idea of a subclass. You don't even have to use it at runtime if you don't want to, you can also just lie to your type checker/IDE:
class Foo:
    bar: Optional[str] = None

    def with_bar(self) -> FooWithBar:
        self.bar = "I'm a string now"
        return self  # type: ignore

class FooWithBar(Foo):
    bar: str

foo = Foo().with_bar()
reveal_type(foo.bar)  # str
 
3:53 PM
@NordineLotfi not Excel, VBA code from Access that already makes pdfs.
 
4:05 PM
@Arne This I find very curious. Yes, there are reasons why it's delayed but would you really use self.delayed = {}?
I'm struggling to find a non-verbose explanation of why I want this
I'd argue that self.delayed = {} makes things worse in my case, because dir() won't list that out for me, and it's really unexpected for me that I would need to find the object state in another dictionary
 
@toonarmycaptain got you. I guess it would help a lot if you could show a small example of what you're doing in vba so I could show a reportlab POC of it :)
 
Let me try put together a better MCVE. I see the holes you've found now
 
Now that I think about it, I don't understand why mypy allows subclasses to change the type annotations. There's nothing safe about this
class Parent:
    foo: Optional[int]

    def reset_foo(self):
        self.foo = None

class Child(Parent):
    foo: int
 
4:28 PM
@NordineLotfi It would, and thanks. I'm not quite there yet. Just exploring packages.
 
4:57 PM
@Arne This is essentially what I'm trying to do here
It's a single plotter interface that will take both un-solved/solved problems and treat them exactly the same way in its method; only the base data needs to change
Actually, I haven't finished that example, sorry
Actually, that's probably enough to see why I have problems with self.new_data = None. It might be None for now, but should it be an empty array?
 
 
3 hours later…
8:06 PM
thinking seriously about this, a couple follow ups:
- (nit) does plotter need to be a class? looks like a function
- does plotter need to treat solved and unsolved scenarios differently, apart from attribute lookup?
I think my hang-up is, if solved and unsolved scenario have the same interface (as far as plotting is concerned), they should be the same class. to the point where you don't know if they are solved or unsolved, and treat an is_solved bool as a leaked internal. but if they have a different interface, they should be different classes
 
@Arne they are the same class?
 
 
Genuinely, I really appreciate the time you take on my problems, but what have we won in that?
I can see that @classmethod is cleaner here, but does it address my original question?
 
by getting rid of the distinction between base_data and new_data, you avoid the problems of one having a not-yet state
 
8:22 PM
Yeah, but that distinction is actually crucial
 
yeah, I was afraid it would be
 
I will give it a go to explain without being too verbose
Having drafted it, perhaps not. I don't want to waste your time. Apologies
 
=D
I don't see it as wasted time, I usually have good fun trying to think about code. But I don't wanna pressure, I'm taking an early rhubarb today anyway
 
I appreciate your input a lot :) rbrb Arne
 

« first day (4619 days earlier)      last day (330 days later) »