« first day (4934 days earlier)      last day (238 days later) » 

05:02
@smci For me, I cannot think of a single thing it would help me with. From being forced to use notebooks, in a browser, using some pseudo-SQL-DataFrame hybrid mess with camelCase syntax, to its own version of git, having to deploy wheels to be able to do imports properly, I could go on...
Some pain points can be reduced, for example soon we will be able to work locally and with real git, and deploy jobs to the spark cluster when needed, which will be a neat feature. However, like I said, I got sqlite to beat its processing (yay, indexes) speed and polars will do larger-than-memory processing much faster for anything I've needed. If we went into billions of rows I might change my tune, but the fact is I'm rarely doing that. And if I was, I'd take redshift any day of the week
I think I fundamentally don't understand the data lakehouse concept. Just because you can store hundreds of parquet files and blobs somewhere and then pretend it's some single SQL entity doesn't mean you should. Like, you're surely making a conscious choice to create all these files rather than using a proper columnar database
05:58
What am I missing with g in Flask here? I don't get why that's an AttributeError: start_time. Looking at dir(g) it looks like a dict (it contains methods like pop and setdefault) but it won't work as a dict, and it doesn't store the attribute I put on it
06:39
I'm guessing that the request context is already gone by the time we get to @after_request, so I can register that with @app.teardown_request but I can't find in the API where I could set g attributes, assuming that @app.before_request suffers the same problem of not being in the request context
 
1 hour later…
07:48
@roganjosh with app.app_context() is unnecessary:
@app.before_request
def log_route_start():
    g.start_time = time.time()
@app.after_request
def log_route_end(response):
    print(f"ended after {time.time() - g.start_time}")
    return response
hmm, now I'm wondering what it was that pushed me to use the context as I was getting errors about being out of context. But indeed, that works, thanks!
Ah, I know. It was when I wasn't using it as a decorator and I ended up conflating the two solutions
 
2 hours later…
10:20
@roganjosh Same, that's what I thought the first time I heard about it. Good to know that my intuition was halfway right
10:30
"halfway right" suggests that I am at least a partial authority on this :P Next week I might embrace the whole notion - "just chunking this down into hundreds of parquet files, because reasons. Gotta get our money's worth out of Spark!"
 
4 hours later…
14:06
@roganjosh I'm playing around with rye at the moment (which uses uv under the hood) and boy does it install packages quickly
Huh, I didn't realise that Armin was so heavily involved in this toolchain
14:21
rye looks pretty cool; I'll have to put some time aside to play with it. Thanks!
14:44
I still find the if let Some(X) construct in Rust so confusing e.g. here. The words of the syntax don't make sense to me
I'm guessing that can't be split into two lines? Like
let Some(sources) = ...
if sources {
Basically, yeah, but I struggle to read it in a control-flow way
 
4 hours later…
18:44
I still don't understand requirements lock files. Are you supposed to commit poetry.lock/requirements.lock or not?
19:28
@Aran-Fey IIRC the point is to have a list of versions the code works with -- because libraries brake code all the time. I'm pretty sure you are meant to commit the .lock so you can say "install the versions in the lock file and the code will 100% work."
Unless you have a different python version or a different OS :D
Touche
Alright, I'll commit them and see if something explodes
20:09
rye is causing more weird problems than I'd hoped :(
not too surprising as it's not as old as Pip. There still some missing features too on Ruff for example

« first day (4934 days earlier)      last day (238 days later) »