« first day (4835 days earlier)      last day (338 days later) » 

05:24
@roganjosh yeah, notebook in general are good for prototyping or concept, but when you need serious usage, it's better to use normal code. I don't know why so many companies in datascience field depend on notebook even when it is impractical
@roganjosh don't know why either, but another possible culprit could be the cdn akamai
 
5 hours later…
10:32
I am not able to solve this despite of searching different stack-overflow thread , can anyone help me resolve this issue ?
`Welcome to Ubuntu 22.04.3 LTS (GNU/Linux 5.15.122-tegra aarch64)

* Documentation: https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support: https://ubuntu.com/advantage

This system has been minimized by removing packages and content that are
not required on a system that users do not log into.

To restore this content, you can run the 'unminimize' command.
11:07
@AnilSarode hello. In future, please follow our room rules for posting large blocks of text and do that off-site, linking back to it here
It's not clear to me what you actually did to trigger that error
11:31
I have an async test case in pytest where a background task is running throughout the whole thing. (Basically I'm testing my web server, so a web server must be running throughout the test.) So I basically have a asyncio.create_task(server.serve_forever()) in the test. My problem is that the server sometimes crashes, leading to a deadlock that's annoying to debug. Is there any way to make this crash obvious?
Printing doesn't work because vscode doesn't display the output, and pytest.fail doesn't seem to work inside of a Task
Turns out Task.cancel takes a message as input, that pretty much solves my problem
test_task = asyncio.current_task()
assert test_task is not None

async def serve():
    try:
        await server.serve_forever()
    except Exception as error:
        test_task.cancel(f"Exception in Server: {error}")

asyncio.create_task(serve())
12:18
@AnilSarode this is too large a block of texts. Feel free to use pastebin or gist.github to upload your code/error/etc so this is easier to read.
@AnilSarode I didn't ask you to apologise for the long code snippets, I told you exactly what we expect in the room rules going forwards
"I didn't ask you to apologize" comes off as pretty rude IMO
That's fair, but so does completely ignoring my initial suggestion, to just break the rules again on the very next post. I apologise for being terse
@NordineLotfi @roganjosh here is the link for pastebin : https://pastebin.com/LULAbjjc . I am getting this error and I tried to solve using this link https://stackoverflow.com/questions/49347726/virtualenvwrapper-initialization-error

and this too: https://stackoverflow.com/questions/29486113/problems-with-python-and-virtualenvwrapper-after-updating-no-module-named-virtu

but could not resolved it.
Thank you for posting the links, and apologies if I was rude earlier. Your ping came through while I'm working on some nonsensical error and I could have worded my response better
12:31
@AnilSarode I think you're focusing your attention on the wrong thing. It looks to me like the virtualenv error is caused by the previous error (the OSError)
@roganjosh no you were not rude and no need to be sorry, It was my mistake.
12:54
Turns out that the nonsensical error comes from polars. Converting this particular file from CSV to parquet in polars then throws a Java IndexOutOfBoundsException once you try query it in databricks. Doing the exact same conversion in pandas instead works just fine once in databricks. Also, the file is 2MB smaller somehow. Alas, 297K rows to the file and not a single clue on what the issue could be
don't know about neither databricks or polars, but maybe what's happening here is because of a possible formatting issue on the CSV. At some point you get an error and the output being 2MB might be because of what little it managed to output before the error happened
It's the same base CSV file in both cases, and both work perfectly fine inside python itself i.e. I can read both generated parquet files in both pandas and polars. It's only once I upload them to databricks that there are issues with the parquet file generated by polars
IndexOutOfBoundsException coming from select max(some_column_name) from file_name is a weird one
13:39
@Aran-Fey yes even I did simple Python3 on the terminal, I am getting following error :

`Error in sitecustomize; set PYTHONVERBOSE for traceback:
OSError: [Errno 74] Bad message: '/usr/lib/python3/dist-packages'`
 
2 hours later…
16:00
@roganjosh spark uses arrow as its parquet backend, and polars might use something else, like fastparquet, which may produce parquet that spark can't parse
I'm surprised because every other CSV that I've converted to parquet via polars works fine, so I think it's something specific to this file. Spark will actually tell me exactly how many rows are in the file, it just blows up if you run a specific query on this one file
It'll also tell me the headers. So it can get through part of it
16:31
Newline, carriage return or similar junk in string field?
It could be any of those things. The fact that it's interoperable between pandas and polars in python proper is the real kicker. The fact that the polars-generated parquet runs fine when reading back into pandas is confusing. It's something very specific that only spark is taking issue with. I might have a poke around over the weekend
I have a feeling it'll be something esoteric and beyond my understanding like big-endian/little-endian that Java finds distasteful. It can read the metadata of the file, clearly, to get the row numbers and the headers, but gets out of bounds errors on a query. Perhaps I won't get my first PR to polars from this one :(
@Arne also, just on that one point, polars is fully backed by arrow
16:47
Apparently Python is #1 in the TIOBE Jan 2024 Index.
17:22
@NordineLotfi The issue is with my network carrier. I run permanently on 4G at home, specifically through Vodafone. I have both laptops running right now, one on my router and one tethered to my phone. The one on my phone (O2) loads everything fine, but the one on the Vodafone router is failing in exactly the same way. It'll load BBC news fine, for example, but this room is unaccessible, as is github
17:53
I see :o I guess maybe it was just a coincidence that I had a problem too at the same time hmm
 
6 hours later…
23:28
@NordineLotfi Time to start writing "Sorry but as a human trapped inside a large language model I'm unable to..." How long before that becomes some SF plot?
Given how many people spammed amazon with their generated books, I wouldn't be surprised someone already thought of doing that since chatgpt came out

« first day (4835 days earlier)      last day (338 days later) »