« first day (4910 days earlier)      last day (32 days later) » 

12:00 AM
There's no problem, I can flatten my video data
 
Ok there you stated you're a final year PhD student (astronomy? physics? geography? CS?), and you're looking for a prediction model of videos from data relating to space weather. Sounds pretty domain-specific ML question; it's not a programming question, . What forums/sites(/journals/conferences) do people doing that use, and what models/approaches do they use? Find those people and start talking to them, not us. What are the most commonly-cited basic papers in that field? Try to replicate them
 
I think there's no problem about doing it
@smci Good suggestions
 
... i.e. first replicate a very basic finding on a commonly-available dataset (pick one that's available to you).
 
What do you think about checking the predictability of my time series using Shannon entropy or another type of technique?
@smci Applied Computing
 
a) Are you a final year PhD student in astronomy? physics? geography? CS? maths? something else? b) What forums/sites(/journals/conferences) do people doing that use? [ok: a) Applied Computing b) ? ]
 
12:03 AM
In Applied Computing
What people? The paper's authors doing this type of forecasting?
 
@Marco I already told you I have no idea, I said I don't do spatial (2D/3D) timeseries forecasting. And I said it wasn't an SO programming question. Find the communities of researchers who do do that and talk to them... Also, you have a supervisor, right? They should be telling you this sort of thing; what was their advice? If your supervisor's not very useful on this topic, find someone in that domain who's useful. They ain't gonna be in this chatroom.
 
Ok, thank you very much for your suggestions
I agree with you that it's not a specific programming problem, I just wanted to get some guidance on where to find someone who can help me, you've already given me good tips, including one of which I was thinking about doing this in the last few days (talk to paper authors who are doing something similar for example).
Yes, I have a supervisor, I prefer not to talk about that part. :(
 
@Marco Yes, what forums/sites(/journals/conferences) do the research community doing that use? Like "ACM Transactions in Geospatial XYZ Modeling" or something? Identify commonly-cited authors/textbooks. Find a related conference/workshop and read what list of algorithms are considered hot. Try to replicate basic findings. Start simple. If you get stuck, reach out to the researcher community discussion groups, or the seminal paper/textbook authors. Don't ask us, talk to the career researchers
 
Yes, when I asked here I was looking for an expert in machine learning in general, not someone extremely specialized in the problem.
But you are right.
I believe that CrossValidated can also help me a lot with several questions, it has been helping me recently.
 
@Marco Ok but you gotta develop some other sources in that community. Absolutely start by replicating basic findings from seminal paper/textbook authors are cited in that field. Start by using a commonly-available (spatial timeseries) dataset whose behavior is well-studied, before you use yours (analogous to the 'iris' or 'diamonds' datasets in ML). Then use that model as a baseline for comparing other approaches.
 
12:15 AM
Just one question, is applying a sliding window to a time series something common and advisable? Can I also use the data without applying a sliding window?
@smci But why would it be interesting to apply it to simpler, known data?
These Koreans predict exactly the kind of data I want to predict, but they use more data than I do, etc. Anyway, they use CONVLSTM too, and lately I'm basically replicating what they did, they even have a github of the code. agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2023SW003763
They predict for the Korean peninsula, I now want to predict for another country.
 
@Marco Yes, but I'm a humble EE. Think of a sliding window as a preprocessing technique trying to isolate part of the timeseries so we can model how that changes. (There are other approaches, like Fourier transform, wavelets etc.)
 
What is EE?
 
(ELectronic Engineer. My point is I haven't done much predicting spatial timeseries like video, so ask those people, not here)
 
So it's better to use a sliding window than not using it, right? I mean, it's more common to use a sliding window than to use the pure series, right?
@smci Ok, thanks
@smci Right, no problem and sorry
 
@Marco I've just said six times I don't know, it's not my domain, I'm not going to conjecture wildly, my uninformed conjecture is worse than any simple experiment you might conduct, go read the basic papers and textbooks. An ideal first experiment for you to start with is try a simple model with a) no sliding-window b) sliding-window with c) various parameters. But definitely, do it on some publicly-available dataset widely used in your field, before you use your own private data...
Happy replication experimenting...!
 
12:24 AM
Ok, thank you very much
And I'm sorry again
 
"Intution is a useful guide, but experiments alwys beat conjecture". Set up simple replication.
 
Hmmm, nice
 
DataScience.SE has 27 questions on ConvLSTM but it still isn't specialized enough for this; find your research community online, their journals, conferences, tutorials etc. Start talking to them. Replicate basic findings. (PS: don't try to immediately come with a State Of the Art-beating finding, esp. if you're working totally on your own)
 
Yes, thanks
 
As a matter of fact, other researchers may be looking for someone to do the grunt work on a study of theirs, sometimes you can get junior coauthor on a paper or else acknowledgment that way. If it helps you get insight, then great. Definitely start getting plugged into journals and conferences on this. Skim the last 10 years archives for key papers, use Google Scholar to see which basic studies are most commonly cited, then get their data.
 
12:31 AM
Ok, Just for the record, this is the paper that introduced ConvLSTM: arxiv.org/pdf/1506.04214.pdf.
 
(There's a lot of good meta-advice about being a PhD and how to do research, see Academia.SE and Workplace.SE . Really you want to forge a functional relationship with someone in your supervisor's dept, or someone in some dept your university or nearby who's working in this. Don't try to solve everything yourself.)
 
Yeah, I'm doing the best I can with these things, believe me.
But I hope everything works out.
 
Don't try to solve everything yourself, you don't have the time, energy or the sanity; noone does. Set up simple experiments and start quickly proving/disproving things, e.g. whether time-windowing (and what parameters) help convergence.
 
I understand you.
 
Ok good luck.
 
12:37 AM
I'll try do the things. Really thank you.
Good night (I hope where you are is night).
Hmm, you are in California.
 
Also, edit your own university webpage describing your research, so like-minded people can find you online.
 
@smci there's no page like that in my case :P
Almost night in California :P
 
@Marco Your research group will have a link to research and personal homepages; create one with a paragraph on what you're researching, and how to contact you.
..if it doesn't, just set up something basic and get listed in the dept directory of researchers.
 
Ok, I'll try to do something like that
Thanks
 
 
4 hours later…
4:29 AM
Hello Python chat, I have an issue I am dealing with and I need your help.

I am not very good with scopes, so bare with me here.

I have a function declared like this:

# quran command
@bot.slash_command(name="qurantest", description="Read Quran through Sheikh Bot", guild_ids=[test_guild])
async def qurantest(ctx, surah: int, verse: int):
    surah_variable = surah
    verse_variable = verse
I am using them as non local variables, and I have an exception handler here:

except quranpy.exceptions.IncorrectAyahArguments:
                nonlocal surah_variable
                nonlocal verse_variable
                verse_variable = 1
                surah_variable += 1
However, it raises this exception:

File "D:\sheikh-bot\advanced_quran_command.py", line 96
    nonlocal surah_variable
    ^^^^^^^^^^^^^^^^^^^^^^^
SyntaxError: name 'surah_variable' is used prior to nonlocal declaration
Here is the full pastebin for whoever needs the full code: pastebin.com/AxdzRxjr
 
@Marco Usually these days they may also have an external web-facing page, but with your email, phone, work address scrubbed.
 
 
3 hours later…
7:30 AM
@NordineLotfi it's probably not open, it's some communication library
 
 
1 hour later…
8:33 AM
@zoomingspeed In Python, blocks like except: etc. don't create a new scope, so variables are scoped to the entire function (or the enclosing function, or the module globals, or builtins) instead (except for some niche cases like inside a comprehension, and that wasn't always the case). If you want surah_variable to be a nonlocal inside this function (i.e., to refer to the name surah_variable in an enclosing function), then the nonlocal declaration should be at the beginning of the function
i.e. in the full code, it should be outside of the try/except, at line 72. but code like this is very difficult to follow. Consider the Zen of Python: Flat is better than nested.
If you're going to create a new NextAndPreviousVerse instance anyway, it would be easier to have that class store e.g. self.surah and self.verse which are initialized for the ctx.respond call and modified as needed as the user interacts with the UI.
(i.e., give it an __init__ that expects arguments for those, stores the values, and calls super().__init__())
with that kind of structure you should be able to un-nest everything and make it separately understandable.
(You may also find it helpful to create a helper function to create the basic discord.Embed and add the standard fields to it, since it looks like you use the same or very similar fields each time)
 
 
4 hours later…
12:51 PM
Should Celery tasks raise if e.g. processing invalid user input?
Is it good practice?
For example my task processes a user request to fetch an item that doesn't exist.
Should I raise?
The issue is that this produces a lot of repetitive logs in Azure (and it costs a lot of $)
But then again, I don't want to use a bad design just because Azure doesn't handle multiline logging well.
 
 
2 hours later…
2:59 PM
@smci Ok, thanks.
 
@PythonForEver depends, but in general raising is good practice, if the function has clearly defined limits and description of what the inputs should be
 
3:33 PM
Yay, I've found the file leak. We've been leaking files since 2016 :)
 
 
1 hour later…
4:39 PM
@PythonForEver That completely depends on what the user expectation is. Fetching a non-existent record in SQL will return an empty set. A HTTP request might return an error code to the caller. Neither of these explicitly throw in themselves - the user makes that decision
 
5:30 PM
This question got me thinking about whether it's really true whether sound cannot travel in space at all. Defaulting to "vacuum" seems lazy when clearly we have large gas clusters in nebulae and multiple stellar events cause mass ejections of huge quantities of gas and other matter. I wonder whether anyone has looked into the (admittedly huge) attenuation of "sound" in these events
Given the ridiculous speed and energy of stellar events, I could imagine a gas shock wave to make it a good few miles in varying levels of compression before it attenuates to nothing
 
6:21 PM
I guess it depends on the density/pressure hmm
there is also this: physics.stackexchange.com/questions/162184/… but this is only about the speed but still related on some level
 
I've just nearly finished drafting my question :)
 
does python have some kind of object that can be used for an error code with some additional checking that ensure the code code is actually checked by the caller?
 
@NordineLotfi gah, I think the last paragraph answers my question. Nice find!
@NordineLotfi This was what I was drafting
dpaste was a decidedly awful place to post plaintext it seems. Sorry
@Mikhail maybe an Enum?
 
so basically if somebody doesn't check it, they should get a runtime error
an adjacent concept is like [[nodiscard]] in C++
 
I don't think that checking it at runtime is a typing issue?
 
6:33 PM
You can set a flag in __bool__ or __eq__ or whatever and then throw an error in __del__ if the flag isn't set
 
Yeah I see constructing that type is straightfoward
but wanted to use something built in
maybe returning a Callable
 
@Aran-Fey is this affected by the GC in any way?
 
I guess so? You don't know when exactly __del__ will be called, but that's all I think
 
thats a good point
 
I guess what I was imagining is that the next operation might delete a file or something before the GC kicks in and throws on the __del__... but it's too late
Rust addresses this kind of thing with Option<> and Result<> but handling them gets enforced at compile-time. I can't think of a fool-proof approach in Python typing that I have seen, especially for runtime
 
6:49 PM
@roganjosh had to reword multiple time because I didn't find this until I used "gas cloud sound travel astronomy stackexchange"
 
I didn't manage to reword it in any way that didn't just give back the answer I wanted to disprove. I must try harder :P
 
@roganjosh ah, now you know how I feel like when I want to ask on SO but I end up solving it before I click on "Post your question"...and the draft is like, multiple paragraph long
@roganjosh it's actually fine if you use the raw link
 
@Mikhail the more I think about this, why don't you have multiple custom exceptions that you raise? That forces the caller to address them
 
reasons are strange
in multinode or process scenarios you typically need to somehow forward exceptions to other nodes or processes
but then you want to reuse the same code paths for the single node case
so basically you should be using error codes to facilitate error state forwarding BUT you don't want people to shoot themselves in the foot when calling the functions either
 
7:17 PM
Does anyone here use hatch? It seems pretty neat, but I don't want to spend 3 hours figuring out how to set up unit tests with multiple python versions and combine the code coverage results for all of them. So I'm hoping I can steal someone's finished setup
 

« first day (4910 days earlier)      last day (32 days later) »