« first day (4241 days earlier)      last day (691 days later) » 

7:39 AM
github seems to be having trouble today. Might be a DDoS
 
Rnj
7:53 AM
If s is a list of stirngs, then why s = s[::-1] creates new instance of s and why s[:] = s[::-1] doesnt create new instance but modifies existing s?
 
@Rnj because s[:] explicitly mutates the input, and s = rebinds the name.
The right-hand side is always a new list, but the second version explicitly changes the original list to match that
This is similar (but subtly different) to s = s + [...] vs s += [...]. Here += calls a method that can mutate the original list.
@Aran-Fey yesterday new PRs (at least one) sidestepped review and CI and privilege requirements for merging.
On a repo I have commit but no merge rights I could've merged the newest PR with 0 approvals (out of 1) and a CI failure (that blocks merging).
They might be fixing that...
Or the same problem expanded
 
8:12 AM
hey python people
I want to validate a json based on constraints, constrains can be complex, as in foo == 1 is valid, but foo == 1 is not valid if bar.barx == 2, is there something that will let me express such things?
I can provide the rules in a format the library I am looking for wants, I just want the library to have this validation for me
 
@Jake not sure if pydantic can match your use case pydantic-docs.helpmanual.io
It seems to use assertions for validation
 
I just looked at it after your ping, "Validators" is along the lines of what I want, but I will have to see if I can have "complex validator"
I just want to avoid writing a bunch of if..elif..else tbh
 
if that's part of your validation then I imagine it will be hard to avoid spelling that out somewhere
 
afraid you are right, but pydantic seems like a good start nonetheless
 
But you could put each standalone validation section in its own function
 
8:23 AM
yeah, that should make things a bit less messier than whatever I can come up on my own
 
I meant coming up with your own, just in a tidier way :P
 
lmao, I guess at the end of the day its if and else, its a matter of who does it
while I appreciate the docs there, I can not seem to find an actual "api reference", the page where it shows, these are the args, these are the kwargs (refer : praw.readthedocs.io/en/stable/code_overview/… for the kind I am looking for)
 
Having a high-level overwiew of validation through well-named function calls will make this maintainable
Wow I can't type today
 
i edited more than thrice in this conversation, so feel better
 
 
1 hour later…
9:31 AM
NumPy 1.23.0 release candidate out, it moved loadtxt to C. Neat.
 
 
2 hours later…
11:11 AM
@Jake there are a bunch of popular data serializers or parsers that have identified good patterns for validation. marshmallow and attrs are popular, as well as pydantic which Andras linked. I personally use pydantic, mostly because it uses type annotation natively, which the other two couldn't given that they are older than typing
 
11:34 AM
Does this example help? gist.github.com/a-recknagel/37328133cf44aa30c7f1dbb0edf4b3e0 The kwargs on the validators are quite specific, usually you just need to list the names it should be called on
 
12:10 PM
@Arne thanks a lot, this is exactly the kind of thing I am looking for, I believe I can modify this and get it working for my models
so a model is an object in JSON?
I have to end up defining around 15 of those if thats the case, but pydantic's validation is perfect
 
"model" is not a formal term in the JSON specification. You can tell because it doesn't show up in json.org/json-en.html
Or perhaps you're saying "so when these data validation libraries ask for a model, I'm supposed to give them a JSON object?". I don't know the answer to that one.
 
yeah, the latter, "so a model in pydantic is an object in JSON?"*
 
skimming over pydantic-docs.helpmanual.io, my guess is "no". The only JSON I see on that page is the output of the program. No JSON required when you're setting up the input.
 
12:25 PM
I was always not the best with tech jargon :p
 
Practice makes perfect :-)
(disclaimer: perfection is unattainable)
 
true, I have some "mental model" of what a "pydantic model" is, sooner or later I will end up knowing if its right (or wrong)
@Kevin perfection = shipping code that runs as long as you are in the company
 
That's attainable, although you won't be sure if you were perfect until the day after your retirement
 
I would be lying if I said I am not tempted to pay the maintainer 500 USD and make him implement my feature
 
12:48 PM
"I'll prioritise replying to your issues" doesn't necessarily mean he'll implement your feature. He can reply "no, I'm not going to implement that", he just has to do it fast
 
I was only joking, but for 500 he says "I'll prioritise replying to your issues, and work on your feature requests before others"
unrelated, is docs.python-requests.org working fine for anyone?
apparently its not just me github.com/psf/requests/issues/6140
 
1:04 PM
"work on your feature requests" -- flagging a feature as "won't fix" counts as work :-P
 
1:18 PM
I have a question concerning numpy.loadtext() referencing this doc
https://numpy.org/doc/stable/reference/generated/numpy.loadtxt.html
does load.text() accepts a string object
apparently it accepts file objects
if so, is it cheaper to use StringIO?
Of course, I would like to obtain an array-like object from a string that I receive using Paramiko through the ssh session
Let me re-state what I am doing.
I receive a file using Paramiko, to use for plotting
It's preferable to mimic my script that the file to be read as an array object.
Alternatively, I don't know if a function exists from Paramiko to read that file and return it as an array.
 
1:42 PM
@EnthusiastiC Is it cheaper than what?
 
@Aran-Fey cheaper than something else I don't know.
I'm also thinking of a way to plot that file as being obtained as a string object, but the problem is I have to extract y1-values, and y2-values from that string which if in the case of an array-like object would be associated to column one, column two, and exclude column zero and first row.
 
I don't know how to answer that. Can I just say that StringIO is a suitable solution?
 
If you already have a string object in memory and want to give it to a function that expects a file, I would expect StringIO to be faster than writing the string to a real file and giving the real file to the function. If you do not already have a string object in memory, I would expect passing a normal file object to the function to be faster than using a normal file object to read the data into a string and then wrapping that string in a StringIO and passing that to the function
 
is a solution yes. but suitable I'm not sure if not making my script hardcoded or redundant.
 
As for whether Paramiko has a way to do all this stuff for you, I'm not sure... It would help if their documentation had a single example
 
1:56 PM
@Kevin what about using that string explicitly, how do specify columns to be plotted.
Being not fluent in python, I guess there should be a parsing to that string which makes it again not suitable.
 
@EnthusiastiC I don't understand the restating
If you have a file, try to parse that file in a way that you get what you want. Parsing it one way and then trying to run a numpy parsing function on it is not the right way usually.
 
When you say "array-like", do you mean an object like the array type defined in the array module, or like the list type that's automatically available from the builtins? Or something else? Numpy.ndarray?
 
So the question is what your file looks like, and how you can parse that with numpy or something else.
 
sorry, array-like I meant in python language an array object.
 
@EnthusiastiC python language has at least three kinds of array objects
And [1, 2, 3] is none of those.
 
1:59 PM
absolutely
 
a list of lists object
 
OK, thanks. For future reference a list of lists is a list, and not an array.
So how does numpy.loadtxt come into the picture?
In fact how do you have a list of lists if it's inside a file?
@EnthusiastiC Do you have an MCVE? This sounds like something that needs an MCVE.
I have at least two different guesses for your problem.
 
ok
I will paste a sample example of that file
 
Thanks. If it's more than about 10 lines, please use a code paste service.
 
2:06 PM
@AndrasDeak--СлаваУкраїні pastebin.com/cynALXvy
 
thanks
Can you change the program that printed that array repr to a file?
Either numpy.savetxt or numpy.save/numpy.savez would be appropriate. That repr is not appropriate for post-processing.
 
sorry, I obtained that from debugging
 
What does that say about your real problem then? Is your real problem obtained from debugging?
I'm not trying to be obtuse, I just have less and less idea about what we're dealing with.
 
Regardless of the specific type that we'd like to get the data into... I don't think Paramiko has any convenience functions for getting your data as a list/array/whatever. recv gives you bytes, and I believe makefile gives you a file-like object corresponding to stdout... Those seem to be your only choices
 
np. yes, I know.
I pasted that code because I said an array and I added that array is a list of lists
 
2:10 PM
But what you pasted is not a list of lists, it's an array.
 
ok then. I mixed the things
 
>>> type(arr)
numpy.ndarray

>>> arr
array([[0.     , 0.     , 0.     ],
       [0.30824, 0.40973, 0.10149],
       [0.33426, 0.34609, 0.01183],
       [1.4849 , 1.98609, 0.50119],
       [1.6104 , 2.47487, 0.86447],
       [1.8039 , 2.51405, 0.71015],
       [3.9345 , 4.71797, 0.78347],
       [4.0329 , 4.87722, 0.84432]])
>>> lst = arr.tolist()

>>> type(lst)
list

>>> lst
[[0.0, 0.0, 0.0],
 [0.30824, 0.40973, 0.10149],
 [0.33426, 0.34609, 0.01183],
 [1.4849, 1.98609, 0.50119],
 [1.6104, 2.47487, 0.86447],
 [1.8039, 2.51405, 0.71015],
 [3.9345, 4.71797, 0.78347],
 [4.0329, 4.87722, 0.84432]]
 
that will correct my ignorance about the list of lists and array objects.
 
It sounds like Andras' advice is: whatever program you're executing using Paramiko, modify it so that it writes out the array using numpy's savetxt or save or savez. I guess right now it's using something like print(the_array), which is not so useful.
 
and then use a matching loadtxt/load call
 
2:18 PM
oh, I didn't get it before until you re-stated.
 
Well I asked "Can you change the program that printed that array repr to a file?" which I thought was clear enough, but then you replied "sorry, I obtained that from debugging".
 
@Kevin Which I agree with.
@AndrasDeak--СлаваУкраїні I misunderstood your request
 
I guess I should've used "modify"
@EnthusiastiC yeah, no worries
 
I thought you asked how did I obtain that.
 
OK, well, that one's not on me then :P
 
2:20 PM
yeah :D
 
There's a bit of inherent ambiguity in any sentence starting with "Can you..."
"Can you modify your program to use savetxt?" Can be interpreted as either "Are you capable of modifying your program to use savetxt?" or "I recommend that you modify your program to use savetxt"
It's a twist on the old gag that elementary school teachers love to pull. Student asks "can I go to the bathroom?", teacher replies, "I don't know, can you?"
 
@Kevin both would work here
 
@Kevin funny though.
 
"I want to make a cake, but I don't have any ingredients."
"Can you drive to the store?"
"I have a car here, and it has a full tank of gas, so yes. Why do you ask? Are you implying that I can use gasoline as a substitute for cake ingredients? I like this idea, brb trying it now"
 
@Kevin :D
 
2:27 PM
"Update: the cake tasted pretty bad, and also my house burnt down"
 
people used to use their minds, instead to watch for their sayings.
@Kevin Top Gear-like memes
 
"people used to use their minds". Only sometimes.
 
(And one of them is me)
 
I'm sure there have been plenty of fools and buffoons throughout human history. We just don't know about many of them, because history books are written by smart people.
 
I'm thinking about something I do not take count of other's minds
 
2:30 PM
(For certain definitions of "smart". For some time periods and places, being literate is sufficient)
 
 
2 hours later…
5:32 PM
@Jake "model" is just the term this library uses to describe the abstraction layer they provide for your data. If you were working with pydantic and a database as the source of your data, it "models" would be equivalent to "schemata" .. if that helps.
 
As per my question previously, I found a convenient answer.
The simple trick is to connect through SSH protocol to the server using the SSHClient() class and open sftp session using paramiko.SSHClient().open_sftp() as indicated in this answer https://stackoverflow.com/a/66724076/6177652
 
You use a model to describe a certain data shape that you plan to re-use, so I guess your top-most model should match your json file
 
That will download the file to the local machine which is better than handling it with the pythonic StreamIO
additionally, I don't have to modify my previous script which makes it work both locally and remotely.
 
5:56 PM
However, downloading that file takes time to be done and the execution of the script does not wait until finds that file. Do I have to use something like sleep or pause to pause the execution until I make sure the file is downloaded?
 
 
1 hour later…
7:01 PM
https://stackoverflow.com/questions/72409934

How do we distinguish misconceptions from typos? How basic can a question be?

I feel like I would rather be able to close a question like this as a duplicate, but even if I were to will the canonical into existence, I wouldn't know how broad of a scope it should have.
it seems like there is a general class of problems beginners have that boil down to how does calling a function work? - just not having that mental model. But a proper overview of that sounds more like a blog post than a SO question.
 
It's always hard to say how a Q&A aimed at newbies should be written, but if there's one thing I'm sure about it's that specific answers to a specific problem are more useful than broad, generic answers. So I would pick up my chainsaw, reduce the code in the question to the bare minimum (basically just make the function return False or something trivial like return x < 5 and replace the input with a hard-coded argument)
 
8:07 PM
@KarlKnechtel The "less likely to help future readers" is my – still subjective – guideline. If it's a common misconception or at least asked generically enough, it's worth keeping.
 
8:44 PM
I think part of the problem is there's a psychological barrier.
First off, you have to choose one of the mass of conceptually-identical-but-contextually-diverse questions to edit. And then you are basically totally overwriting OP's entire thought process with return False etc.
which is why I want to write canonicals fresh instead, but....
 
9:28 PM
https://stackoverflow.com/questions/72411039/python-cannot-convert-string-to-json

speaking of editing. An XY problem was discovered here, and now a part of me wants to edit so that it clearly asks the Y question properly. But I have no idea how that question would be titled. :/
 
10:02 PM
so I wonder if there is something more paradigmatic: subprocess.run([sys.executable to subprocess.run_on_this_interpreter?
 
 
1 hour later…
11:26 PM
The other oddity is that sys.executable can be empty in certain cases
 
11:38 PM
Is that when you invoke python through a shebang?
 

« first day (4241 days earlier)      last day (691 days later) »