« first day (3254 days earlier)      last day (1919 days later) » 

 
5 hours later…
user10984358
05:14
Heya guys, a certain use case of mine requires dumping stuff into a “temp” folder. I need to make sure the user doesn’t mess with that folder during runtime. Say, using os.makedirs to create a folder there or delete stuff I have stored there. Is there a way I can prevent those? Kinda like “locking” it.
05:42
@TheNamesAlc How about using a TemporaryDirectory?
06:13
Is there an easy way to check if my program is running on a laptop or a desktop PC?
Rob
Rob
Probably none that are completely reliable. I guess you could check the battery level/charging state?
It doesn't need to work in the general case, just needs to work on my laptop and my desktop
hmm, there doesn't seem to be an easy, cross-platform way of checking the battery level
I've also considered querying the size of the hard disk (laptop = 500GB, desktop = 2TB), but there doesn't seem to be an easy way to do that, either...
Rob
Rob
If it's just your two particular machines... couldn't you just put a config file somewhere?
Or check the machine name?
06:28
the thing is that I have multiple operating systems installed on each machine, so I'd need 3 copies of this config file
and I tend to (re)install various linuxes rather often
Rob
Rob
@Aran-Fey Would this not work? stackoverflow.com/a/48929832
machine name might not be a bad idea... I'm sure each of my 3 OS's uses a different one, but that shouldn't be too hard to fix
This gives you the size of a partition, not the whole disk. — Aran-Fey 19 mins ago
:(
Rob
Rob
Oh, you were already there
I still think a config file (merely check for the existence of it) would be easiest. At least, pretty simple in comparison to re-installing an OS and copying your app over
It could even be part of the distributables, and you delete it on either your laptop/desktop
oh, right, I forgot I have a separate /home partition. So I'd only need to create the config file twice, once on Windows and once for all the linuxes
or maybe I should just auto-mount the windows partition and only have the file there
*grumble* maybe sometime in the next century Windows will learn to support file systems other than NTFS and FAT32
alright, got it all set up. Let's see how long it takes for something to go wrong
07:12
@Aran-Fey vendor lock-in...
Is that what it is? How does Windows's lack of support for other file systems force users to keep using Windows?
user10984358
There’s something called APFS Incase anyone isn’t aware and it’s a pain in the neck to not have your old hard disks in that format.
@Aran-Fey Being able to use both from windows opens the gate for using other OSes seamlessly. At which point users would notice that windows sucks.
user10984358
Also @Aran-Fey that module you suggested is what I want. But can I give my own path ? I have access to a limited part of the system.
It's hard for me to accept that anyone could be unaware that Windows sucks :/
07:17
The harder it is to use win and anything else the more costly it is for users to switch to anything else as long as they need windows for certain things (games, adobe, etc.)
so you make it harder for people to switch, but at the cost of them hating you? not sure if that's a good tradeoff
MS users have been hating MS for decades, haven't they?
heh, maybe they have
@TheNamesAlc I know the documentation isn't the best, but there's clearly a dir parameter
I'm just saying MS of all companies could've figured out ext4. It's for want of motivation, not competence.
though if you create it in a directory where the user can see it, the chances of them tampering with it are a lot higher
user10984358
07:24
I keep getting OSError
user10984358
i passed in a perfectly legal r”” as a path
what confuses me most about the Windows situation is that a lot of companies pay for it even though they absolutely do not need it
user10984358
It was onedrive not allowing me. Sorry for that.
My workplace: 28 computers with Win10, the boss is annoyed by the constant problems Windows causes (updates and other stuff basically making the PC unusable for hours at a time), and yet the only thing people ever interact with is a web browser and a VoIP software
Sure, that specific software probably wouldn't work on linux, but I'm sure half a dozen better ones would
xkcd 2200 is moderately amusing:
07:47
@Aran-Fey unfortunately skype is probably still the standard choice for VoIP, and...guess who's maintaining that
@PM2Ring the next one, however, was quite ugh
08:20
@AndrasDeak Agreed, although the title text almost saves it. I don't expect BHG to do surreal stuff like that, that's more Beret Guy's schtick, although he doesn't usually do intentionally malicious stuff.
08:52
@PM2Ring Perhaps he's done with this planet xkcd.com/2174
09:07
@AndrasDeak :) I liked that one, and it spawned a long discussion on the forum, mostly consisting of people recounting their first news memories.
I hope the forum isn't permanently dead. It's still offline. :(
09:21
Hi guys, what are some methods/tools you used to map out complexity?
I tend to spend too much time trying to link together or sometimes miss out connections between variables
Hm, that seems a bit vague, not sure what quite you're referring to
One thing that almost always helps me is making and using functions. It helps me modularise code into logical "blocks" with appropriate names. Once i am happy with the way a function works, i can essentially forget about the internal working, and just treat the function call as a block that just works. Helps make the overall flow of the program much nicer.
Right although I would guess most people use some pen and paper to make flowcharts and stuff prior to coding to simplify things and organize how the program will work first
I found it is often difficult to go straight to coding ideas as you go without mapping things out
Different people work differently. Find what works for you best.
Flowcharts are definitely a popular option, but i personally haven't used one. Not saying whether they would help me or not, but just haven't felt the need to resort to a flowchart for code so far.
Yeaah I have not found an efficient way yet though. I spend too much time getting lost while trying to connect them and miss out some connections in the end
And end ups bugging my program with wrong calculations somewhere down the road
With some special cases
If a top down approach (sort of what flowcharts support) makes it harder for you to work, perhaps give a bottom up approach a try? Think of your smallest "logical unit". Write down a list of responsibilities it has. Work on it without worrying about how others use the data it outputs initially.
Once you're satisfied with a "unit", you can then focus on building other "units" that need it. So essentially, a list of "things this unit must do", 1 unit at a time.
If i had to describe my coding process, i think that would come close. I may not explicitly write down everything, but that's generally the thought process i go through when coding.
Having said that however, find out what works best for you. give different things a try, see what clicks.
09:37
Right I think I should work with breaking them all down first. What I do is I try to create the connections while looking mainly on the code and it becomes complex too fast and I end up re-reading the same lines over and over again
I guess I planned it very poorly and I take longer to do something and tend to miss out on things
Thanks i'll just check on some system maps to try organize the relationships
Connections flow naturally if you can trust your "units" to do their tasks correctly. The approach i personally take: Link things later, get them working first. Your mileage may vary.
@PM2Ring aww :(
I wish i could open xkcd links at work :(
Yeah I think that's my problem. I try to think of all the factors affecting it and link them all right away in one go lol
@ParitoshSingh Filter or just frowned upon?
09:44
the former, im afraid.
How about explainxkcd.com? Has each comic included. Or is it hotlinked from xkcd.com?
hm, let's find out
or if they block xkcd they have a whitelist
Oh, that went through!
Time to be productive :P
The explanations on this site are pretty nice actually
well they're crowdsourced, so they're as good as the crowd
10:01
cabbage
Does anyone know an easy way to iterate over a dataframe left to right over the columns on even rows and right to left on odd rows? I'd be happy with a few pointers
Are you okay with doing it in two steps? Once for even rows, once for odd rows?
(i wouldn't bet there are other options)
but doing it in two steps seems most sensible.
I need to make one list, but I guess doing two steps would work by adding the results of each row together
Splitting it up in two already makes a lot more sense
Indeed. (though i will say, you may also want to take a look at your dataframe itself. This kind of iteration makes no sense to me in any tabular data. should your dataframe be stored/aligned differently in the first place?)
But im only guessing here without knowing anything about your actual data, so it's for you to decide really.
10:07
Unfortunately it's how the data got delivered to me, makes no sense to me either.
Ah. bogus data upstream :/
If there is a way to just reverse each odd row that would also be great
that sounds doable too honestly.
@mtbrands nothing stops you from making it sane on your end, right after you get it
^ pretty much
i'd take that as my first "preprocessing" step.
10:08
@mtbrands so what do columns represent? This sounds a lot like an XY problem or a "hit the person giving you the data until they stop" situation.
you typically can't reverse rows because then the columns don't match up
Very true, but I couldn't think of a good way to reverse rows either
I think this isn't really a tabular data where the columns have meaning, so much so as simply a 2d array of numbers/values. (speculation continues)
I otherwise can't justfiy such a structure.
>>> df
   a  b
0  2 -4
1  3 -5
2  4 -2
3  5 -5
4  6 -1
>>> df.loc[:, ::-1]
   b  a
0 -4  2
1 -5  3
2 -2  4
3 -5  5
4 -1  6
The columns themselves dont represent too much, the data is certain codes that determine an ordering, but for some reason whoever made this goes over it zigzagging....
Ah, so the columns have to stay. I'm not a pandas user but I'd use df.values[::2, ::-1] to get a reversed odd block, and assign it back
>>> df
   a  b
0  2 -4
1  3 -5
2  4 -2
3  5 -5
4  6 -1
>>> df.loc[::2, :] = df.values[::2, ::-1]
>>> df
   a  b
0 -4  2
1  3 -5
2 -2  4
3  5 -5
4 -1  6
10:11
I think the columns can go, in the end I just need a long list of codes by zigzagging down the dataframe
@mtbrands do you even need a dataframe?
i think the dataframe can go
Nope
then just pick out df.values and use 2d arrays for sanity
and if you need a 1d stream then you'll do arr = df.values; arr[::2, :] = arr[::2, ::-1]; data_1d = arr.ravel() for row after row
>>> df
   a  b
0 -4  2
1  3 -5
2 -2  4
3  5 -5
4 -1  6
>>> arr = df.values; arr[::2, :] = arr[::2, ::-1]; data_1d = arr.ravel()
>>> data_1d
array([ 2, -4,  3, -5,  4, -2,  5, -5,  6, -1])
odd vs even is a bit ambiguous, because the first row is the zeroth, but that's only an index away
i just find it annoying 0 is the odd block here
10:13
And more columns with not all rows being filled equally?
ha. was going to say, that took me a second to parse :P
The empty ones is just take out nans at the end
@mtbrands hmm?
Some rows have 12 items, others only 5
you might want to ask your real question perhaps, with an MCVE, rather than shoehorning more and more patches onto a half-baked question
10:15
Youre right
(the working-class imagery got a little mixed up there)
also i don't see why this should blow up with nans to be honest. Try running it and seeing how it* goes
I think they want a contiguous 1d stream of the data zigzagging, with nans being ignored
but I'd rather see a proper problem statement
aye, though filtering out nans from a 1d array is just 1 extra step. Once you get the data in the correct structure, every problem should be easy to deal with from that point on
Sorry my thinking was Id get underway with a good start
But that didnt work out so well
10:16
well you got a good start :P
Yeah this should work
10:45
Hey guys - a quick terminology check
(aka, how to improve my google-fu)
I'm looking at trying to get a thread to yield mid-process, so any other threads can handle any I/O before I continue intensively crunching again
If I use the word "yield", I get results referencing to the keyword/generators/et-al
"release" the gil
What term should I be using
something along the lines* of release should work
or suspend may also work
Thanks
@OldTinfoil The threading module docs describe a few tools for doing that sort of thing. Eg, a Barrier may be good for that job. docs.python.org/3/library/threading.html#barrier-objects
10:54
Yeah, i started just thinking about/exploring this problem, and one thing i just realised is that i can always write a simple lock release call, but there's no "guarantee" i would have allowed some other IO/thread to run before i re-acquire* it, is there?
I should add though, this isn't an area im familiar with.
@OldTinfoil can you clarify what your problem is? threads release the GIL automatically at regular intervals
Well, normally I wouldn't find myself in this position. When it comes to this sort of level of co-operation, I would rather use different processes that use a DB or some kind of MQ to communicate
do you have starved thread?
@MisterMiyagi I am using pymodbus and using code similar to the UpdatingServer example. When I'm updating, I'm doing some queries on a database that can take up to 10s to complete. When this is happening, any queries to the modbus server/slave time out until the twisted looping thread is completed.
So can lock out a client connecting to the server for around 30s, whereas if I can reduce that to around 10s or so that might be acceptable until I can work out a better way to perform the tasks (ie, spin up a new process, do the query and wake the updating thread with the data so the server thread is not starved)
Essentially, it'll be a bit of a faff to make an MCVE - hence why I was trying to improve my google-fu before going into it ^^
I mentally translated your question into "how to set priority on threads in python"
the results were actually pretty negative, since the GIL would not be aware of the priorities you assign. One "hack" suggested seems to be to use ctypes and the os dependent priorities for thread.
But all this makes me squirm, and Im not in any position to give good advice here.
11:07
admittedly, I do not quite understand where threads enter the problem. if you are doing 10s of IO/query, thread rescheduling should happen immediately.
@OldTinfoil is the updating happening in a thread?
To update the modbus server, you need to create a thread to access that server's read-only registers.
perhaps that's what missing
I suppose one question i also want to ask, you create the thread in python? or is the thread creation abstracted away somewhere
11:10
I create the (twisted.internet.loopingcall) thread, but how it accesses/modifies the registers safely is hand-waving magicks that I might have to source dive to look at
(with the implication that maybe we're looking at the wrong place. if the actual thread generation is handled within the library, either the docs may help, or heck, the threads might be created out of python's domain for all we know)
  def validate(schema, dictionary):
    return all(k in schema and dictionary[k] is not '' for k in dictionary)
It's not an MCVE, but I've stripped out various db calls etc for clarity
is this correct?
  def validate(schema, dictionary):
    return all(k in schema and dictionary[k] != '' for k in dictionary)
or empty string?
11:16
it's not correct. don't use is for comparison to empty strings
use equals for equality
  def validate(schema, dictionary):
    return all(k in schema and dictionary[k] != '' for k in dictionary)
@ParitoshSingh like this?
something like that. now, the question remains, what do you want to ensure, can dictionary contain extra keys? can schema contain extra keys?
ideally they shouldn't
@Aurelius What kind of object is schema ?
Then you may want to alter the condition accordingly. right now, schema can have extra keys and the code won't complain as long as dictionary contains a subset of schema. is schema a dict?
11:24
if self.validate({ 'username', 'password' }, data):
I'd just break it into two conditions
def validate(schema, dictionary):
    return schema == set(dictionary.keys()) and all(val != '' for val in dictionary.values()) #the .keys() is optional
the first ensures all keys match, the second just checks all values for non empty
if you can rely on truthiness of the values, you can change that to all(dictionary.values()) to shorten it.
@OldTinfoil note that if your actual queries cannot be interleaved, no amount of releasing threads is going to save you
can you find out which action exactly is blocking others?
@Aurelius Does validate really need to be a staticmethod, rather rhan a plain function?
@MisterMiyagi I run 2-3 long running queries. I was hoping I could yield between them to reduce the timeout from 30s to 10s
11:43
@PM2Ring I know what that means in most programming languages, but I didn't even know that that function was static in Python
@ParitoshSingh for truthiness you mean all val != '' right? That's more important to check than the exclusive presence of keys
@Aurelius as in, if you are sure that you will always get strings in values, then '' is "falsey" and non empty strings are "truthy".
So, i don't explicitly need to write a check for comparing with '', and can just rely on all(dictionary.values()) and it will get me the boolean result as needed.
So no, no relation to keys vs values, just "truthiness" in python for things like empty strings and so on
@ParitoshSingh so you mean that if we rely on all(dictionary.values()) and I receive a number or an array instead of a string for the password, the test wouldn't work?
It would depend on what's "desired"
do you want an int with 0 to pass or fail
do you want an empty list to pass or fail
because both of those are "falsey"
(don't call lists arrays, because lists are lists and arrays are arrays)
@Aurelius are you effectively trying to do something similar to marshmallow?
12:04
@ParitoshSingh for this very specific case, which is a basic project, I would love to receive only strings, but I can't rely on that
then keep the condition explicit
since it's a very basic and simple test I thought of writing this simple function, otherwise I would have installed pypi.org/project/schema
(though generally, you should always have sanity checks for your inputs. you should know what you're getting)
@JonClements something more similar to pypi.org/project/schema
@AndrasDeak lists are linked lists?
no, lists are lists.
12:08
@ParitoshSingh isn't this check that we wrote a sanity check?
@Aurelius python lists are honestly pretty close to arrays, but there's array.array and bytearray and third-party numpy.array etc. which beg the distinction
the big thing with python lists is that you can have heterogeneous data in them.
this class with this validate method is used in a django rest framework view to validate the input body of a post request
typically, arrays don't allow that in any* implementations except js-sigh (that i know of so far)
12:09
I could/should(?) also use serializers for this maybe
@ParitoshSingh JS? Probably the main source of confusion
sigh.
js needs to undergo what python 2 and python 3 did.
If it weren't for JS people might not call [1, 2, 'foo'] an array
I vaguely understand they have done a lot of releases that "allow" good writing, with new syntax, and what not. but they refuse to let go of the mess
@ParitoshSingh probably typescript and strict mode
you're saying "refuse to let go of the mess" as if python 2 code is ever going to go away
everything is full of legacy technology and the internet front-end is probably the worst, with everyone and their grandmother inventing new things incompatible with everyone else
Plus everyone's doing it and as we know "everyone" as a whole writes horrible code and has very bad habits. Even if someone created JS 3000 the debt would still be there
12:21
@Aurelius Well, your def validate(schema, dictionary): doesn't have a self arg, but you're calling it as a method in self.validate({ 'username', 'password' }, data):.
@PM2Ring should I add a self value even if I don't use it?
You can do that, if you really want to. And if so, the usual way to make it work is to use the @staticmethod decorator on the definition of validate. See docs.python.org/3/library/functions.html#staticmethod
it is a method of a class from which I am inheriting
@Aurelius Normal instance methods must have a self arg as the 1st arg. However, it can be named anything (even schema), the name self isn't magic, unlike this in some other languages.
  def validate(self, schema, data):
    return schema == set(data) and all(val != '' for val in data.values())
12:28
That's fine.
thank you very much for letting me know
The other option is to just define it as a plain function, outside of any class, since it doesn't need or use self.
That way, validate can easily be used by anything that needs to perform that sort of validation.
But it's ok to define it inside the class, if it"s only ever used by that class (or its children).
Some people will insist that such methods be created as staticmethods, to make it clear what's going on, but in Python it's fairly common to be a bit sloppy and just chuck in the self to make it a normal method.
It does make sense to bundle everything related to a class inside the class itself. But unlike some other OOP languages, Python doesn't force you to do that. You can even add methods to a class after the class is defined. You can even add new methods to instances, if you really want to.
Thank you Kevin. This is much faster than my previous code but is it possible that I have in results LNL68NLttN8Lh4NLh I need all combination from symbols but with some limits — Pijes 17 hours ago
Oh, look, he wanted the cartesian product with limits all along. Can I guess them or what?
@Aurelius No worries. Without that self arg, you'd get some pretty strange error messages when you try to run the code.
Your magic ball is well tuned, Kevin.
spooky-Friday-13th-And-Lunar-New-Moon-cbg \o
12:43
What other types of new moons are there? Plus it's full :P
cbg though
ah, explains it all :D
how are things, Moo?
thinking about finding a new job, but other than that it's nice. weather getting cooler which is good. what about you?
12:45
A Solar New Moon is when the moon isn't visible because the sun has moved directly between it and the earth. Very very rare.
@MooingRawr trying to keep my cold at sublethal levels, otherwise all's fine, thanks :)
@Kevin as beautiful as taht sounds, I would be deathly afraid of the consequences. unless some how we yeet the moon out of our orbit and into the sun's orbit...
hold on
:-)
@Kevin I wonder if that'd have freaked out our ancestors more than eclipses or not :)
12:47
:P
do you guys know what are these "ports"?
@Kevin I knew your spidey senses were tingling after his response to:
19 hours ago, by Kevin
I agree, but it wouldn't be the first time that we've been given logically contradictory requirements ;-)
@Aurelius sounds like a CPU instruction that works on multiple registers at the same time or something like that
> Figure 1. The AVX-512 VNNI VPDPBUSD instruction multiplies 64 signed 8-bit with 64 unsigned 8-bit values and accumulates to 16 signed 32-bit values per clock cycle per FMA (Intel® Xeon® processors 6000 series and above have two FMAs per core). Credit: Israel Hirsh.
@Kevin oh dear. I guess I'll stick with my existing heuristic then!
12:50
If the earth and its inhabitants were made out of an unmeltable material, I don't think a Solar New Moon would be particularly more interesting than any other sun-moon-earth configuration where the sun is that close to earth. I'm guessing the entire dome of the sky would be completely illuminated, so you wouldn't be able to tell where the Moon is at all.
Ah, you asked specifically about the ports. No idea.
@Kevin google tells me the Moon is 384k km away and the Sun is 1400k km across
A bit of a tight squeeze, I admit.
@OldTinfoil since the GIL is held/managed in C code, either the library you use properly releases the GIL or not. you cannot inject GIL releases into a library without recompiling it. That's kind of the point of the GIL.
@Kevin Just a bit toasty at that distance. ;)
though I am surprised to see it fail on a library that explicitly supports various thread and async methods. could be that you should choose another concurrency approach - they list support for several. if it is the DB driver that is non-concurrent, I don't think it is realistic to circumvent that.
12:54
Since Hypothetical Earth is made out of unmeltium, it should be able to survive inside the sun's radiative zone. unmeltium also has zero friction so the plasma atmosphere won't drag the earth's orbit down.
I'm a bit worried about the Lunar orbit though

« first day (3254 days earlier)      last day (1919 days later) »