« first day (4266 days earlier)      last day (699 days later) » 

12:44 AM
Since it was agreed stackoverflow.com/questions/72682119/… was not a dupe I threw in a rather longwinded answer.
but I am going to be slightly annoyed if we have all these metaclass driven questions that effectively boil down to this very explanation. the other question I linked could also benefit from this explanation...
 
1:38 AM
CBG
How to get the screenshot url/path on pytes report html?
what If I want to send the screenshot url to actual report? — Avinash Raj 38 mins ago
 
 
2 hours later…
3:38 AM
https://stackoverflow.com/questions/36901

I kinda wish it were still possible to "merge" questions. Although it would probably be good enough to just update the question, and expand the title to include "and arguments". Possibly edit some answers to reflect the new question properly; but many of the answers already go off on tangents (i.e., about using `*` and `**` for arguments) that weren't originally prompted.
@Aran-Fey 100% agreed, FWIW.
 
Agreed with both points above ^
 
 
3 hours later…
6:53 AM
For productivity when programming, which mouse and keyboard would you suggest?
 
Whichever is comfortable for you.
Programming isn't the same as typing/clicking.
 
I haven't used many different mouses and keyboards, so I don't think I know what suits me :P
You are right though, that there isn't that much typing and clicking. But still there is some. I was wondering if there are published papers on that.
 
7:15 AM
no, but theres a bunch of youtube videos or recommendation lists, none of which will completely agree with each other because this is ultimately subjective more so than objective.
 
7:54 AM
I want to ask about whether my processor is good for development tasks, for lets say will it be fine for 2-3 years from now or will it be outdated
Intel(R) Core(TM) i5-8250U CPU @ 1.60GHz 1.80 GHz
I am asking so that I upgrade this system (memory etc) or otherwise if it is not a good processor then will buy another computer
 
That's 5 years old as far as I can tell. I'd say it's outdated already.
Depends on your budget, of course. If you're not doing compute expensive things then it's still fine.
 
Will basically use for personal coding, like web development or something similar
nothing related to deep learning or graphic designing
 
8:18 AM
depends on budget. if you have the budget, get something that's from current generation.
 
actually I do stuff in intelliJ or other IDE and it is slow so will upgrading memory solve my problems or I will have to go for new generation processrs
 
If you want to change anything, I'd swap out the entire machine. While your current one is still useable it's already very dated; every investment you make into it will be a short-term benefit only.
 
8:37 AM
I'd expect HDD vs SSD to be more important in many cases. Considering Python doesn't compile the way C does.
But I've never had an SSD :P
 
Wait, for real?
 
you dont have ssd really!
 
8:58 AM
okay remember how I complained about the := walrus operator? I finally found a fun way to abuse it
from enum import Enum, EnumMeta
Color = EnumMeta('Color', (Enum,), (_ := EnumMeta.__prepare__('Color', (Enum,),)) and any(map(_.__setitem__, *(zip(*{'RED': 1}.items())))) or _)
rather, where I needed it to make that grotesque one-liner work
(details, see the thread that was not a dupe I linked above)
 
*(zip(*{'RED': 1}.items())) oO
Why not just ["RED"], [1]?
 
readability, people might want to throw in more values and it gets progressively more annoying if changes are needed later, so I added a couple more colours because I realized people will complain
mostly, to make it feel like people can just supply a normal looking dict at the thing, haha
like, people shouldn't write classes this way unless they have a damn good reason, but hey
here we are.
anyway, this was disgusting, I am going to take a shower
 
9:24 AM
@MisterMiyagi I'm just a poor boy from a poor family eastern bloc country
If I get a 128/256 GB SSD or a 1 TB HDD I'll always opt for the latter...
right now I've got 740 GB of stuff in my $HOME
 
cbg
 
@AndrasDeak--СлаваУкраїні That's a good point.
I kinda got used to having 250GB SSD because I barely put anything on my machine to begin with.
 
10:02 AM
@AndrasDeak--СлаваУкраїні 1 TB HDDs are usually awful value for money though
You'll pay less per TB if you get a 10-14 TB drive
 
@AndrasDeak--СлаваУкраїні now I'm going to have Bohemian Rhapsody in my head all day... thanks :p
 
@JonClements my pleasure :P
@vaultah perhaps, but I don't need more than 1 or 2 TB. Even if it has a larger relative overhead, I still pay substantially less for what I need.
plus the largest I can find right now in 2.5" size is 5 TB :P
But yeah, 1.779 units per data for the 1 TB drive, 1.116 for the 5 TB drive.
 
Ah you're on a laptop
 
objectively it still costs 3x more
 
I love SSD... think the default would be that for OS and swapfiles and traditional for any heavy storage though
 
10:13 AM
I once bought a 2 TB drive because I didn't think I needed more, filled it completely in a few months, purchased a 14 TB drive on sale for 3.5x the price of the 2 TB drive. Grrrr
 
@MisterMiyagi that being said I've wanted a 1 TB SSD for several years now... I'd probably even consider going for it if I weren't unhappy with my current newish laptop.
 
(although I do have a 50tb raid5 nas array...)
 
@AndrasDeak--СлаваУкраїні If you're happy with your rig otherwise, swapping to SSD probably ain't worth it. It makes some things much smoother but in many areas RAM will handle most of the recurring cases.
We've got pretty obscure hardware policies at our sciency place as well, so micro-optimising just ain't no fun.
 
@AndrasDeak--СлаваУкраїні Queen is certainly on the playlist now... Don't Stop Me Now... haven't listened to that one in ages...
 
@MisterMiyagi not sure what it is about my system but often I notice things (e.g. importing a large library after having switched back and forth between commits and virtualenvs, probably harming caching) lagging and having to wait 10 seconds with 400% iowait. My assumption is that these would go away with an SSD.
also firefox doing the same thing, it seems to do a lot of IO
often it's just too many things going into swap at which point I can just restart firefox and flush the swap...
 
10:31 AM
You can try moving Firefox profile and cache to RAM with profile-sync-daemon to minimize disk writes
 
It's already eating up all my RAM as it is :D Thanks for the suggestion though, I'll come back to this if things get real annoying or if I get 32+ GB of RAM.
 
I've never seen firefox become sluggish. It takes forever to start, but once it's running no amount of swapping will slow it down
 
Do you also have a lot of tabs open?
And do you also have an HDD?
 
I do have a HDD, but I think we're gonna have different ideas about the meaning of "lots of tabs"...
I have like... half a dozen
 
10:39 AM
Wait, are you on linux? You're on linux, aren't you
 
@Aran-Fey lots? :P
 
I only have 656 tabs open right now... got it down from 666 the other day.
2.5 GB RAM to open up firefox, +1.5 GB to refresh the about dozen tabs I actually often look at.
 
In my experience linux is terrible at swapping, so disregard everything I said. And the only reason why I have experience with swapping on linux is because it's terrible and happens when it shouldn't
 
yeah, I'm not happy with it
I've considered lowering swappiness, or just disabling swap altogether...
 
10:45 AM
umm... generally linux does a recently decent job of swapping
(tends less to hot potato like windows sometimes can)
 
Hold on, I think my laptop could house an SSD in addition to my HDD.
Guess I could buy a smaller SSD to put my system on... not sure about swap. Doesn't that strain an SSD too much?
 
@MisterMiyagi No, lots would be somewhere around 10 :P
 
@AndrasDeak--СлаваУкраїні errr... you want swap to be on an SSD
 
For the UX, sure, but the question is if that will kill the SSD sooner than I'd like
 
I managed to snag 64GB RAM back when there was a glut and I disabled swap and mount /tmp as tmpfs
 
10:50 AM
@AndrasDeak--СлаваУкраїні Nah, no chance you'll wear out an SSD yourself.
 
@MisterMiyagi but "swap" is hardly "myself", is it?
 
You would have to hammer one constantly for years.
 
which is what I'd expect swap to do :P
 
unless it's a really yammy SSD... they're generally better than HDD (eg - not moving parts even if you thrash the disk)
 
@AndrasDeak--СлаваУкраїні It's still going to be limited by your actions in the system.
 
10:51 AM
@AndrasDeak--СлаваУкраїні how long do you need it to last?
 
There's just not enough automation going on on a personal machine to seriously harm an SSD.
 
@vaultah 5 years, say?
@MisterMiyagi that's reassuring, thanks
 
Doesn't really matter if your disk is going to break in 10, 15, 20 or even more years when it's completely obsolete after 5.
 
yup
and this new laptop isn't exactly sturdy... I ditched the previous one after 4 years or so when it was again falling apart after 2 base cover replacements
this new one had a screen hinge break off after 6 months :/
 
@AndrasDeak--СлаваУкраїні 5 years is easily doable. Depends on the TBW rating, but any modern SSD should be alright after 5 years.
 
I'd get an M.2 SSD though, as you probably won't find many laptops with 2.5" slots in the future
If it helps, I have just 15 TB written on my SSD after one and a half years
And my disk is rated for 750 TBW
 
@vaultah that's exactly how it would fit beside my 1 TB HDD
thanks for the tips
 
11:29 AM
Hey all, I have a (hopefully) quick question about some code that I'm working on. If it should be a whole question on SO just let me know and I'll post it there, but I thought I'd start here first. Here is the code: pastebin.com/DVKthgEH
 
Your code only ever creates a single lists and appends to that.
 
I figured that was the problem, what is actually causing it?
oooohhhhh
 
Create a new list in generate_stream; you shouldn't use self.stream at all, and in fact I'd say you shouldn't use a class to contain generate_stream and generate_trial to begin with.
 
Yeah, why is that even a class? The only thing you use the class for is to store state (self.stream), but that's exactly what you don't want...
 
well the class has a lot more methods and more general purpose than what's in the MRE
i just removed the parts that i knew weren't causing the problem
 
11:35 AM
As a sidenote, you generally don't need to define __init__ on an @dataclass.
 
I had a specific reason for defining my own __init__ that's not clear from the MRE, but thank you for the tip :)
 
Thanks for building an MRE. ;)
 
 
2 hours later…
1:19 PM
Cbg
 
 
2 hours later…
3:23 PM
How do I use python 3.10 on Pycharm? Do i have to install it first in my main environment?
 
You have to install it somewhere. PyCharm expects a path to the interpreter for most types of environment.
A Conda environment will apparently pull in the required interpreter on demand.
 
Can i install it in the project's venv?
 
Usually, you install it to the system and then tell PyCharm to create that ven in the project.
 
Won't that interfere with my previous python installation?
 
interfere no, but having multiple python installs means you will need to keep track of what's running where. having said that, miyagi is spot on, python 3.10 would be a separate install from any previous python versions, and how your IDE/editor interacts with python is essentially a separate, independent concern
venvs are for versioning packages and making environments "on top" of an existing python install, so a venv itself cannot contain a different python version.
(and also yes, conda environments are different and capable of managing python installs itself too. thats primarily because conda environments are a lot more than just a venv, it's essentially an environment for arbitrary binaries as well. let's you manage more than just python. if you dont use conda however, dont worry about it)
 
3:44 PM
Thanks all.
 
4:27 PM
usually the answer to anything involving numpy and loops is: stop using loops. do it in a vectorized fashion. So, question to you then: can this operation be done in a vectorized fashion? (ps. i dont know the answer)
 
Indeed. I am struggling to see how to do it in a vectorized form. Basically I want to insert a 2x2 matrix at specific positions of an MxM matrix. I also see that my latest edit got rid of the code formatting, sorry for that.
 
If you like, you could try to post the formatted version again, and I'll whisk the old version away
 
use code pasting services, best way to share codes. if youre formatting code in chat, make sure not to mix text and code in the same chat block, they dont play nice together.
 
FWIW, if you want to count up an index in a loop use for i, phi in enumerate(phis):, but since you shouldn't be looping in the first place...
There's a sandbox if you want to play around.
It's linked in the formatting guide.
 
Yes, that is a good idea. Sorry.
M = 300
g = 0.01
EZ = 0.1
EX = 0.2
ham = np.ones((M,M))
s0 = np.eye(2)
ham = np.kron(ham, s0)
phis = np.linspace(0, 2*np.pi, 200)
i = 0
for phi in phis:
    ham_phi = np.array([[EZ/2, EX/2],
                        [EX/2, -EZ]])*(1+g*np.cos(phi))
    array_phi = np.zeros(M)
    array_phi[i] = 1
    mat_phi = np.diag(array_phi)
    ham += np.kron(mat_phi, ham_phi)
    i += 1
Oh. Let's say g = 0.01.
 
what's g?
 
Of course one could start with a 2M by 2M matrix, but there is a separate routine that results in the M by M matrix, so then I would have to think of a way to combine its construction with this.
 
it seems to me this is possible, but since i dont understand any of the operations here, i wanted to understand, the final output array we're interested in is ham?
 
@user129412 you're only ever using the diagonal of array_phi, it seems unnecessary. What's the closed-form value of mat_phi?
 
@ParitoshSingh yes, we are interested in ham.
I actually didn't plan this very well, I need to leave now. Thanks for your thoughts and help. I'll ponder on this a bit more.
 
4:40 PM
Drop by anytime :-)
 
@ParitoshSingh There is also pyenv
 
hm, i must admit ive never really used that, never needed to
 
Not to say that it's any better than conda. At work, our workspaces are Docker images with pre-installed python so you're a bit stuck with what you can do (I've never tried conda in one). If Engineering decide to bump the python version, though, and you need to build a new workspace, you wouldn't otherwise have a choice whether or not that suddenly breaks all your code overnight
 
@user129412 Maybe I am getting this wrong, but mat_phi will basically have one single element of 1 (at i,i?) and so np.kron basically just selects that element from ham_phi. Is that correct?
 
seems correct to me
 
5:05 PM
@roganjosh what do you use for package management and dependencies, if not conda?
 
Just plain pip
 
@MisterMiyagi That's what I'm saying, it seems unnecessary constructing a diagonal matrix and then only using its diagonal. That's just an obfuscated scalar.
 
That's true. I wasn't sure how to insert the matrix at the desired position otherwise. But I'm probably just not thinking about it in the right way.
 
5:22 PM
it may help to think of it in terms of the big picture.
...which clearly im incapable of doing. i still dont understand the kron operation after reading the np page twice :(
 
@ParitoshSingh it's just a tensor product :P
build a block matrix by multiplying a matrix which element of another matrix in turn
Block (i, j) is a_{ij} * B
 
6:29 PM
Yes, indeed. The array_phi part is to say where in the (2Mx2M) matrix the 2x2 submatrix is to be added.
Essentially, starting from an MxM matrix, I want to go to a (2Mx2M) matrix where a 2x2 matrix is added on 'the diagonal' (which is now a 2x2 block). And that 2x2 matrix depends on phi.

If I knew how to easily generate a 2Mx2M version of ham_phi that satisfies this, I am probably done..
 
Well, i can share what i've got so far, but fair warning, its bad. its fast, but its making me grimace. I presume theres a better way to do this.
 
If it helps me bring down my fitting routine from lasting 4 days to something more bearable, I'll be happy for it to be bad
 
I cant think of a cleaner way to align the arrays to the correct spots, but also i am willing to bet one such way exists that's better than what i did here
 
Well, other than that I misled you with a mistake, that seems quite alright! I put M = 300 up top, but then later hardcoded M=200 in some spots (which is where you put 200).
 
however, getting rid of the loop is clearly important enough that even this method gets huge time savings
 
6:35 PM
Yes, a factor of 100, that is huge :)
Do you think that, appropriately phrased, this could warrant a question on the main site? Given my trouble in describing what the code does, it seems hard to make it useful for others.
 
generally, if your code works and you want to make it faster, codereview is a better place for it. usual caveats about reading "good question faqs" for their site applies.
also, i feel like even if the answer could help others, questions like these wont really reach those looking for help. "how do i make this work faster" is just too vast of a question, even if this specific answer applies in this specific scenario
 
That makes sense
 
(having said that, ive seen that one user really do a number on these numpy related questions. cant believe im blanking out on the name.)
the fact that i cant find this user is bothering me
 
I've now found out that what I am trying to construct (ham_phi) is called a banded matrix. Maybe that will help in my search for a solution. But what you came up with probably already does it, in terms of speed, so thanks a lot
 
6:48 PM
https://stackoverflow.com/questions/5082452

Where are all the questions that actually *explain how to use individual formatting techniques*, as opposed to comparing them? I hate that when I try to look up `str.format` there's literally just this and then a million pieces of junk
 
Also, found them! it was divakar. really good answers from them that may be worth reading in your spare time. link
Their answers may help you start thinking creatively about how you can vectorize things in general, even ones that may seem tough or impossible at first.
 
because as "canonical" as that question is, it does not adequately or directly answer stuff like stackoverflow.com/questions/72705410/…
 
i just realised maybe kron is the answer again, i just need to understand it better...
 
I've got a question about threading vs python's asyncio library. While using an ASGI server like uvicorn for example, let's say I have a view that looks like this:
@router.get("example_endpoint")
async def example_endpoint():
    do_threaded_ledslib_call(callback_function, function_name, etc.)
    # The results of the threaded call can only be accessed in the callback function, so I can't return the info that a user is asking for in this request
    return {"results": "???"}
I was talking to another engineer I work with and he was saying that we need to add an await call so that we can get the results of the do_threaded_ledslib_call function (which would use a thread) but I don't think that is possible
 
7:05 PM
@user129412 take a look at this as well. it's apparently not quite as fast as hardcoding it, but yeah
 
@pythonweb well, you need an await "call". If that's not possible, we should talk about why do_threaded_ledslib_call doesn't make it possible.
 
Interesting!
 
google fu to the rescue, looks like scipy already has a function for this kind of diagonal work (docs.scipy.org/doc/scipy/reference/generated/…), now that i have a better sense of what im doing. which leads to another significant speedup
going to stop here, last version from me. paste
 
@user129412 I can also take a look soon; I've been busy
 
7:15 PM
@MisterMiyagi So I think I phrased my question poorly. What is happing is that there are functions that do things in a synchronous manner in a library that was created. The main thing the synchronous code does is make a lot of database calls using the psycopg2 library at it's core.
So this other engineer is asking if we could somehow make the synchronous code run asynchronously on the ASGI server and then use the await syntax on them, but without actually changing the functions (for example making them async functions)
@Kevin This seems like a good place to start, although would running synchronous code in a thread give better performance?
 
Avv
@Hello Guys,

In case I have a project written in Python, what methodology you use to build architecture for it please?
 
7:37 PM
@pythonweb If you don't want to change things, then wrapping things in asyncio's threads is your only option.
 
@Avv That's a pretty vague question. Is the project a program or a library (or both)? And what exactly do you mean by "methodology"? Are you talking about things like TDD (Test-Driven Design)?
 
That's a huge waste, so it's only worth it if the bulk of the code is other, proper async operations.
 
@MisterMiyagi What do you mean by huge waste
 
The point of async is not to use threads. So heavy use of threads to make sync code awaitable defeats the point.
async functions can task-switch quickly and you can easily have tens of thousands of them. thread-switching is costly and you can only have dozens to hundreds, depending on the HW.
 
@MisterMiyagi HW?
 
7:46 PM
Hardware
 
@MisterMiyagi Ok, yeah that makes total sense to me. I figured that there must be some sort of limitation of threading otherwise there wouldn't be a need for libraries like aiohttp, etc.
 
@user129412 nevemind, you've asked already and got an answer
 
8:09 PM
how can i pass all the values of a dict list that have similiar keyword? im trying doing it like this but its not working
 
@AndrasDeak--СлаваУкраїні thanks for thinking along!
 
x = population['x']
y = population['y']
 
What's a "dict list"?
And how does it "not work"?
 
And when you say "pass", what does that mean? Pass arguments to a function? Create variables?
 
i have these numbers
[{'x': 1.909607353244957e-98, 'y': 0.29004955192672}, {'x': 7.763465128880745e-95, 'y': -0.618405622641303}, {'x': 4.947537989226403e-96, 'y': 1.5193977696382022}, {'x': 7.76138782636963e-95, 'y': -0.6168801123926879}, {'x': 7.809585788134308e-95, 'y': -0.619067362698118}, {'x': 7.876673797816167e-95, 'y': -0.5650046701120272}, {'x': 7.763465128880745e-95, 'y': -0.618405622641303}, {'x': 4.9609434130360905e-96, 'y': 1.5206757431562428}, {'x': 4.876830378021896e-96, 'y': 0.7245509877253996}]
and i want to pass every value 'x' to a list x
 
8:13 PM
So that's a list of dicts.
x = [dct['x'] for dct in list_of_dicts]
 
ty it worked
 
9:00 PM
Thanks in advance
 
@Marco just to be sure: did you do what I asked, restarting your kernel and checking that executing every cell from top to bottom still reproduces your issue?
 
Yes, of course
 
df = pd.read_csv('df.csv') <- I get an error here
 
Are you executing as an IPython Notebook?
The first line downloads the df
!gdown 1OF-HqkedSqUWcbPoqCKLfLBgyB8BAHZZ #download df
 
9:04 PM
Good
 
I doubt that will work anywhere outside google colab
 
Yeah, maybe
 
How large are these example CSV files? Approximate number of rows and columns.
 
hi can someone help me with this stupid json error?
 
@Ocryol no, because we ask not to ask for help here with fresh questions on the main site, see our rules
 
9:10 PM
Alright, I'm sorry
 
@AndrasDeak--СлаваУкраїні The first df has 3635 rows and 19 columns
But it doesn't take 1 second to download each one, and they are only stored on Google Colab server
 
9:35 PM
Strange that the annotation is not being plotted, right?
 
10:11 PM
@Marco no. Your non-MCVE says that the ylim for ax2 is (6.385, 15.515) but you're trying to annotate point (2, 2).
 
I can't believe I didn't pay attention to it
Sorry and thanks
About the "non-MCVE" is because of the csv's files sizes (rows and columns)?
 
10:35 PM
im getting this error in my code:
new_worst_individual = population[9]
IndexError: list index out of range

how can i fix it ?
my full code: pastebin.com/T2cvENp7
 
@Thorsen don't index a list with length less than 10 with index 9
if you check len(population) right before that line you'll get a number less than 10
@Marco 1. because there are csv-s that would have to be fetched from a random google drive, 2. because there's a lot of code there that doesn't have anything to do with the issue.
 
sorry didn't get it
 
so i need to do size = len(population) and then new_worst_individual = population[size]?
 
@Thorsen absolutely not
I said "check" len(population). By e.g. writing print(len(population)) before the line that raises the error.
you will see a number, e.g. 9, which will be less than 10
 
10:44 PM
got it
 

« first day (4266 days earlier)      last day (699 days later) »